Field simulator - SCADA - DMS - SQL Server

Today I have achieved very important goal - I have managed to setup almost live environment in VirtualMachine.
I have used xMaster as a field simulator. OASyS SCADA is set to accept the changes which are coming from the field (from xMaster). Schneider Electric DMS is connected to OASyS SCADA, and DMS is writing the changes which are coming from the field (and through all these components) to SQL Server.

Here is a picture of components:

BCP is also influenced by health of indexes and valid statistics

I have been running some bcp out statements, something like this:

bcp "select * from DatabaseName..TableName where Timestamp>='2013-12-23 00:00:00' and Timestamp<'2013-12-24 00:00:00'" queryout
C:\temp\TableName_2013-12-23.dat -S SQLServerMachineName\DatabaseInstanceName -T -n

I have noticed that for some days I am getting slower output, then for another days.

Since the TableName is partitioned by Timestamp, I did the index rebuild on partitions where "slow" dates exists. But it did not solve my problem. BCP was still slow.

Then I asked myself - why BCP is still slow? 

The answer came to my mind when I noticed that bcp is using SQL statement. Then I knew that the problem must be in that statement. Then I took that SQL statement, generate a query execution plan for a day when bcp is ok, and for a day when bcp is slow, and I noticed that they have different execution plans. 

The reason why they have the different execution plans is that statistics for indexes on TableName were wrong for the dates which have slow bcp. I thought that partitioned index rebuild will automatically update statistics, but it did not update it. So, I run update statistics manually, for all the indexes on that table.

After that, bcp was fast for all the days.

All maintenance actions on one place

I have manually started index rebuild on one system and noticed that it goes slower that I would expect. I issued sp_who2 to see what is happening and noticed that some other application is doing backup of the same database I am rebuilding the index. I got frustrated with that fact.
And how would I even know that this backup is scheduled, when this application has its own schedule which is not present in SQL Server, but in this application.
So, I have been thinking that all maintenance should be done in SQL Server. Or, that SQL Server is aware of that maintenance, and if someone is issuing something that SQL Server is not aware then SQL Server would, for example, forbid this action. So, maybe one way of protecting SQL Server (and other systems) would be that all the actions (all or some of them) should be pre-announced.

Good strategy for entering smart grid market

By reading Landis & Gyr FutureReady newspaper, I come to the idea that they have found a strategy for entering smart grim market.

From this newspaper "in the US 3000 municipal and electric cooperatives deliver power to almost one-third of the nation's total population".

It means that these "small" power distribution companies can have a huge benefit from using cloud based smart grid services (as also suggested by Landis & Gyr). Instead of having an army of different types of professionals which are needed when power distribution company has its own solution (which is maybe ok for bigger players), small utility usually is not able to support in-house solution(usually expensive). So, cloud based solution would be very good strategy for attracting this kind of clients - they can choose limited functionality to test and try what this software can give them, to expand their choice easily (by having in-house solution, obtaining and installing new functionality would increase costs significantly, because it again requires different types of actions to be done. In cloud they would just subscribe to another function and start using it, in the beginning maybe as a trial version, and if they found is useful, they can use it permanently.

And if they are not satisfied they can abandon the solution without too much costs.  

This cloud based solution would be also beneficial for the solution providers (from the technical aspect), and it would reduce the costs and in that way more utilities would install and use the smart grid solutions.

How cloud based databases could be used in smart grid

I have finally found some good usages of cloud databases - storing some non-critical data. Instead of storing it in Your local database (SQL Server, Oracle), I think that the good place to store some non - critical data are cloud. In case that something happened to these data - the damage will not be critical, and You will save lots of resources - storage, machines, OS, licences, and all the things needed for having local database.
As an example of data (from smart grid field) which can be stored in the cloud are MDM data.
By reading some news lately, I have concluded that there are several utilities which  are using the cloud. With MDM data. But, maybe, this is just a shy first step, and that in the feature more things could be pushed into cloud.

I have also been experimenting with Windows Azure SQL database. Good starting point to try it by Yourself is here http://www.windowsazure.com/en-us/documentation/articles/sql-database-get-started/

HA DR Oracle and SQL Server solutions

From version 2012, SQL Server has a HA DR solutions, similar to Oracle (Active) Data Guard. Here is some (I would say Microsoft sponsored - oriented) comparison of these two solutions.

http://joedantoni.wordpress.com/2012/03/29/alwayson-versus-active-data-guard-a-value-proposition/

Also, one of the articles which describes complexity of Oracle RAC installation (team I have been working in has done it with Oracle 10g  - so I could say that is correctly written - installing RAC on 10g was a nightmare) is

http://download.microsoft.com/download/C/D/C/CDCA80B2-FD63-49C5-AC66-8E3A179E31F7/SQL_Server_2012_Why_Oracle_RAC_Might_Not_Be_Suitable_For_You_Apr2012.pdf

and inside that document there is a link to the Oracle RAC installation overview

http://www.linxcel.co.uk/Whitepapers/Real-World%20Challenges%20for%20Oracle%20RAC%20Implementation-Issue%201.pdf

I am not preaching not for SQL Server nor for Oracle. Although these two documents are describing complexity of Oracle RAC installation and usage, if we are looking on the Oracle RAC not only as a DR solution but also as a platform with which You can scale up, than we could tell that Microsoft does not have that solution. 

Smart grid test lab

http://tdworld.com/grid-opt-smart-grid/inside-pges-smart-grid-lab

This is the place where I would like to see our DMS software testing...Built in an American style. Being in USA several times showed me that boundaries of "possible" are not so strict - so I have a feeling that in USA - everything is possible. USA have something which other countries does not have - organization, hard work, vision. They are also believers in that vision. It is quite hard to define all the things which led to that culture of achievement.