Increased SCADA System Reliability with a Redundant Server

The SCADA Server is a vital component of the Water Utility – providing 24×365 critical services in the areas of realtime monitoring, historical trending, alarming, remote access, and supervisory control.

It is noteworthy that Navionics Research’s SCADA Server design is not based upon a master/slave architecture. Therefore a loss of the Server will not cause a loss of automatic control operations throughout the filtration plants, pumping stations, water towers, and other sites. However, if the SCADA Server were to suffer a catastrophic failure, the Water Utility would need to resort to human oversight in order to maintain equivalent vigilance over the system’s operation.

For large Water Utilities, human oversight would generally be very difficult; as the number of sites that require monitoring – and the distance between them – could be substantial. A SCADA Server provides monitoring and alarming around the clock; and therefore the diversion of personnel from their normal duties and the extra overtime work to provide equivalent oversight would be costly to the District, as well.

In order to minimize the negative effects of a catastrophic failure to a SCADA Server, Navionics Research recommends that large Water Utilities purchase and store a Redundant, Cold-Swappable SCADA Server which would be ready for quick deployment in case of failure of the primary SCADA Server.

Preparation. The Redundant SCADA Server should be thoroughly prepared in advance so that the switchover can be performed quickly and easily.  The procedure should basically consist of switching over the signal and power cables from the Primary Server to the Redundant Server.

 

Hot-Swappable Versus Cold-Swappable: A hot-swappable redundant server refers to an extra server that is always powered-up and automatically activates if the primary server is lost for any reason. A cold-swappable redundant server refers to an extra server (disconnected from power and stored in a safe location) that requires manual installation by Water Utility personnel. Because certain conditions that could cause loss of the primary Server may also simultaneously damage the redundant hot-swappable Server (e.g. power surge), we recommend a cold-swappable Redundant Server.

Navionics Research has deployed redundant SCADA Servers, and concurrently developed documentation that details the steps required to build, test, store, and deploy a redundant, cold-swappable SCADA Server.  The procedures are briefly summarized in this article:

Construction and Configuration:

1. Operating System Installation. The Operating System (e.g. Microsoft Windows 7 Professional) should be installed onto the target machine.
2. Device Driver Installation. The motherboard Device Drivers should be installed onto the target machine.
3. Peripherals Installation. Extra peripheral devices should be installed onto the target machine. Examples include USB-Serial Adapters, Graphics Adapters, etc.
4. Operating System Updates. The Operating System should be updated to include all available patches. For example, Microsoft Windows Update.
5. User Account Setup. The required User Account(s) should be configured on the target machine.
6. SCADA Software Installation. The SCADA Software should be installed onto the target machine.
7. SCADA Software Configuration. The client-specific configuration files should be installed onto the target machine. These files should be identical to those installed on the primary machine.
8. Network Configuration. The client-specific network settings should be programmed into the target machine. These settings should be an exact match to the primary machine.
9. TeamViewer Installation and Configuration. TeamViewer should be installed and configured on the target machine. This will allow remote management of the system immediately after deployment.

Testing:

1. The Redundant SCADA Server should be placed into operation to verify that the redundant machine will provide identical service as the primary machine.

Storage:

1. The Redundant SCADA Server should be stored in a sealed plastic bag, along with all peripherals. The Operating System CD/DVD and Device Driver CD/DVD should be included.

Deployment Procedures:

1. The Redundant SCADA Server should be retrieved from its sealed plastic bag and placed alongside the damaged primary SCADA Server.
2. Power to the Damaged Primary Server should be turned OFF; and power to the Monitor(s) should also be turned OFF. The Redundant Server should not yet be powered up.
3. One-by-one, the following cables should be moved from the Primary Server to the Redundant Server:
   • Serial Cable(s)
   • Monitor Cable(s)
   • Network Cable
   • Mouse Cable (Unless Redundant Mouse provided)
   • Keyboard Cable (Unless Redundant Keyboard provided)
   • Power Cord
4. Turn ON power to the Display Monitors.
5. Turn ON power to the Redundant Server, and press the ON button on the front of the machine. Turn ON power to the Monitor(s).
6. The Redundant SCADA Server should boot up normally.
7. After the boot-up procedure completes, the monitor(s) should display the SCADA screens.
8. Initially, all remote sites should show communication talkback failures (flashing RED antenna icons). These should all clear out within about 15-20 minutes, as normal communications are re-established.

  

Redundant SCADA Server with Connected Peripherals and Software DVD’s.  Sealed Plastic Bagging Added For Protection During Storage.

 

Would a redundant SCADA Server benefit your Water Utility?  Give us a call; and we’ll be glad to discuss the details!

Telemetry, SCADA, & Controls Newsletter

Was this article helpful?  Would you be interested in receiving updates such as these in our occasional email-delivered newsletter?  If so, here’s our sign-up page:

Subscribe to Navionics Research’s “Telemetry, SCADA, & Controls Newsletter”