Warm Standby Redundancy

To provide redundancy, the network operator can maintain a secondary Legacy Orchestrator system that is ready to take charge if the primary Legacy Orchestrator system fails.

Legacy Orchestrator supports two redundancy methods:

• Warm Standby Redundancy (covered in this section). In a warm standby setup, the database is mirrored to the secondary system at regular intervals. Failover is initiated manually.
• Hot Standby Redundancy. In a hot standby setup, the database is replicated on the standby system continuously. Failover is triggered automatically. See Hot Standby Redundancy (Docker).

Introduction

This article explains how the Legacy Orchestrator Warm Standby Redundancy feature works and covers the requirements and limitations of the feature.

Initial Setup

The initial setup for Warm Standby Redundancy is as follows:

• The Legacy Orchestrator secondary site and the Legacy Orchestrator primary site now have the same version of Legacy Orchestrator software
• The primary site normally manages Skylight devices. It periodically sends a backup of its data store to the secondary site
• The secondary site is in standby mode, ready to replace the primary site in case of failure
• A cleanup task at midnight will delete backup files older than 7 days in the /home/skylight/backups/ directory
• The Docker host of the Legacy Orchestrator secondary site has enough disk space to store the backup files

The following figure shows the initial setup for warm standby.

Manual Failover

For warm standby, failing over to the secondary site requires human intervention. The failover scenario for warm standby is as follows:

• If the Legacy Orchestrator primary site fails, the operator manually activates the Legacy Orchestrator secondary site with the latest database from the primary site.
• When the failure on the Legacy Orchestrator primary site has been corrected, the operator can fall back to the Legacy Orchestrator primary site. After the fallback procedure, the Legacy Orchestrator primary site is back in charge of managing Skylight devices.

The following figure shows the failover scenario for warm standby. Failover is to the Legacy Orchestrator secondary site.

Recommendation

For a warm standby setup, we recommend increasing the number of allowed CLI sessions on Skylight devices to the maximum value of 5. This ensures the fastest possible switchover and avoids the need to wait for network element CLI sessions to time out if the primary server fails.

This change can be done on all Skylight devices using a configuration job or a configuration flow. For more information about configuration jobs and flows, see Commissioning.

Limitations

The limitations of a warm standby setup are the following:

• Failover requires manual intervention.
• The last backup of the primary site that is present on the secondary site is at least 30 minutes old. Any changes to the configuration of the primary site during the last 30-minute period may be lost if a failure occurs.
• If the primary site remains unrecoverable, some performance monitoring data may be lost in the un-transmitted CSV files that contain the data.