Upgrading Skylight orchestrator 23.12
  • 19 Dec 2023
  • 16 Minutes to read
  • Contributors
  • Dark
    Light
  • PDF

Upgrading Skylight orchestrator 23.12

  • Dark
    Light
  • PDF

Article summary

This Skylight orchestrator Upgrade article includes procedures that will help the personnel of mobile network operators, Cable Multi-Service Operators (MSOs), business service providers and cloud and data center services upgrade and maintain their performance assurance solutions.

Specifically, this article explains how to upgrade Skylight orchestrator to version 23.12 from any of these previously-installed versions:

  • Virtual machine running 22.10 5K profile
  • Virtual machine running 22.10 15K profile
  • Virtual machine running 22.10 30K profile
  • Virtual machine running 22.10 60K profile
  • Virtual machine running 23.04 5K profile
  • Virtual machine running 23.04 15K profile
  • Virtual machine running 23.04 30K profile
  • Virtual machine running 23.04 60K profile
  • Virtual machine running 23.04.1 5K profile
  • Virtual machine running 23.04.1 15K profile
  • Virtual machine running 23.04.1 30K profile
  • Virtual machine running 23.04.1 60K profile

Prior installation data and configuration information is retained during the upgrade process.


Note: Support for virtual machine-based deployments will be 2 years after General Availability as part of a Long-Term Support (LTS) program. Please ensure migration to Docker before that time.

Pre-Upgrade Requirements and Procedures

This section describes the requirements and procedures that must be followed before performing an upgrade.

The following sections cover the procedures for upgrading a KVM, or VMware machine running Skylight orchestrator 22.10, Skylight orchestrator 23.04 or Skylight orchestrator 23.04.1. During the upgrade, the system will be unavailable for approximately 90 minutes.

Before performing the upgrade procedures, you must read through the Release Notes for Skylight orchestrator 23.12.

If you are running in a redundant configuration, please run this procedure for each site of the redundant system.

IMPORTANT: You cannot upgrade hardware appliances to version 23.12 as they have been retired.

IMPORTANT: Skylight orchestrator will no longer support rehoming directly from the hardware or the virtual machine running in ESXi release prior to 6.5.


WARNING: If you are using Skylight Performance Analytics, the Roadrunner component must be upgraded to the latest release before upgrading to Skylight orchestrator 23.12. Please contact the Skylight Analytics support team to verify that your Roadrunner version is compatible with Skylight orchestrator 23.12.

Requirements

You must ensure you have the following items before starting the upgrade procedure. If you are missing anything, contact Accedian Technical Support.

  1. SSH client (such as PuTTY)
  2. SCP client (such as WinSCP)
  3. Username and password for the visionems account for all virtual machines running Skylight orchestrator software.
  4. The target machine must have 20 GiB of free space in its mysql partition.
  5. The target machine must have 6 GiB of free space in its root partition.
  6. The target machine must have 9.5 GiB of free space in its data partition.
  7. To upgrade to version 23.12, you must have access to the following package files:

    upgradeAppliance-SO_v23.12_11_package-prod.tar SO_v23.12_11.fsa


    Note: These files contain all required files and scripts for the upgrade.
  8. Check and ensure the current deployment profile matches with resource requirements (vCPU, RAM, disk). Please refer to the Deployment Profile Configuration and Virtual Machine Disk Configuration articles for additional information​.
  9. Check number of acceptable firmware loads, as outlined below:

Number of Acceptable Firmware Loads

Before upgrading or starting app_server on Skylight orchestrator, you must validate the number of acceptable firmware loads based on the specific profile of the system.

For example, if orchestrator is using a root partition of 28GB, it will switch to root partition 18GB after upgrading. In this case, you need to check that the number of firmware loads on the orchestrator, before upgrading, is not higher than the limitations shown below.


Note: When the requirements from the table below are met, the upgrade can be performed and the system will restart. After the system restarts, the partition can be checked. The partition must have at least 1 GB of free space remaining.

Skylight orchestrator ProfileAcceptable Firmware Loads
60K profile system4 sensor: control or 20 performance element loads
30K profile system8 sensor: control or 40 performance element loads
15K profile system16 sensor: control or 80 performance element loads
5K profile system24 sensor: control or 120 performance element loads​

Pre-Upgrade Procedures

This section explains the procedures that must be performed before upgrading to version 23.12.

The pre-upgrade procedures are:

  1. Ensuring the FTP export method is not used on Inventory Export and CSV Export
  2. Backing up the Data Store
  3. Stopping the Application
  4. Transferring Files to the Machine

Ensuring the FTP export method is not used on Inventory Export and CSV Export

From release 23.04.1, the FTP transfer method is not supported for Skylight orchestrator functions (inventory export and CSV export). Therefore, changing to another transfer method is required to ensure no corruption of the transfer file after upgrading.

The FTP transfer method MUST be changed to another supported transfer method (SFTP or Rsync) before upgrading if it is being used on any Skylight orchestrator functionality:

  1. In the inventory export method:
    a. Change to the SFTP method and ensure that it works correctly.
    b. Please refer to Exporting Inventory Reports for more information.
  2. In the CSV export method:
    a. Change to SFTP or SFTP/Key or Rsync method and ensure it works correctly.
    b. Please refer to Configuring Global Collection Settings for more information.

Backing up the Data Store

A database backup of the system is needed.

Back up the data store

  1. Open an SSH terminal session to Skylight orchestrator on port 22.

    Note: If you are running a redundant configuration, the back-up must be taken from the active site.
  2. Log in as the visionems user.
  3. Change to the scripts directory by entering:
cd /opt/accedian/skylight/scripts
  1. Create a backup of the data store by entering:
./ems_backup_mysql.sh -d

The backup is created and saved in the /opt/accedian/backups directory.

Stopping the Application

The application must be stopped before running the upgrade scripts.

Stopping the application for a Hot Standby configuration

Active Site

  1. Open an SSH terminal session to the Skylight orchestrator on port 2200.
  2. Log in as the visionems user.
  3. Stop the Hot Standby process by entering:
redundancy control stop
  1. Start mysql by entering:
service start mysql

Standby Site

  1. Open an SSH terminal session to the Skylight orchestrator on port 2200.
  2. Log in as the visionems user.
  3. Start mysql by entering:
service start mysql

Stopping the application for a Non-Hot Standby configuration

  1. Open an SSH terminal session to the Skylight orchestrator on port 2200.
  2. Log in as the visionems user.
  3. Stop the application by entering:
service stop app_server

Transferring Files to the Machine

This procedure explains how to install the required upgrade package files to the appropriate location.

The following upgrade files must be available:

  • upgradeAppliance-SO_v23.12_11_package-prod.tar
  • SO_v23.12_11.fsa

To transfer files

The procedure must be executed as the visionems user.

  1. From your computer, use an SCP client (such as WinSCP) and the visionems account to transfer the required files to the /data folder.

    CAUTION: The location of the required files must be /data.

Upgrading a Virtual Machine

This procedure contains instructions on how to upgrade to Skylight orchestrator 23.12. Upgrades are supported from the following releases:

  • Virtual machine running 22.10 5K profile
  • Virtual machine running 22.10 15K profile
  • Virtual machine running 22.10 30K profile
  • Virtual machine running 22.10 60K profile
  • Virtual machine running 23.04 5K profile
  • Virtual machine running 23.04 15K profile
  • Virtual machine running 23.04 30K profile
  • Virtual machine running 23.04 60K profile
  • Virtual machine running 23.04.1 5K profile
  • Virtual machine running 23.04.1 15K profile
  • Virtual machine running 23.04.1 30K profile
  • Virtual machine running 23.04.1 60K profile

This upgrade requires the following procedures to be performed:

  1. Upgrading to the New Software Version
  2. Setting the Deployment Profile


Note: If you are running a redundant configuration, this procedure must be executed on each virtual machine of the redundant pair.

Upgrading to the New Software Version

This procedure will install the upgraded software.

To upgrade to the new software

  1. Open an SSH terminal session to port 22 of the virtual appliance.
  2. Log in as the visionems user
cd /data
  1. Un-tar and change the permissions on the upgrade file by entering:
tar -xvf upgradeAppliance-SO_v23.12_11_package-prod.tar
rm upgradeAppliance-SO_v23.12_11_package-prod.tar
sudo chmod 770 ./upgradeAppliance

If prompted, provide the visionems user password.

  1. Execute the script that does the partition upgrade:
./upgradeAppliance -up /data/SO_v23.12_11.fsa


Notes:

Only the absolute path file will be supported; otherwise you will get a relative file error message. This step will apply a new OS to the inactive partition.

This archive is a new operating system that will host the Skylight orchestrator software. During execution, the script will prompt you to increase SSH timeout if timeout is less than 10 minutes.



This script will ask you to reboot system to take effect.
Testing the system configurations before upgrading:
Authentication key-based OK
Mysql partition status OK
Data partition status OK
Virtualization OK
Appliance connection OK
Hostfile aliases OK
Database status OK
Database version OK
Database upgrade compatibility OK
Extra hostfile aliases OK
IP address OK
System requirements OK
|INFO| System information:
Linux CoreKVMC 4.4.0-154-generic #181-Ubuntu SMP Mon Dec 18 05:29:03 UTC 2023
x86_64 x86_64 x86_64 GNU/Linux
Ram : 30882660 Kb
##################################
The system is ready to be upgraded
##################################
Extract to temporary folder

  1. When prompted enter:
y

SSH timeout is too low and need to be updated. Do you want to continue?
[y/N] y
Please log out to take affect and login back to upgrade.

  1. When prompted enter:
y

Skylight orchestrator appliance will be upgraded on inactive partition. Do you
want to proceed? [y/N] y

Applying upgrade...
Stopping mysql service
Mysql partition has been backup
Archive has been applied
UUID has been updated
Change owner folder backups
OS configuration backup taken
SSH key files backup taken
Grub configuration has been updated
Inactive partition mounted
Updating fstab file on the updated partition...
Inactive partition umounted

Copying active partition configuration to inactive partition
Backup file name is CONFIG_DUMP_2023-11-26_06:20:19.tar
Applying backup to inactive partition
Inactive partition mounted
Configuration copied to inactive partition
Copying Appmon backup directory...
Inactive partition unmounted

* Setting up grub configuration for inactive partition
Applying switch...
Inactive partition mounted
Copying fstab file....
Fstab file was copied
Inactive partition unmounted
Inactive partition mounted
Installing for i386-pc platform.
Installation finished. No error reported.
Boot grub loader configured
Grub configuration has been updated
Inactive partition unmounted
Appliance configured
Inactive partition is now configured as the boot partition

  1. When prompted enter:
y

A system reboot is required to take effect. Proceed with reboot? [y/N] y
Applying reboot…


Note: Wait for the reboot to complete.

CAUTION: During upgrade, previous visionems account password is not preserved; it has resorted back to default. When you re-log in on the new upgraded partition, use the default visionems password.

  1. Reopen an SSH terminal session to port 22 of the virtual appliance.
  2. Re-log in as the visionems user:
cd /data
  1. Run the following command:
./upgradeAppliance -u

Testing the system configurations before upgrading:

Authentication key-based OK
Mysql partition status OK
Data partition status OK
Virtualization OK
Appliance connection OK
System requirements OK

|INFO| System information:
Linux CoreKVMC 4.4.0-169-generic #198-Ubuntu SMP Mon Dec 18 10:38:00 UTC
2023 x86_64 x86_64 x86_64 GNU/Linux
Ram : 30882656 Kb

##################################
The system is ready to be upgraded
##################################
Extract to temporary folder
[sudo] password for visionems:
Cleaning installed packages
Installing the dependencies
Configuring node
Installing JVM
Installing drivers
Installing Ha
Installing Hamon
Installing Deployment tool
Installing mysql
Extract dbtool
Restore mysql partition
Archive has been applied
UUID has been updated for mysql partition
Successfully restored the Database
Upgrading the Database
Successfully upgraded the Database
Installing Skylight-orchestrator
Installing Ptexport
Installing appliance monitor
Appliance monitor correctly installed.
Skylight-driver correctly installed.
Skylight-jvm correctly installed.
Skylight-orchestrator correctly installed.
Skylight-mysql correctly installed.
Skylight-singlenode correctly installed.
skylight-ha correctly installed.
skylight-hamon correctly installed.
skylight-deploymenttool correctly installed.
Skylight orchestrator has successfully been upgraded to version 23.12

    Hardening OS for active partition
  Hardening active partition
  All done

Setting the Deployment Profile

To set the deployment profile

  1. Open an SSH terminal session to the Skylight orchestrator on port 2200.
  2. Log in as the visionems user.
  3. Start mysql by entering:
service start mysql


To reapply the deployment profile when upgrading from the Skylight orchestrator multiple mediation instances


Note: Skip this section if the deployment profile is a single mediation instance (5K profile).

  1. Display the current configuration by entering:
deployment show configuration

Example:
Skylight: deployment show configuration

[ Deployment configuration ]

  Deployment type : single-node
  Profile name    : 30K

  User profile name : user-20
 Max user session : 20

  Cluster configuration :
   Cluster type       : medcluster
   Node Configuration :
Node name       : medn1
Total instances : 6

  1. Assign a profile by entering:
deployment edit type single-node profile <X>

Where:

  • X is the profile prior to upgrading.

Example:
deployment edit type single-node profile 30K

The profile is applied.

  1. Confirm the change by entering:
deployment show configuration

An output similar to the following is displayed:

Skylight: deployment show configuration

[ Deployment configuration ]

  Deployment type : single-node
  Profile name    : 30K

  User profile name : user-20
 Max user session : 20

  Cluster configuration :
   Cluster type       : medcluster
   Node Configuration :
Node name       : medn1
Total instances : 6

Post-Installation Procedures

This section explains the procedures applicable after your upgrade.

The post-installation procedures are:

  1. Restarting the Application
  2. Post-Upgrade Checks

Restarting the Application

This procedure will start the upgraded software.

Restarting the application for Hot Standby configuration

  1. Open an SSH terminal session to the Skylight orchestrator designated as the preferred active
    site on port 2200.
  2. Log in as the visionems user.
  3. Configure the preferred site to last active site, enter:
redundancy config preferred {site-a | site-b}
  1. Start the Hot Standby process by entering:
redundancy control start
  1. Verify redundancy status, enter:
redundancy show status


Notes:

It may take up to 10 minutes for the app_server to go from starting to started.

When “redundancy show status” reports that replication status is up, the application can be accessed.


Restarting the application for a Non-Hot Standby configuration

  1. Open an SSH terminal session to the Skylight orchestrator on port 2200.
  2. Log in as the visionems user.
  3. Start the application by entering:
service start app_server


Note: It may take up to 10 minutes for the app_server to go from starting to started.

Verifying the Startup Status

To verify the startup status

  1. Enter:
service show

Skylight orchestrator Upgrade Manual 21.bmp

A list of services (that are currently running) is displayed. Ensure that the state of all services is shown as Started.

  1. Close the terminal session to port 2200 by entering:
exit

Post-Upgrade Checks

The final step of the upgrade is to check the network inventory and ensure that Skylight orchestrator is operational. The following steps constitute a basic verification of the state of Skylight orchestrator.

To verify network inventory and check Skylight orchestrator functionality

  1. Log in to the Skylight orchestrator web application in a supported web browser.

  2. Click the Accedian Skylight orchestrator Upgrade Manual 21_logo icon.bmp icon to display the About window.
    The Build Version should indicate version 23.12.

  3. Access the Device ▶ Managed Elements tab and verify that the total count of network elements at the bottom-left corner is accurate.

  4. Click the Skylight orchestrator Upgrade Manual 21_alarm button.bmp icon at the bottom-right corner and verify that the current alarm count is accurate.

  5. Access the Commission ▶ Jobs tab and verify that any pre-existing configuration jobs are still present.

  6. Access the Collection ▶ Reporting Status tab and verify the following:

    • The list of performance elements with a pm collect agent.
    • The CSV time and Transfer time have both recently been updated for all elements with streaming enabled.

  7. Check your session status by accessing Sessions tab.

Rollback Procedure

This section explains how to roll back to the original operating system partition and restore the system to its initial state.

If you must roll back a cluster deployment, you must repeat the procedure on all appliances on which the upgrade was performed (successfully or unsuccessfully).

Rollback for Skylight orchestrator Procedure

This procedure explains how to roll Skylight orchestrator back to the previous release.

You would only need to perform the rollback procedure if problems occur during the appliance upgrade procedure, such as:

  • An error message occurs during the upgrade process that prevents you from proceeding.
  • The upgrade process terminates unexpectedly.
  • The application does not function as expected after the upgrade.

Procedure Requirements

The following requirement applies to the procedure:

  1. This procedure must be executed as the visionems user. To execute these procedures, you must know the visionems credentials.

Stopping the Application

Stopping the application for Hot Standby configuration

  1. Open an SSH terminal session to the Skylight orchestrator on port 2200.
  2. Log in as the visionems user.
  3. Stop the Hot Standby process by entering:
redundancy control stop


Stopping the application for a Non-Hot Standby configuration

  1. Open an SSH terminal session to the Skylight orchestrator on port 2200.
  2. Log in as the visionems user.
  3. Stop the application by entering:
service stop app_server

Rolling back the upgrade

Repeat this procedure on all appliances that were upgraded.

To roll back the upgrade

  1. Using an SSH terminal session to port 22 of the appliance, log in to the appliance as the visionems user.
  2. After logging in, change to the directory where the scripts are stored by entering:
cd /data
  1. Execute the script that switches to the secondary partition OS (inactive partition):
./upgradeAppliance -sp

The following output is displayed as the script runs:

vision@visionems:/data$ ./upgradeAppliance –sp

[sudo] password for visionems:

* Setting up grub configuration for inactive partition
About to switch boot partition to inactive partition.

  1. When prompted, enter:
y

Do you want to switch? [y/N] y
Applying switch…
Inactive partition mounted
Copying fstab file….
Fstab file was copied
Inactive partition unmounted
Inactive partition mounted
Installing for i386-pc platform.
Installation finished. No error reported.
Boot grub loader configured
Grub configuration has been updated
Inactive partition unmounted
Appliance configured
Inactive partition is now configured as the boot partition

  1. When prompted to confirm that the system reboot is required, enter:
y

A system reboot is required to take effect. Proceed with reboot? [y/N] y
Applying reboot…

This will reboot system automatically. Otherwise you must reboot the system to apply the rollback. If you do not say yes at this point, you will have to reboot the system manually.


Note: The visionems password reverts back to its initial value, i.e. the same as it was at the beginning of the upgrade procedure.

  1. Using an SSH terminal session to port 22 of the appliance, log in as the visionems user.

  2. After logging in successfully, enter:

cd /data
  1. Execute the script that restores the partition table that was used before the upgrade, by entering:
./upgradeAppliance -rp

The following output is displayed as the script runs:

./upgradeAppliance -rp

Extract to temporary folder
[sudo] password for visionems:

Mysql partition umounted
Archive has been applied
Mysql partition mounted
Installing for i386-pc platform.
Installation finished. No error reported.
Boot grub loader configured
Grub configuration has been updated

Successfully restored partition

  1. After the script runs successfully, open an SSH terminal session to port 2200 of the virtual appliance.
    This gives you access to the Appliance Monitor command line.
  2. Log in as the visionems user.
  3. Check the version by entering:
board show info
  1. Reimport the deployment configuration by entering:
deployment re-import


Note: Running the deployment re-import command will delete the contents of the /data/medn*instance and reapply the old structure folders.

Starting the Application

This procedure will start the application.

Starting the application for Hot Standby configuration

  1. Open an SSH terminal session to the Skylight orchestrator designated as the preferred active site on port 2200.
  2. Log in as the visionems user.
  3. Configure the preferred site to last active site, enter:
redundancy config preferred {site-a | site-b}
  1. Start the Hot Standby process by entering:
redundancy control start
  1. Verify redundancy status, enter:
redundancy show status

Starting the application for Non-Hot Standby configuration

  1. Open an SSH terminal session to the Skylight orchestrator on port 2200.
  2. Log in as the visionems user.
  3. Start the application by entering:
service start app_server

© 2024 Accedian Networks Inc. All rights reserved. Accedian®, Accedian Networks®,  the Accedian logo™, Skylight™, Skylight Interceptor™ and per-packet intel™, are trademarks or registered trademarks of Accedian Networks Inc. To view a list of Accedian trademarks visit: http://accedian.com/legal/trademarks/. 


Was this article helpful?

Changing your password will log you out immediately. Use the new password to log back in.
First name must have atleast 2 characters. Numbers and special characters are not allowed.
Last name must have atleast 1 characters. Numbers and special characters are not allowed.
Enter a valid email
Enter a valid password
Your profile has been successfully updated.