Skylight orchestrator 23.12.3 Release Notes

Cisco is pleased to announce the 23.12.3 release of Skylight orchestrator. These release notes cover the new features, changes, and corrected issues for Skylight orchestrator version 23.12.3. Read the release notes before installing this firmware version.

The following topics are addressed in this document:

• New Features: Covers the new features introduced in this release.

• Corrected Issues: Lists issues that have been corrected in this version of the software.

• Technology Support: Covers the applicable product information.

• Upgrade Paths: Details the supported upgrade paths for this release.

• Deployment Considerations: Covers general limitations of the current release.

• Release Lifecycle: Lists the planned lifecycle dates of this software release. Includes table with milestones.

For more information, see the Features and User Material documentation.

New Features

Skylight orchestrator 23.12.3 introduces the following features and enhancements.

Enhanced Device Backup Information

The Skylight orchestrator Backup & Restore tab for devices includes additional information for increased clarity; the firmware version of each device is visible at the time the backup was taken, providing users with essential data for system management and troubleshooting.

Skylight orchestrator Certificate Management Update

This release removes the use of default device certificates (pre-generated private CA certificates). Users must generate self-signed certificates or import their own certificates for each device. This change enhances security and affects the Skylight orchestrator reverse proxy functionality.

Provider Connectivity Assurance Sensor Compatibility

Skylight orchestrator has been enhanced to be backward compatible with Provider Connectivity Assurance Sensors, including release 25.07 for Assurance Sensors LT, LX, and GT as well as release 24.08 for Assurance Sensors F100 and F25.

Provider Connectivity Assurance Sensor Control Compatibility

Skylight orchestrator has been enhanced to operate with Assurance Sensor Control releases 25.07 and 25.07.1. Please note that Skylight orchestrator releases prior to release 23.04 are unable to provide RTT values for TWAMP and ETH-DM using Sensor Control releases 22.12 or later.

If you intend to deploy Sensor Control 22.06 - 24.x to your network, please ensure you have first upgraded to Legacy Orchestrator release 24.09 or later.

If you intend to deploy Sensor Control 25.07 or 25.07.1 to your network, please ensure you have first upgraded to Skylight orchestrator release 23.12.3 (this release).

See the Assurance Sensor Control Release Notes 25.07.1 for more information.

Corrected Issues

As part of our commitment to providing ongoing security updates to the 23.12 LTS release, the following key issues have been addressed in Skylight orchestrator 23.12.3.

Ubuntu Security Vulnerabilities

• Ubuntu Security Notification for GLib Vulnerability (USN-7114-1) affecting Ubuntu 16.04 LTS, 18.04 LTS, 20.04 LTS, 22.04 LTS, and 24.04 LTS

• Ubuntu Security Notification for Corosync Vulnerability (USN-7478-1) affecting Ubuntu 20.04 LTS, 22.04 LTS, 24.04 LTS, and 24.10

• Ubuntu Security Notification for Linux Kernel Vulnerabilities (USN-7293-1) affecting Ubuntu 18.04 LTS and 20.04 LTS

• Ubuntu Security Notification for Kerberos Vulnerability (USN-7257-1) affecting Ubuntu 14.04 LTS, 16.04 LTS, 18.04 LTS, 20.04 LTS, 22.04 LTS, 24.04 LTS, and 24.10

• Ubuntu Security Notification for Intel Microcode Vulnerabilities (USN-7149-1) affecting Ubuntu 16.04 LTS, 18.04 LTS, 20.04 LTS, 22.04 LTS, 24.04 LTS, and 24.10

• Ubuntu Security Notification for libxml2 Vulnerabilities (USN-7240-1) affecting Ubuntu 20.04 LTS, 22.04 LTS, and 24.04 LTS

• Ubuntu Security Notification for FreeType Vulnerability affecting Ubuntu 20.04 LTS and 22.04 LTS

• Ubuntu Security Notification for Python Vulnerabilities (USN-7348-1) affecting Ubuntu 14.04 LTS, 16.04 LTS, and 20.04 LTS

• Ubuntu Security Notification for GNU C Library Vulnerability (USN-7541-1) affecting Ubuntu 18.04 LTS, 20.04 LTS, and 22.04 LTS

• Multiple Linux Kernel Vulnerabilities affecting Ubuntu 18.04 LTS and 20.04 LTS (including USN-7159-1)

• Ubuntu Security Notification for Libxslt Vulnerability (USN-7361-1) affecting Ubuntu 20.04 LTS, 22.04 LTS, 24.04 LTS, and 24.10

• Ubuntu Security Notification for Setuptools Vulnerability (USN-7544-1) affecting Ubuntu 14.04 LTS through 25.04

• Ubuntu Security Notification for libxml2 Vulnerabilities (USN-7302-1) affecting Ubuntu 14.04 LTS through 24.10

• Ubuntu Security Notification for rsync Vulnerabilities (USN-7206-1) affecting Ubuntu 14.04 LTS through 24.04 LTS

• Ubuntu Security Notification for SQLite Vulnerabilities (USN-7528-1) affecting Ubuntu 20.04 LTS through 25.04

• Ubuntu Security Notification for libxm Vulnerabilities (USN-7467-1) affecting Ubuntu 20.04 LTS through 24.10

• Ubuntu Security Notification for Bind Vulnerabilities (USN-7241-1) affecting Ubuntu 20.04 LTS through 24.10

Additional Security Fixes

• SSH Terrapin Prefix Truncation Weakness (CVE-2023-48795)

Technology Support

These release notes apply to the following product:

The following table lists key functionality of Assurance Sensors, Assurance Sensor Control, Assurance Sensor Modules, and Assurance Sensor SFPs supported by Skylight orchestrator 23.12.3.

Supported in Skylight orchestrator Version 23.12.3

* CLI step support for Device Commissioning
** Y.1564 with flow template support

Upgrade Paths

The upgrade to Skylight orchestrator 23.12.3 requires that you are running a system with one of these previously-installed software versions:

Virtual machine running:

• 23.12.1 5K profile

• 23.12.1 15K profile

• 23.12.1 30K profile

• 23.12.1 60K profile

• 23.12.2 5K profile

• 23.12.2 15K profile

• 23.12.2 30K profile

• 23.12.2 60K profile

Note: Release 23.12.3 does not have a supported direct upgrade path to Docker-based Skylight orchestrator releases. Instead, a migration procedure is provided for moving from VM-based to Docker-based deployment.

Consider migrating to the Skylight orchestrator Docker before proceeding with this maintenance release. Depending on your setup, refer to either "Migrating from Hot Standby Virtual Machine to Docker" or "Migrating from a Non-Hot Standby Virtual Machine to Docker" for the appropriate procedure.

Detailed procedures for upgrading to this release are provided in Upgrading to Skylight orchestrator 23.12.3.

IMPORTANT: You cannot upgrade hardware appliances to version 23.12.3.

For customers running SkyLIGHT Director software on hardware appliances, a rehoming procedure is available that allows the transfer of the software and its configuration to a new virtual machine.

Detailed procedures for rehoming on a virtual machine are provided in Virtual Machine Rehoming for Releases Prior to 20.05.

Deployment Considerations

Hypervisor Guidelines

Skylight orchestrator has been tested against KVM and VMware hypervisors. The following guidelines should be used when deploying to these environments:

• The engineering guidelines of the product must be respected. The product ships with a minimal configuration that in most cases needs to be adjusted to meet the requirements stated in Deployment Profile Configuration. Failure to respect these engineering settings can lead to application downtime and possible data loss.

• The product was designed assuming MAC addresses and UUIDs assigned on installation of the virtual appliance will persist for the lifetime of the virtual appliance. Licensing functions are tied to these identifiers and unexpected behavior can occur if they are changed. This is especially important when installing the virtual appliance in a cluster. These assignments must be static.

• When using hypervisor functions that involve dynamic load balancing of guest virtual appliances in a hypervisor cluster (i.e. VMware DRS), the balancing policy should limit rebalancing operations to be done on virtual appliance startup and/or hypervisor host failures.

• When the Skylight orchestrator VM is deployed on the KVM host, ensure to enable all ports after they are added to Skylight orchestrator.

Device Interoperability

The following section details the firmware releases that are known to interoperate with this release of Skylight orchestrator. Please refer to the Administration documentation for a listing of supported functions for each firmware version.

Support for sensor element software releases 4.7.x, 5.4.x and 5.5.x is deprecated. Plans should be put in place to upgrade your performance elements to more recent versions.

For the releases mentioned below, support for maintenance releases is also included.

ESXi Support

Skylight orchestrator release 20.05 and later will not support ESXi hypervisors running release 5.5 and 6.0. Systems running ESXi 6.5 can be upgraded to release 20.05 and later. ESXi 7.0 is supported for release 22.10 and later.

HTTP Interface

The HTTP interface of Skylight orchestrator is no longer supported for GUI operations. HTTP operations are still supported for the REST and XML APIs, but web service clients are urged to switch to the more secure HTTPS protocol.

Customers using browsers with HTTP protocol on port 6080 will automatically be redirected to HTTPS protocol on the same port (6080). This means no firewall changes are required as part of this change.

Disk Sizing for New Installations

The default image size for Skylight orchestrator is now 150 GiB. This image is large enough to support small networks of under 100 network elements and 1,000 performance sessions. For larger networks, the disk size of the image must be increased. This can be achieved via VMware and KVM management tools.

Example disk sizing:

• For 5,000 elements and 20,000 performance sessions with Round trip delay mode disabled: 370 GiB is recommended

• For 5,000 elements and 20,000 performance sessions with Round trip delay mode enabled: 430 GiB is recommended

• For 52,000 performance sessions with Round trip delay mode disabled: 482 GiB is recommended

• For 52,000 performance sessions with Round trip delay mode enabled: 638 GiB is recommended

• For 125,000 performance sessions with Round trip delay mode disabled: 920 GiB is recommended

• For 125,000 performance sessions with Round trip delay mode enabled: 1295 GiB is recommended

Please consult the Skylight orchestrator documentation for sizing guidelines and for procedures on how to increase the disk size.

IPv4 Addresses for Hot Standby

The hot standby feature requires the use of IPv4 addresses for its configuration. For planning purposes, it should also be noted that hot standby configuration requires three (3) distinct subnets for the management, data replication, and heartbeat functions.

IPv6 addresses can be used in addition to these IPv4 interfaces for connectivity to the application interfaces (web and REST APIs) and devices being managed. IPv4 addresses are only required strictly for the hot standby functions.

If you wish to make use of this feature, please contact your Solution Engineering representative to initiate network planning activities.

XML Interface Deprecated

The XML interface is deprecated and has been phased out. Northbound systems are urged to migrate to the CSV file format for PM data reception.

Manager Module Interoperability

Conflicts will arise if both Manager Module and Skylight orchestrator are configured to manage the same Skylight sensor: control. Skylight orchestrator will effectively take over all management of the performance sessions for the Skylight sensor: control, and leave the Manager Module with un-acquired supervision endpoint.

Using this release to add Manager Modules as managed elements is not recommended.

Manager API

The performance session management calls defined at the /nbapi REST endpoint were introduced in SDV 1.6 to duplicate the calls present in the Skylight Director Manager Module. This endpoint was preserved in a backwards compatible fashion to allow REST API clients of the Manager Module to seamlessly switch to the Skylight orchestrator.

This endpoint is to be considered capped. Any new features pertaining to performance session management will be implemented using the /nbapiemswsweb REST endpoint in future releases.

CLI Additions and Changes

The CLI command set of the appliance monitor interface was enhanced to duplicate the mgr-commands that are present in the SkyLIGHT Director Manager module.

These commands were preserved in a backwards compatible fashion to allow customers familiar with the Manager Module command set to migrate to the Skylight orchestrator.

This command set is to be considered capped; no new features will be developed on the CLI functionality.

Browser Support Limitations

The minimum recommended screen resolution to operate Skylight orchestrator is 1360 x 768. Lower resolutions will not provide an optimal experience.

Performing a zoom on your display is not recommended due to incompatible implementations with browsers. If a zoom must be applied, please use a Firefox browser as it has the most standard support for this function.

Due to issues with browser compatibility mode and web sockets, Internet Explorer is no longer supported. Please use Google Chrome (version 70 or higher) or Firefox (version 52 or higher) to access the Skylight orchestrator web interface.

For Windows users, it is also recommended to leave the scale and layout option of the Display settings at a value of 100%.

Restoring Virtual Machine Configuration Backups

Configuration backups of the virtual machine (done via the configuration export command) are not portable from release to release. That is to say, a configuration backup from a 1.6 system should not be used to restore a 19.12 system. Unexpected behavior may occur.

Please ensure that configuration backups are restored on systems running the same release from which the backup was taken.

Metrics Collection CSV Filename Timestamp

In order to align with the CSV filename convention of all other Accedian products (Skylight sensor: control and Skylight elements), this naming strategy is changed in Skylight orchestrator. The timestamp in the CSV filename represents the time when the file was created.

In all cases, the timestamps of the metrics themselves (present within the content of the file) represent the time of when the metric was produced. Only the timestamp contained in the CSV filename is impacted by this change.

Vision Collect Streaming Limitations on 6.4.1.2 and 6.4.2

When using release firmware 6.4.1.2 with Vision Collect, a disconnection from Skylight orchestrator may incur data loss. The data retention periods are not respected and data loss can occur after a few seconds of disconnection.

When using firmware release 6.4.2 in Skylight elements in high resolution mode with packet loss greater than 10% of the management network, some reporting periods may be lost. Please ensure a reliable management network is in place.

These issues are corrected in Skylight element firmware releases 6.4.3 and higher.

The First Result Records for a New Session Are Skipped by CSV Export

When the CSV producer detects a new performance session, it marks its data for extract from the first time it views the session. If the CSV producer is configured to run every five (5) minutes, this can mean that the first five minutes of result data for a new session will not be exported.

This is expected behavior; all performance data after this initial detection phase will be captured by the CSV producer.

Number of Acceptable Firmware Loads

Before upgrading or starting app_server on Skylight orchestrator, you must validate the number of acceptable firmware loads based on the specific profile of the system and upgrade version.

Skylight orchestrator Profile and Acceptable Firmware Loads

Note: When the requirements from the table above are met, the upgrade can be performed and the system will restart. After the system restarts, the partition can be checked. The partition must have at least 1 GB of free space remaining.

vMotion Support Limitations

VMWare’s vMotion feature, technology that enables live migration of a virtual appliance from one physical server to another, has certain limitations when used with Skylight orchestrator:

• vMotion should be configured static; no load balancing

• MAC preservation

• Cannot be used in conjunction with Skylight orchestrator HA or standby protection

• If using VSAN disk shared between many hosts, ensure disk IOPs of Skylight orchestrator will meet the minimum requirement.

Result Records for Sync Sessions Are Not Exported in Real Time

Sync sessions are not properly aligned with CSV export. Results are not being exported in real time as Assurance Sensor Control is delivering measurement results and Skylight orchestrator is generating CSV files at the exact same time.

If the CSV producer is configured to run every five (5) minutes and sync session has five (5) minute interval, this can indicate that result data for that session will be exported five minutes late.

Memory Limitations

In the event of an excessive number of simultaneous commands being executed, it is recommended that JVM memory be increased.

FTP Support

Support for the FTP transfer method has been removed for the functions below:

• CSV export

• Inventory export

Please ensure the FTP export method is not used on Inventory Export and CSV Export before upgrading.

Device Password Management Considerations

When supporting or upgrading devices, be aware of password management requirements and potential limitations that may affect device access and integration.

• When supporting Assurance Sensor Control 24.09 or later and Assurance Sensors 24.07 or later, users may not be aware when a device password change is required. As a result, the device may remain in-service but not function properly because most commands are disabled in Skylight orchestrator. To change the password, users must either manually access the device(s) via Web GUI or CLI or access reverse proxy before adding the device(s) to the Skylight orchestrator GUI.

• When upgrading Assurance Sensor Control 24.09 or later and Assurance Sensors 24.07 or later to version 25.07 or when performing a factory reset, Skylight orchestrator cannot automatically be configured as a trap receiver for the device. The password must be updated on the device before adding the device to Skylight orchestrator.

• When either Assurance Sensor Control or Assurance Sensors running version 25.07 or later has an expired password, the device will be out of service, and users may not be notified in advance. To change the password, access the device directly (via Web GUI or CLI). Alternatively, you can disable the password policy. Disabling the policy prevents password expiry for the credential being used to manage the device on Skylight orchestrator.

• It is not possible to change the password for Assurance Sensor Control or Assurance Sensors running version 25.07 or later during the first login if accessing through a reverse proxy. To change the password, access the device directly (via Web GUI or CLI).

23.12.3 Lifecycle

This section lists the planned lifecycle dates of this software release. See the table below outlining the following milestones:

Note: Version 23.12.3 is the final maintenance release for the 23.12 branch. Support for virtual machine-based deployments under the Long-Term Support (LTS) program concludes with this release. Please ensure all deployments are migrated to Docker, as no further updates or support will be provided for VM-based environments.