✨ New: Try our AI‑powered Search (Ctrl + K) — Read more

Installation

  • Published on May 14, 2025
  • 7 minute(s) read
Prev Next

Once you have completed the prerequisites, you can begin the installation of the Cisco Provider Connectivity Assurance solution.

Have you completed the prerequisites?

Before you proceed with installation, make sure that you have:

    Downloaded the compressed tar archive of Provider Connectivity Assurance
    Installed operating system dependencies
    Passed preflight checks
For more information, see Prerequisites.

1. Run the Installer

  1. Switch to the admin user with superuser privileges.

    $ sudo su - admin

  2. Extract the tar archive.

    $ tar -zxvf airgap_skylight-xx.xx.xx-init.tar.gz skylight-installer/

### Example ###
$ tar -zxvf airgap_skylight-25.02.27-init.tar.gz skylight-installer/

  3. Change the directory to skylight-installer.

    $ cd skylight-installer

  4. Run the SSH authentication agent.

    $ eval `ssh-agent`; ssh-add ~/.ssh/id_rsa

    Note: Run this command whenever starting a new session.

  5. Set the environment variables and run the installation script.

    $ INSTALLER_VERSION=xx.xx.xx [ARCHIVE=/some/path] ./run_installer.sh [--gen-vars]

### Example ###
INSTALLER_VERSION=25.02.27 ARCHIVE=/home/admin/airgap_skylight-25.02.27-init.tar.gz ./run_installer.sh --gen-vars
    • Parameter descriptions:

      • INSTALLER_VERSION=xx.xx.xx

        • Defines the variable INSTALLER_VERSION with the version number (xx.xx.xx).
        • This allows the installer script to reference it during execution.

      • [ARCHIVE=/some/path](Optional)

        • If included, sets ARCHIVE to the specified location of the archive package (/some/path).
        • ARCHIVE is only needed if the --gen-vars switch is used. Otherwise, the archive path in the inventory file is used.

      • ./run_installer.sh

        • Executes the script run_installer.sh from the current directory (.).

      • [--gen-vars](Required for initial install; optional for subsequent runs)

        • If used, this switch runs the Deployment Configurator questionnaire for a new deployment. This questionnaire prompts you for all the variable inputs, generates inventory and variables.env files, and then puts them in the proper location for the remainder of the run_installer.sh to continue.
        • The questionnaire can only be used for deployments with a maximum of eight nodes.

        For information on customizing variables and inventory files, see Variables Definition.

  6. If [--gen-vars] is used, the Deployment Configurator questionnaire displays. Follow the prompts and enter the details where required.

    • Initial page of the questionnaire:

      image.png

  7. Save the Unseal Keys and Root Token.

    When running the installer for the first time, a message displays instructing you to save the Unseal Keys and the Root Token, as shown in this example:

    • Example of the installer running

      TASK [skylight-ssh-prep : Add trusted CA to sshd config] ****************************************************************************************************
ok: [your-hostname]

TASK [skylight-ssh-prep : Restart service sshd] *************************************************************************************************************
skipping: [your-hostname]

TASK [Share important information] **************************************************************************************************************************
Pausing for 1 seconds
(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
[Share important information]
Change: Vault initialize!

###########IMPORTANT#############IMPORTANT#############:
Save the following information:

{"keys": ["2c2*****79"], "keys_base64": ["LCf*********nk="], "root_token": "s.bs*******0"}

Change: Vault Unsealed!
Change: Secrets engine "secret" enabled!
Change: Secrets engine "ssh-client-signer" enabled!
Change: admin-ssh-users policy created!
Change: deployer-secrets-read policy created!
Change: SSH key generated
Change: deployer/nso-gw/kanatagrid.io/performance/secrets.yaml secret created!
Change: landlord/datahub-creds/default secret created!
Change: datahub-admin/default secret created!
Change: secret/data/deployer/ramen/server_private_key secret created!
Change: deployer/jwt/performance-analytics secret created!
Change: deployer/jwt_report_svc/performance-analytics secret created!
Change: deployer/onprem/ca secret created!
Change: deployer/gcs/ro-key secret created!

ATTENTION!!!: Save the keys listed above
:
ok: [your-hostname]

PLAY RECAP **************************************************************************************************************************************************
your-hostname        : ok=34   changed=2    unreachable=0    failed=0    skipped=15   rescued=0    ignored=0
    💡IMPORTANT

    Make sure to save and secure the Unseal Keys, as these cannot be recovered. This is the master key for all secrets. Do not store this information as plaintext.

    The Unseal Key is used to unseal the vault after restarts and to generate Root Tokens.

    The initial Root Token can be reissued with the use of the Unseal Key. Keep this token accessible throughout the installation process.

The script may prompt for additional details post-execution, such as:

  • a vault Unseal Key
  • a vault Root Token

These prompts are contingent on the current state of the deployment.

2. Validate and Wait for Services

Check the state of the swarm by using the following command:

$ docker node ls

Example of expected output:

ID                            HOSTNAME   STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
dgu23eqgzmrwmi0z6f00korlc *   leader   **Ready**     Active         Leader           24.0.2
598zqxoxb1lzg3wfo2qfsnxe2     worker  **Ready**     Active                          24.0.2

3. Run the Deployer

  1. On the leader node, run the onprem-deploy.sh script.

    $ eval `ssh-agent`; ssh-add ~/.ssh/id_rsa
$ bin/onprem-deploy.sh 25.02.27 ./variables.env ./inventory

    Example of expected output with “failed=0”

    PLAY RECAP ***************************************************************************************************************************
16-156-0-leader            : ok=105  changed=61   unreachable=0    failed=0    skipped=57   rescued=0    ignored=0
16-156-0-worker            : ok=2    changed=0    unreachable=0    failed=0    skipped=8    rescued=0    ignored=0

  2. List all running services.

    $ docker service ls

    • Example of output:

      ID             NAME                                 MODE         REPLICAS               IMAGE                                                                                            PORTS
hjmts9448lic   aod_airflow                          replicated   1/1                    skylight-repo.internal:5000/airflow-2.7.3:0.58.0
v2efauwduvaw   aod_airflow-scheduler                replicated   1/1                    skylight-repo.internal:5000/airflow-2.7.3:0.58.0
uu58gc9uo8bw   aod_alert-export-service             replicated   1/1                    skylight-repo.internal:5000/alertexportservice:0.14.0
t2bn46or8ip1   aod_alert-service                    replicated   1/1                    skylight-repo.internal:5000/alertservice:0.59.0
6pdp02fyb4dj   aod_beacon                           global       0/0                    skylight-repo.internal:5000/sosbeacon:1.3.1
zntg3bcqzwq5   aod_blackbox-exporter                replicated   1/1                    skylight-repo.internal:5000/3rdparty/quay.io/prometheus/blackbox-exporter:20231205
9je5d68pyitr   aod_broker                           replicated   1/1                    skylight-repo.internal:5000/docker-druid-27.0.0:0.42.0
0p0mb9zq8tbi   aod_cadvisor                         global       2/2                    skylight-repo.internal:5000/cadvisor/cadvisor:v0.47.2
xrur9y22m9j5   aod_cas                              replicated   1/1                    skylight-repo.internal:5000/cas-accedian-6.0.3:0.12.0
fmzm2bvlqlsl   aod_consul                           global       1/1                    skylight-repo.internal:5000/3rdparty/consul:1.8.6
k5e4zinj4sb5   aod_coordinator                      replicated   1/1                    skylight-repo.internal:5000/docker-druid-27.0.0:0.42.0
rvu6lhhgmkhf   aod_couchdb                          replicated   1/1                    skylight-repo.internal:5000/docker-couchdb:0.26.0
tq1yllp7y386   aod_couchdbstats                     replicated   1/1                    skylight-repo.internal:5000/3rdparty/gesellix/couchdb-prometheus-exporter:v30.9.0
vldxmwzevm2i   aod_custom-reports                   replicated   1/1                    skylight-repo.internal:5000/custom-reports:0.208.0
7xczfb8nexht   aod_datanode1                        replicated   1/1                    skylight-repo.internal:5000/hdfs-hadoop-3.1.3:0.9.0
5a6bhaylpbhz   aod_dgraphsolo                       replicated   1/1                    skylight-repo.internal:5000/dgraph/standalone:v23.1.0
zxkk42zqzhm0   aod_elasticsearch01                  replicated   1/1                    skylight-repo.internal:5000/elasticsearch:0.12.0
g1yig00vornk   aod_elasticsearch-metrics-exporter   replicated   1/1                    skylight-repo.internal:5000/3rdparty/quay.io/prometheuscommunity/elasticsearch-exporter:v1.6.0
wnvni4s5agfg   aod_fedex                            replicated   1/1                    skylight-repo.internal:5000/adh-fedex:0.166.0
q27muaj8q9u1   aod_foxtrot-service                  replicated   1/1                    skylight-repo.internal:5000/foxtrotservice:0.136.0
rx1p2gxa1oxh   aod_gather                           replicated   1/1                    skylight-repo.internal:5000/adh-gather:0.1487.0
atlr3003f38o   aod_grafana                          replicated   1/1                    skylight-repo.internal:5000/grafana:0.183.0
g0zmg2hwgbop   aod_grafana-exporter                 replicated   1/1                    skylight-repo.internal:5000/grafana-impex:0.1.2
lpynmgb8scri   aod_health-clinic                    replicated   1/1                    skylight-repo.internal:5000/health-clinic:0.32.0
japdvm8czaur   aod_historical                       replicated   1/1 (max 1 per node)   skylight-repo.internal:5000/docker-druid-27.0.0:0.42.0
td2of3vjxldi   aod_ignite                           replicated   1/1 (max 1 per node)   skylight-repo.internal:5000/ignite:0.38.0
n2vvoyoredat   aod_interlock                        replicated   1/1                    skylight-repo.internal:5000/interlock:0.17.0
slprm4oya0k1   aod_kafka                            replicated   1/1                    skylight-repo.internal:5000/kafka_3.6.1:0.8.0
7u564wsudi5g   aod_kafka-jmx-exporter               replicated   1/1                    skylight-repo.internal:5000/kafka-prometheus-monitoring:dev
hxeso0inzvy7   aod_kafka-topic-metric-exporter      replicated   1/1                    skylight-repo.internal:5000/3rdparty/danielqsj/kafka-exporter:20231116
irq9cgjs2ewz   aod_kafkalagmonitor                  replicated   1/1                    skylight-repo.internal:5000/adh-spark-app:0.413.0
pc6ujs5nd4tk   aod_latentapp                        replicated   0/0                    skylight-repo.internal:5000/adh-spark-app:0.413.0
u6coup02mp71   aod_lighthouse                       replicated   1/1                    skylight-repo.internal:5000/lighthouse:1.3.1
yme1rshifauc   aod_middlemanager                    replicated   1/1 (max 1 per node)   skylight-repo.internal:5000/docker-druid-27.0.0:0.42.0
sv6vtl2k5mrf   aod_middlemanager-manager            replicated   0/0 (max 1 per node)   skylight-repo.internal:5000/docker-druid-27.0.0:0.42.0
0s6erxxz5adi   aod_minio                            replicated   1/1                    skylight-repo.internal:5000/minio-haproxy:0.6.0
myu07s7tyjeq   aod_minio1                           replicated   1/1                    skylight-repo.internal:5000/minio:RELEASE.2019-10-02T21-19-38Z-0.12.0
hzgchb8bf8zy   aod_minio2                           replicated   1/1                    skylight-repo.internal:5000/minio:RELEASE.2019-10-02T21-19-38Z-0.12.0
82it1x7g7u57   aod_minio3                           replicated   1/1                    skylight-repo.internal:5000/minio:RELEASE.2019-10-02T21-19-38Z-0.12.0
1y8hxyto7zwf   aod_minio4                           replicated   1/1                    skylight-repo.internal:5000/minio:RELEASE.2019-10-02T21-19-38Z-0.12.0
3m8e5yrawkkf   aod_namenode                         replicated   1/1                    skylight-repo.internal:5000/hdfs-hadoop-3.1.3:0.9.0
myi6nsg6a7w1   aod_nginx                            global       1/1                    skylight-repo.internal:5000/nginx:0.22.0                                                         *:80->80/tcp, *:2443->2443/tcp
lxb3tk4jejts   aod_ofelia                           replicated   1/1                    skylight-repo.internal:5000/ofelia:0.6.0
rmg3ingrjnsy   aod_overlord                         replicated   1/1                    skylight-repo.internal:5000/docker-druid-27.0.0:0.42.0
uu8ada2ecwy1   aod_postgres                         replicated   1/1                    skylight-repo.internal:5000/docker-postgres:0.31.0
e6epvfszimri   aod_postit                           replicated   1/1                    skylight-repo.internal:5000/adh-postit:0.11.0
ag6h9wx9w7gw   aod_prometheus                       replicated   1/1                    skylight-repo.internal:5000/3rdparty/quay.io/prometheus/prometheus:v2.54.1
bxujuww7sd2t   aod_prometheus-gateway               replicated   1/1                    skylight-repo.internal:5000/3rdparty/prom/pushgateway:v1.10.0
iz9o25l32i6c   aod_rabbitmq                         global       1/1                    skylight-repo.internal:5000/dhiq-rabbitmq-mqtt-ws:0.6.0
qid1osinbua0   aod_redis                            replicated   1/1                    skylight-repo.internal:5000/3rdparty/redis:7.2.3
ioh1njg9y7f4   aod_report-service                   replicated   1/1                    skylight-repo.internal:5000/report-service:3.230.33
b6b2sp8ufmw0   aod_router                           replicated   1/1                    skylight-repo.internal:5000/docker-druid-27.0.0:0.42.0
i3jd6whoncef   aod_secondarynamenode                replicated   1/1                    skylight-repo.internal:5000/hdfs-hadoop-3.1.3:0.9.0
f8ajjcxbea25   aod_skylight-aaa                     replicated   1/1                    skylight-repo.internal:5000/skylight-aaa:1.36.0
zzinppdfy39r   aod_skylight-foss-presenter          replicated   1/1                    skylight-repo.internal:5000/skylight-foss-presenter:0.7.0
vuiuzhl2dn82   aod_skyweather                       replicated   1/1                    skylight-repo.internal:5000/skyweather:0.1.1                                                     *:30007->8000/tcp
iopohtpoo36r   aod_sparkbatch                       replicated   0/1                    skylight-repo.internal:5000/adh-spark-app:0.413.0
w4dcbw3x796q   aod_sparkmaster                      replicated   1/1                    skylight-repo.internal:5000/spark-3.5.3-hadoop-3.3.1:0.56.0
lqq9vohgcy0f   aod_sparkmasterlatent                replicated   1/1                    skylight-repo.internal:5000/spark-3.5.3-hadoop-3.3.1:0.56.0
qei476ggsu9h   aod_sparkworker1                     replicated   1/1                    skylight-repo.internal:5000/spark-3.5.3-hadoop-3.3.1:0.56.0
aj4rkhp2xev2   aod_sparkworkerlatentstreaming       replicated   1/1                    skylight-repo.internal:5000/spark-3.5.3-hadoop-3.3.1:0.56.0
r5pk5r8stc3g   aod_stitchit                         replicated   1/1                    skylight-repo.internal:5000/stitchit:0.74.0
tlfweo7n0ayj   aod_stonehenge-exporter              replicated   1/1                    skylight-repo.internal:5000/stonehenge-exporter:1.11.0
x34soik5iath   aod_streamingapp                     replicated   1/1                    skylight-repo.internal:5000/adh-spark-app:0.413.0
qef400h9nyut   aod_ui                               replicated   1/1                    skylight-repo.internal:5000/skylight-ui:6.87.13
kcw0ul3no5mh   aod_updateserver                     replicated   1/1                    skylight-repo.internal:5000/updateserver:0.47.0
oytpg4hbjc76   aod_weld-api                         replicated   1/1                    skylight-repo.internal:5000/weld-api:6.54.15
o88n54ekibed   aod_weld-data                        replicated   1/1                    skylight-repo.internal:5000/weld-data:6.19.14
any3lxdp8gt9   aod_weld-ignition                    replicated   1/1                    skylight-repo.internal:5000/weld-ignition:6.4.11
x4td1are57nc   aod_weld-watcher                     replicated   1/1                    skylight-repo.internal:5000/weld-watcher:6.25.0
inmspkmnxqp8   aod_zeppelin                         replicated   1/1                    skylight-repo.internal:5000/zeppelin-0.11.2:0.16.0
ja7yatt8s14q   aod_zitadel                          replicated   1/1                    skylight-repo.internal:5000/zitadel:0.1.0-fips
pn2zls11qslv   aod_zookeeper                        replicated   1/1                    skylight-repo.internal:5000/aod_zookeeper-3.9.1:0.11.0

4. Refresh Stored Secrets

Refresh stored secrets for an on-prem deployment to ensure that the correct secrets are applied to the environment.

  1. Determine the key for your certificates stored in the vault.
    On the leader node, as the admin user, run the following command in the /home/admin/skylight-installer directory.

    $ . /etc/aod/.env;./bin/vaultCmd.sh kv list secret/deployer/certs/onprem/${DEPLOYMENT_NAME}/deployment-ca

    Example of expected output:

    Keys
----
1806848091000000000

  2. Run the following command using the key from the output of the previous step.

    $ . /etc/aod/.env;./bin/landlord.sh refreshSecrets onprem ${DEPLOYMENT_NAME} https://${DEPLOYMENT_IP} --leader-ip ${DEPLOYMENT_IP} --vault-address http://localhost:8200 --cacert-base-path deployer/certs/onprem/${DEPLOYMENT_NAME} --cacert-root-path deployer/certs/onprem/${DEPLOYMENT_NAME}/deployment-ca/<<key from above - ex. 1806848091000000000>>

    This command takes a few minutes to run. It is regenerating Docker secrets and restarting some services.

    Example of expected output:

    verify: Waiting 1 seconds to verify that tasks are stable...
verify: Service aod_skylight-aaa converged
**INFO	2025/04/04 20:42:35.903042 refreshSecrets completed**

5. Confirm that Services are Running

Confirm that all services are up and running on the system by attempting a login with the curl command from the leader node.

  1. Obtain the default user credentials from the vault (from the skylight-installer directory).

    $ ./bin/vaultCmd.sh kv get secret/datahub-admin/default

  2. Log in with the curl command from the leader node.

    $ curl -k --noproxy '*' -X POST https://<<LEADER_HOSTNAME>>/api/v1/auth/login -H 'Content-Type: application/x-www-form-urlencoded' -d "username=<<USERNAME_FROM VAULT>>&password=<<PASS_FROM_VAULT>>

    Example of expected output:

    {"data":{"attributes":{"emailAddress":"admin@datahub.com","groups":null,"name":"Datahub Admin","roles":["skylight-admin","skylight-admin"],"tenants":["performance - Deployment"]},"id":"1","type":"users"}}

    Once this returns a successful login, you can proceed with tenant creation.

6. Create a Tenant

While Provider Connectivity Assurance is a multi-tenant system, private cloud infrastructure typically supports only a single tenant.

All data within the platform—including monitored objects, metrics, metadata, dashboards, and user profiles—is associated with a specific tenant. Each tenant maintains its own Role-Based Access Control (RBAC) rules and user base.

At least one tenant must be created. Once all of the services are running, create a tenant with the following command:

$ bin/createTenant.sh <<Your Variables file>>

### Example ###
$ bin/createTenant.sh ./variables.env

The createTenant.sh script updates the tenant secrets.

Example of expected output:

INFO	2025/04/04 20:49:14.641165 RotateCACertificates: removing secrets from services, this will trigger some restarts
INFO	2025/04/04 20:50:12.888139 RotateCACertificates: deleting secrets from swarm
INFO	2025/04/04 20:50:13.541903 RotateCACertificates: creating secrets in swarm
INFO	2025/04/04 20:50:14.197258 RotateCACertificates: adding secrets to services, this will trigger some restarts
**INFO	2025/04/04 20:51:12.122741 The tenant analytics was SUCCESSFULLY provisioned**

7. Access the Web UI

Test access to the Web UI of the deployed solution by navigating to https://<LEADER_IP_ADDRESS>/ in a web browser, where the Provider Connectivity Assurance splash screen should display.

Note: You must have connectivity to the LEADER IP address from your machine.

This completes the on-prem installation of the Provider Connectivity Assurance platform.

Next Steps

Configure Provider Connectivity Assurance for first time use. See Configuration for Initial Use.

© 2026 Cisco Privacy Statement Cookies Legal