Deploying Sensor Collector in Docker

System Requirements

• 100 GB disk space
• 4 CPU
• 2 GB RAM
• Docker 24.0.4 or later installed from the Docker software repositories and not a snap
• Tested operating systems: Debian 11 or higher, CentOS 7.3 or higher - should work with anything that supports Docker 24.0.4. You can follow this on how to install Docker on your system.
• No Firewall or if required, traffic to outside world on TCP port 443 (secured port for HTTPS) should be allowed.

Setting up your environment

To standardize deployments

We have come up with a directory structure to follow for deployments.

1. Create a directory for your Sensor Collector (formerly Roadrunner) software in /opt/
   > sudo mkdir -p /opt/roadrunner
2. Create a directory for your data in: /var/lib/data/
   > sudo mkdir -p /var/lib/data/roadrunner

Setting up Sensor Collector Configuration

To set up Sensor Collector configuration

1. Log in to your Cisco Provider Connectivity Assurance (formerly Skylight performance analytics) deployment.

2. Navigate to Settings on the bottom left of the application navigation menu.

3. Navigate to Session setting menu at the top.

4. Sensor Collector configurations are under the Connectors submenu.

5. Click the + button on the right to add a Sensor Collector configuration.

6. Add a name for this connector.

7. Select your datasource type (FileWatcher for CSV upload).

8. Select the Path to your data which should match to the directory in your environment (i.e. /var/lib/data/roadrunner)

9. Specify a Zone name (MUST BE UNIQUE per connector)
   Sample Configuration:
   

10. Press the ✅ on the top-right to save the configuration.

11. Wait for the success message.

12. Download the Sensor Collector by pressing the ellipsis (...) beside the connector you added on the table and select Download Connector Config.

Running Sensor Collector

After you have downloaded the Sensor Collector archive, extract it and run Sensor Collector.

To run Sensor Collector

1. Copy the newly downloaded 'connectornameyyyy-mm-dd'.tar.gz to your VM.
2. Move it to the path you have previously set up for Sensor Collector software:

sudo mv 'connectornameyyyy-mm-dd'.tar.gz /opt/roadrunner/.*

3. Go to that directory:

cd /opt/roadrunner*

4. Extract the tar.gz file:

sudo tar xfv 'connectornameyyyy-mm-dd'.tar.gz*

5. Run Sensor Collector:

sudo ./run.sh

On first startup, Sensor Collector sets up M2M authentication and will require your Cisco Provider Connectivity Assurance credentials. When it asks you for password, enter the same credentials that you used for logging in to DataHUB.

Upgrading Sensor Collector

To upgrade Sensor Collector

1. Go to your Cisco Provider Connectivity Assurance portal (login if necessary).
2. Go to Settings ▶ Session ▶ Connectors.
3. Select the Connector you wish to upgrade
4. Click the connector's Ellipsis icon (...) on the row and choose to Download Connector Config.
5. Wait until the browser has downloaded the file. The file will be named: 'connectornameyyyy-mm-dd'.tar.gz.
6. Copy the downloaded file to your existing Sensor Collector directory.
7. Extract the downloaded file using the following command (this will overwrite the old Sensor Collector docker image)

sudo tar xfv 'connectornameyyyy-mm-dd'.tar.gz*

4. Verify that your file sizes and dates have changed.
5. Move the old Sensor Collector certs so it can be refreshed:

sudo mv .rr_ssh .rr_ssh.bk

6. Now you can restart Sensor Collector by executing the run.sh file:

sudo ./run.sh*

7. (Optional) Clean up docker images after a successful major upgrade.
   For information, see docker image prune and docker image rm.

Appendix A: Filtering Sessions

Sometimes you don't always want Sensor Collector to send all of the object data to Cisco Provider Connectivity Assurance - you want it to be selective. We have session filtering for this purpose.

Session filtering is now implemented by using APIs. Follow the link below to implement this:

Session filtering

Appendix B: Troubleshooting Connectivity

Below are some common steps you can take to troubleshoot connectivity problems between Sensor Collector and Cisco Provider Connectivity Assurance.

Checking Sensor Collector Logs

To check Sensor Collector logs

1. Confirm Sensor Collector container is running:

sudo docker ps

CONTAINER ID | IMAGE | COMMAND | CREATED | STATUS | PORTS |NAMES

a1a7e7d0b0cc gcr.io/npav-172917/adh-roadrunner:latest "/docker-entrypoint.…"   2 hours ago Up About an hour 2222/tcp aod-connector-for-<some name>.npavlabs.accedian.net`

2. Tail logs:

sudo docker logs -f <container id>

2019/01/28 15:58:33 Using config file: /tmp/config/adh-roadrunner.yml
15:58:33.392 Run ▶ INFO 001 Runner pointing to <some name>.npavlabs.accedian.net:443
15:58:33.392 certExpiryChecker ▶ INFO 002 checking certs for renewal
15:58:33.393 certExpiryChecker ▶ INFO 003 no need for cert renewal
15:58:33.393 connect ▶ INFO 004 Attempting to establish meta connection with DH on host <some name>.npavlabs.accedian.net in zone: ChrisLab
15:58:33.393 GetCerts ▶ INFO 005 found cert and private key
15:58:33.632 connect ▶ INFO 006 Successfully connected to DataHub (meta connection) on host: <some name>.npavlabs.accedian.net in zone: ChrisLab.`

3. Check for errors.

Checking Cisco Provider Connectivity Assurance Reachability

By default, Sensor Collector requires outbound connectivity to Cisco Provider Connectivity Assurance via port 443 . Here are some things to check:

• Confirm the URL to which your Sensor Collector is trying to connect. This can be found in your Sensor Collector config file as the dhHost parameter. Note that this may not be the same URL that you're using to access your Cisco Provider Connectivity Assurance tenant.

cat /opt/roadrunner/adh-roadrunner.yml | grep dhHost

dhHost: "<some name>.npavlabs.accedian.net"

• Try to ping it. If reachable, move on to checking if the port is blocked.

ping <some name>.npavlabs.accedian.net

PING <some name>.npavlabs.accedian.net (111.222.222.111) 56(84) bytes of data.
64 bytes from 227.169.193.35.bc.googleusercontent.com (111.222.222.111): icmp_seq=1 ttl=63 time=35.7 ms`

• Try to Telnet to it on port 443. This port needs to be open for outbound connections by the customer's IT department.

telnet <some name>.npavlabs.accedian.net 443

Trying 111.222.222.111...
Connected to <some name>.npavlabs.accedian.net.

• Try to interact with port 443 via cURL:

> curl -fv https://<some name>.npavlabs.accedian.net

* Rebuilt URL to: https://xyz.npavlabs.accedian.net/
* Trying 35.239.xyz.xyz...
* TCP_NODELAY set
* Connected to xyz.npavlabs.accedian.net (35.239.xyz.xyz) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* Successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
<more text cut off for verbosity>

• We have seen cases where even though the cURL/Telnet test works, the container is still not able to reach the target and times out. In those cases it is recommended to restart (stop/start) the Docker service on your system, so it would reset proper routing rules internally. If using Linux, here is an example:

sudo service docker stop; sudo service docker start

or

sudo systemctl stop docker; sudo systemctl start docker

Appendix C: Dealing with Proxies

While Cisco Provider Connectivity Assurance will soon add support for proxy specification via the UI when configuring your connector/Sensor Collector, here's a workaround to add the right Docker environment variables to make it work.

You can either set the HTTPS_PROXYenvironment variable in your system environment before you run Sensor Collector, or you can add it to Sensor Collector's .env file. The .env file can be found at /opt/roadrunner/if the above instruction set was followed.

In both cases, your HTTPS_PROXYenvironment variable definition should look something like this:

HTTPS_PROXY=username:password@IP_Address_or_URL_of_proxy-server

GOOD: 
HTTPS_PROXY=someProxyURL.com

BAD: 
HTTPS_PROXY=https://someProxyURL.com

Appendix D: Troubleshooting Missing Data

After going through this document (including appendices A and B) you should now have a Sensor Collector, which is running and has connectivity to Cisco Provider Connectivity Assurance. If you're still not seeing data in your system, ensure the following:

• Make sure Polling Frequency is set to something relatively low (start with 60ms and adjust if necessary but generally not above 300ms)
• You are generating data for Sensor Collector to consume
• Your data is being generated in the directory with which Sensor Collector was configured

The directory the CSV files are being generated in MUST be the same directory you configured Sensor Collector with during the initial connector configuration (see figure below).

Appendix E: Troubleshooting issues with certificates, impacting data transfer

If while reviewing logs you get errors like the following, you must renew certificates in order to fix the problem:

12:17:38.118 connect   ERRO 009 Could not connect to DataHub (meta
connection) on host: dheu.npavlabs.accedian.net in zone: PoC_Lab.
Error: asn1: structure error: tags don't match (16 vs {class:0 tag:28
length:33 isCompound:true}) {optional:false explicit:false
application:false defaultValue:<nil> tag:<nil> stringType:0
timeType:0 set:false omitEmpty:false} certificate @2.

12:17:38.118 metaExecutionFunc   INFO 00a Attempting to Reconnect in 60
seconds

The solution is to delete the hidden directory of the certificates located in your Sensor Collector home folder (in this example it is /opt/roadrunner), in order to "force" a new creation of certificates like this:

sudo cd /opt/roadrunner; sudo rm -fr .rr_ssh

Finally, restart your Sensor Collector container again; this time it will prompt you to enter credentials to log into Cisco Provider Connectivity Assurance:

sudo ./run.sh