Deploying Roadrunner in Docker

System Requirements

• 100 GB disk space
• 4 CPU
• 2 GB RAM
• Docker 24.0.4 or later installed from the Docker software repositories and not a snap
• Tested operating systems: Debian 11 or higher, CentOS 7.3 or higher - should work with anything that supports Docker 24.0.4. You can follow this on how to install Docker on your system.
• No Firewall or if required, traffic to outside world on TCP port 443 (secured port for HTTPS) should be allowed.

Setting up your environment

To standardize deployments

We have come up with a directory structure to follow for deployments.

1. Create a directory for your Roadrunner software in /opt/
   > sudo mkdir -p /opt/roadrunner
2. Create a directory for your data in: /var/lib/data/
   > sudo mkdir -p /var/lib/data/roadrunner

Setting up Roadrunner Configuration

To set up Roadrunner configuration

1. Log in to your Skylight analytics deployment.

2. Navigate to Settings on the bottom left of the application navigation menu.

3. Navigate to Session setting menu at the top.

4. Roadrunner configurations are under the Connectors submenu.

5. Click the + button on the right to add a Roadrunner configuration.

6. Add a name for this connector.

7. Select your datasource type (FileWatcher for CSV upload).

8. Select the Path to your data which should match to the directory in your environment (i.e. /var/lib/data/roadrunner)

9. Specify a Zone name (MUST BE UNIQUE per connector)
   Sample Configuration:
   

10. Press the ✅ on the top-right to save the configuration.

11. Wait for the success message.

12. Download the RR by pressing the ellipsis (...) beside the connector you added on the table and select Download Connector Config.

Running Roadrunner

After you have downloaded the RR archive, it is time to extract it and run RR.

To run Roadrunner

1. Copy the newly downloaded 'connectornameyyyy-mm-dd'.tar.gz to your VM.
2. Move it to the path you have previously set up for RR software:

sudo mv 'connectornameyyyy-mm-dd'.tar.gz /opt/roadrunner/.*

3. Go to that directory:

cd /opt/roadrunner*

4. Extract the tar.gz file:

sudo tar xfv 'connectornameyyyy-mm-dd'.tar.gz*

5. Run Roadrunner:

sudo ./run.sh

On first startup, Roadrunner sets up M2M authentication and will require your Skylight analytics credentials. When it asks you for password, enter the same credentials that you used for logging in to DataHUB.

Upgrading Roadrunner

To upgrade Roadrunner

1. Go to your Skylight analytics portal (login if necessary).
2. Go to Settings ▶ Session ▶ Connectors.
3. Select the Connector you wish to upgrade
4. Click the connector's Ellipsis icon (...) on the row and choose to Download Connector Config.
5. Wait until the browser has downloaded the file. The file will be named: 'connectornameyyyy-mm-dd'.tar.gz.
6. Copy the downloaded file to your existing RoadRunner directory.
7. Now extract the downloaded file using the following command (this will overwrite the old RoadRunner docker image)

sudo tar xfv 'connectornameyyyy-mm-dd'.tar.gz*

4. Verify that your file sizes and dates have changed.
5. Move the old Roadrunner certs so it can be refreshed:

sudo mv .rr_ssh .rr_ssh.bk

6. Now you can restart Roadrunner by executing the run.sh file:

sudo ./run.sh*

7. (Optional) Clean up docker images after a successful major upgrade.
   For information, see docker image prune and docker image rm.

Appendix A: Filtering Sessions

Sometimes you don't always want Roadrunner to send all of the object data to Skylight analytics - you want it to be selective. We have whitelisting!

Session filtering is now implemented by using APIs. Please follow the link below to implement this:

Session filtering

Appendix B: Troubleshooting Connectivity

Below are some common steps you can take to troubleshoot connectivity problems between RoadRunner and Skylight analytics.

Checking Roadrunner Logs

To check Roadrunner logs

1. Confirm Roadrunner container is running:

sudo docker ps

CONTAINER ID | IMAGE | COMMAND | CREATED | STATUS | PORTS |NAMES

a1a7e7d0b0cc gcr.io/npav-172917/adh-roadrunner:latest "/docker-entrypoint.…"   2 hours ago Up About an hour 2222/tcp aod-connector-for-<some name>.npavlabs.accedian.net`

2. Tail logs:

sudo docker logs -f <container id>

2019/01/28 15:58:33 Using config file: /tmp/config/adh-roadrunner.yml
15:58:33.392 Run ▶ INFO 001 Runner pointing to <some name>.npavlabs.accedian.net:443
15:58:33.392 certExpiryChecker ▶ INFO 002 checking certs for renewal
15:58:33.393 certExpiryChecker ▶ INFO 003 no need for cert renewal
15:58:33.393 connect ▶ INFO 004 Attempting to establish meta connection with DH on host <some name>.npavlabs.accedian.net in zone: ChrisLab
15:58:33.393 GetCerts ▶ INFO 005 found cert and private key
15:58:33.632 connect ▶ INFO 006 Successfully connected to DataHub (meta connection) on host: <some name>.npavlabs.accedian.net in zone: ChrisLab.`

3. Check for errors.

Checking Skylight analytics Reachability

By default, Roadrunner requires outbound connectivity to Skylight analytics via port 443 . Here are some things to check:

• Confirm the URL to which your Roadrunner is trying to connect. This can be found in your Roadrunner config file as the dhHost parameter. Note that this may not be the same URL that you're using to access your Skylight analytics tenant.

cat /opt/roadrunner/adh-roadrunner.yml | grep dhHost

dhHost: "<some name>.npavlabs.accedian.net"

• Try to ping it. If reachable, move on to checking if the port is blocked.

ping <some name>.npavlabs.accedian.net

PING <some name>.npavlabs.accedian.net (111.222.222.111) 56(84) bytes of data.
64 bytes from 227.169.193.35.bc.googleusercontent.com (111.222.222.111): icmp_seq=1 ttl=63 time=35.7 ms`

• Try to Telnet to it on port 443. This port needs to be open for outbound connections by the customer's IT department.

telnet <some name>.npavlabs.accedian.net 443

Trying 111.222.222.111...
Connected to <some name>.npavlabs.accedian.net.

• Try to interact with port 443 via cURL:

> curl -fv https://<some name>.npavlabs.accedian.net

* Rebuilt URL to: https://xyz.npavlabs.accedian.net/
* Trying 35.239.xyz.xyz...
* TCP_NODELAY set
* Connected to xyz.npavlabs.accedian.net (35.239.xyz.xyz) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* Successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
<more text cut off for verbosity>

• We have seen cases where even though the cURL/Telnet test works, the container is still not able to reach the target and times out. In those cases it is recommended to restart (stop/start) the Docker service on your system, so it would reset proper routing rules internally. If using Linux, here is an example:

sudo service docker stop; sudo service docker start

or

sudo systemctl stop docker; sudo systemctl start docker

Appendix C: Dealing with Proxies

While Skylight analytics will soon add support for proxy specification via the UI when configuring your connector/Roadrunner, here's a workaround to add the right Docker environment variables to make it work.

You can either set the HTTPS_PROXYenvironment variable in your system environment before you run Roadrunner, or you can add it to Roadrunner's .env file. The .env file can be found at /opt/roadrunner/if the above instruction set was followed.

In both cases, your HTTPS_PROXYenvironment variable definition should look something like this:

HTTPS_PROXY=username:password@IP_Address_or_URL_of_proxy-server

GOOD: 
HTTPS_PROXY=someProxyURL.com

BAD: 
HTTPS_PROXY=https://someProxyURL.com

Appendix D: Troubleshooting Missing Data

After going through this document (including appendices A and B) you should now have a Roadrunner which is running and has connectivity to Skylight analytics. If you're still not seeing data in your system, please ensure the following:

• Make sure Polling Frequency is set to something relatively low (start with 60ms and adjust if necessary but generally not above 300ms)
• You are generating data for Roadrunner to consume
• Your data is being generated in the directory with which Roadrunner was configured

The directory the CSV files are being generated in MUST be the same directory you configured Roadrunner with during the initial connector configuration (see figure below).

Appendix E: Troubleshooting issues with certificates, impacting data transfer

If while reviewing logs you get errors like the following, you must renew certificates in order to fix the problem:

12:17:38.118 connect   ERRO 009 Could not connect to DataHub (meta
connection) on host: dheu.npavlabs.accedian.net in zone: PoC_Lab.
Error: asn1: structure error: tags don't match (16 vs {class:0 tag:28
length:33 isCompound:true}) {optional:false explicit:false
application:false defaultValue:<nil> tag:<nil> stringType:0
timeType:0 set:false omitEmpty:false} certificate @2.

12:17:38.118 metaExecutionFunc   INFO 00a Attempting to Reconnect in 60
seconds

The solution is to delete the hidden directory of the certificates located in your Roadrunner home folder (in this example it is /opt/roadrunner), in order to "force" a new creation of certificates like this:

sudo cd /opt/roadrunner; sudo rm -fr .rr_ssh

Finally, restart your RoadRunner container again; this time it will prompt you to enter credentials to log into Skylight analytics:

sudo ./run.sh