Deploying Telemetry Collector
Setting up Your Telemetry Collector Working Directory
Create a working directory for your Telemetry Collector to operate from.
In that directory, you will create a docker-compose.yml file that resembles the following:
services:
telemetry-collector:
container_name: "telemetry-collector"
image: "gcr.io/sky-agents/agent-telemetry-amd64:r24.09"
environment:
AGENT_MANAGEMENT_PROXY: "{{ sensorcollector-ip }}"
AGENT_MANAGEMENT_PROXY_PORT: 55777
ports:
- "57000:57000"
volumes:
- ./secrets/secrets.yaml:/run/secrets/secrets.yaml
Notes:
AGENTMANAGEMENTPROXYcorresponds to your local Sensor Collector. The{{ sensorcollector-ip }}placeholder should be replaced with the IP of your deployed Sensor Collector.AGENTMANAGEMENTPROXY_PORTcorresponds to the Agent Proxy port you chose when creating your Sensor Collector configuration.55777is the default value. However, if you choose something else, then this must match.- You will notice we created a volume mount which references a file on the host called
./secrets/secrets.yaml. Don’t worry—we’ll create that file in the next step.
Retrieve Your Telemetry Collector Secrets
Your Telemetry Collector still needs one thing before we can successfully start it: the secrets.yaml file containing its agent identifier, and an initial Java Web Token (JWT) for its handshake when connecting to the Management web socket in Sensor Collector.
There are two options for creating the secrets.yaml file, via either the User Interface method or API method.
User Interface Instructions
Selecting the Generate Auth Token button on the Sensors ▶ Telemetry Collector page will create the file contents for you. You must copy the results (see screenshots below), create a secrets.yaml file, and store it in the path that you specified in the Docker compose file in the previous step (i.e. ./secrets/secrets.yaml:/run/secrets/secrets.yaml).
Starting the Telemetry Collector
You are now ready to start your Telemetry Collector:
❯ docker compose up -d
If all went as expected, you should see Telemetry Collector logs indicating that your Telemetry Collector has started and connected both its Management and Data links to your Sensor Collector.
Below are some logs to monitor; these show our blue box app starting (telemetry is our application):
telemetry-agent | 2024-04-12T15:44:05.918387+00:00 info telemetry Blue Box application Telemetry Collector
telemetry-agent | 2024-04-12T15:44:05.923348+00:00 info monitor Received signal from ciscomdt, continuing start sequence!
telemetry-agent | 2024-04-12T15:44:05.923435+00:00 info monitor Starting Telemetry
telemetry-agent | 2024-04-12T15:44:05.923552+00:00 info telemetry Blue Box application starting on agent c390d231-1376-4aae-8a7b-59c0a77ed85e.
telemetry-agent | 2024-04-12T15:44:05.923858+00:00 info telemetry Waiting for system
telemetry-agent | 2024-04-12T15:44:05.924103+00:00 info monitor Startup sequence completed
These show the Telemetry Collector successfully establishing its management connection with Sensor Collector:
telemetry-agent | 2024-04-12T15:44:09.515127+00:00 info management Connecting to management server: host.docker.internal:55777; Attempt: 1
telemetry-agent | 2024-04-12T15:44:09.632144+00:00 info management Connection to management established: Compression is Enabled
telemetry-agent | 2024-04-12T15:44:12.933773+00:00 info telemetry ix Subscribe topic service/telemetry/config client telemetry
telemetry-agent | 2024-04-12T15:44:15.740444+00:00 info agentHealth Starting CPU monitoring thread
telemetry-agent | 2024-04-12T15:44:16.528192+00:00 info management Authenticating to management server: host.docker.internal:55777; Attempt: 1
telemetry-agent | 2024-04-12T15:44:16.532611+00:00 info management Sending identification
telemetry-agent | 2024-04-12T15:44:16.532694+00:00 info management Waiting for configuration
telemetry-agent | 2024-04-12T15:44:16.980984+00:00 info management Authentication successful!
These show the Telemetry Collector sending a configuration request to Provider Connectivity Assurance over the Sensor Collector Management link and getting back its configuration:
telemetry-agent | 2024-04-12T15:44:17.239656+00:00 info management Received configuration Update from management: TenantId: 9ca4447b-e32a-4dc2-af71-2a1219cc3373; Revision: 150da539-2ca2-4b9e-90bd-fdeb3da97939
telemetry-agent | 2024-04-12T15:44:17.240925+00:00 info data Configuration update: Server: 'host.docker.internal'; Port: '55888'; Use SSL: true; Allow Self Signed; Allow Invalid Certs; Allow Expired Certs; Skip Cert Hostname Check;
telemetry-agent | 2024-04-12T15:44:17.242900+00:00 info data Updated metadata
telemetry-agent | 2024-04-12T15:44:17Z I! Config file modified
telemetry-agent | 2024-04-12T15:44:17Z I! Reloading Telemetry config
telemetry-agent | 2024-04-12T15:44:17.949130+00:00 info telemetry applying new Telemetry configuration: [[inputs.cisco_telemetry_mdt]] ## Telemetry transport can be "tcp" or "grpc". TLS is only supported when ## using the grpc transport. transport = "tcp" ## Address and port to host telemetry listener service_address = ":57000" ## Grpc Maximum Message Size, default is 4MB, increase the size. max_msg_size = 104857600 # 100 MB in bytes ## Enable TLS; grpc transport only. # tls_cert = "/etc/Telemetry/cert.pem" # tls_key = "/etc/Telemetry/key.pem" ## Enable TLS client authentication and define allowed CA certificates; grpc ## transport only. # tls_allowed_cacerts = ["/etc/Telemetry/clientca.pem"] ## Define (for certain nested telemetry measurements with embedded tags) which fields are tags embedded_tags = [ "Cisco-IOS-XR-man-ipsla-oper:ipsla/operation-data/operations/operation/statistics/latest/target/specific-stats/op-type", "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/input/service-policy-names/service-policy-instance/statistics/class-stats/class-name", "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/input/service-policy-names/service-policy-instance/statistics/class-stats/child-policy/policy-name", "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/input/service-policy-names/service-policy-instance/statistics/class-stats/child-policy/class-stats/class-name", "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics/class-stats/class-name", "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics/class-stats/child-policy/policy-name", "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics/class-stats/child-policy/class-stats/class-name", ] ## Define aliases to map telemetry encoding paths to simple measurement names [inputs.cisco_telemetry_mdt.aliases] if_stats = "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces/interface-xr/interface" policy_out_stats = "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics" policy_in_stats = "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/input/service-policy-names/service-policy-instance/statistics" sla_stats = "Cisco-IOS-XR-infra-sla-oper:sla/protocols/Cisco-IOS-XR-ethernet-cfm-oper:ethernet/statistics-historicals/statistics-historical" cpu_util = "Cisco-IOS-XR-wdsysmon-fd-oper:system-monitoring/cpu-utilization" node_summary = "Cisco-IOS-XR-nto-misc-oper:memory-summary/nodes/node/summary" channel_stats = "Cisco-IOS-XR-telemetry-model-driven-oper:telemetry-model-driven/channel-statistics" twamp_lite = "Cisco-IOS-XR-perf-meas-oper:performance-measurement/nodes/node/interfaces/interface-details/interface-detail" ipsla = "Cisco-IOS-XR-man-ipsla-oper:ipsla/operation-data/operations/operation/statistics/latest/target" [[processors.template]] order = 1 namepass = ["if_stats*"] tag = "sessionName" template = '{{ .Tag "source" }}_{{ .Tag "interface_name" }}' [[processors.template]] order = 2 namepass = ["if_stats*"] tag = "sessionId" template = '{{ .Tag "source" }}_{{ .Tag "interface_name" }}' [[processors.template]] order = 3 namepass = ["if_stats*"] tag = "objectType" template = 'cisco_mdt_interface' [[processors.template]] order = 3 namepass = ["if_stats*"] tag = "direction" template = "0" [[processors.starlark]] order = 4 namepass = ["if_stats*"] source = ''' def apply(metric): if metric.fields.get("state") == "im-state-up": metric.fields["availability_percent"] = 100 else: metric.fields["availability_percent"] = 0 if metric.fields.get("interface_statistics/stats_type") == "basic": metric.fields["interface_statistics/basic_interface_stats/bytes_received_rate"] = metric.fields.get("interface_statistics/basic_interface_stats/bytes_received") metric.fields["interface_statistics/basic_interface_stats/bytes_received_delta"] = metric.fields.get("interface_statistics/basic_interface_stats/bytes_received") metric.fields["interface_statistics/basic_interface_stats/bytes_sent_rate"] = metric.fields.get("interface_statistics/basic_interface_stats/bytes_sent") metric.fields["interface_statistics/basic_interface_stats/bytes_sent_delta"] = metric.fields.get("interface_statistics/basic_interface_stats/bytes_sent") elif metric.fields.get("interface_statistics/stats_type") == "full": metric.fields["interface_statistics/full_interface_stats/bytes_received_rate"] = metric.fields.get("interface_statistics/full_interface_stats/bytes_received") metric.fields["interface_statistics/full_interface_stats/bytes_received_delta"] = metric.fields.get("interface_statistics/full_interface_stats/bytes_received") metric.fields["interface_statistics/full_interface_stats/bytes_sent_rate"] = metric.fields.get("interface_statistics/full_interface_stats/bytes_sent") metric.fields["interface_statistics/full_interface_stats/bytes_sent_delta"] = metric.fields.get("interface_statistics/full_interface_stats/bytes_sent") return metric ''' # Template config pertaining to Environment Objects # Template for constructing object ID for node environmentals [[processors.template]] namepass = ["cpu_util*", "node_summary*"] tag = "sessionName" template = '{{ .Tag "source" }}_{{ .Tag "node_name" }}' [[processors.template]] namepass = ["cpu_util*", "node_summary*"] tag = "sessionId" template = '{{ .Tag "source" }}_{{ .Tag "node_name" }}' [[processors.template]] namepass = ["cpu_util*", "node_summary*"] tag = "objectType" template = 'cisco_mdt_environment' [[processors.template]] namepass = ["cpu_util*", "node_summary*"] tag = "direction" template = '-1' [[processors.rename]] order = 5 namepass = ["if_stats"] [[processors.rename.replace]] field = "interface_statistics/basic_interface_stats/input_errors" dest = "interface_statistics/full_interface_stats/input_errors" [[processors.rename.replace]] field = "interface_statistics/basic_interface_stats/output_errors" dest = "interface_statistics/full_interface_stats/output_errors" [[processors.rename.replace]] field = "interface_statistics/basic_interface_stats/packets_received" dest = "interface_statistics/full_interface_stats/packets_received" [[processors.rename.replace]] field = "interface_statistics/basic_interface_stats/packets_sent" dest = "interface_statistics/full_interface_stats/packets_sent" [[processors.rename.replace]] field = "interface_statistics/basic_interface_stats/input_drops" dest = "interface_statistics/full_interface_stats/input_drops" [[processors.rename.replace]] field = "interface_statistics/basic_interface_stats/output_drops" dest = "interface_statistics/full_interface_stats/output_drops" [[processors.rename.replace]] field = "interface_statistics/basic_interface_stats/bytes_received_rate" dest = "interface_statistics/full_interface_stats/bytes_received_rate" [[processors.rename.replace]] field = "interface_statistics/basic_interface_stats/bytes_received_delta" dest = "interface_statistics/full_interface_stats/bytes_received_delta" [[processors.rename.replace]] field = "interface_statistics/basic_interface_stats/bytes_sent" dest = "interface_statistics/full_interface_stats/bytes_sent" [[processors.rename.replace]] field = "interface_statistics/basic_interface_stats/bytes_sent_rate" dest = "interface_statistics/full_interface_stats/bytes_sent_rate" [[processors.rename.replace]] field = "interface_statistics/basic_interface_stats/bytes_sent_delta" dest = "interface_statistics/full_interface_stats/bytes_sent_delta" [[processors.rename.replace]] field = "interface_statistics/basic_interface_stats/bytes_received" dest = "interface_statistics/full_interface_stats/bytes_received" # Restricts the number of tags that can pass through this filter and chooses which tags to preserve when over the limit. [[processors.tag_limit]] order = 6 ## Maximum number of tags to preserve limit = 4 ## List of tags to preferentially preserve keep = ["sessionId", "sessionName", "objectType","direction"]
telemetry-agent | 2024-04-12T15:44:17Z I! Config watcher started
telemetry-agent | 2024-04-12T15:44:17Z I! Config watcher started
telemetry-agent | 2024-04-12T15:44:17Z I! Loading config: /etc/Telemetry/Telemetry.conf
These show the Telemetry Collector successfully establishing its data connection with Sensor Collector:
telemetry-agent | 2024-04-12T15:44:17.982056+00:00 info data Connecting to data broker: host.docker.internal:55888; Attempt: 0
telemetry-agent | 2024-04-12T15:44:17Z I! Loading config: /etc/Telemetry/Telemetry.d/cisco_mdt_dialout.conf
telemetry-agent | 2024-04-12T15:44:17Z I! Starting Telemetry 1.29.4 brought to you by InfluxData the makers of InfluxDB
telemetry-agent | 2024-04-12T15:44:17Z I! Available plugins: 241 inputs, 9 aggregators, 30 processors, 24 parsers, 60 outputs, 6 secret-stores
telemetry-agent | 2024-04-12T15:44:17Z I! Loaded inputs: cisco_telemetry_mdt
telemetry-agent | 2024-04-12T15:44:17Z I! Loaded aggregators:
telemetry-agent | 2024-04-12T15:44:17Z I! Loaded processors: rename starlark tag_limit template (8x)
telemetry-agent | 2024-04-12T15:44:17Z I! Loaded secretstores:
telemetry-agent | 2024-04-12T15:44:17Z I! Loaded outputs: health websocket
telemetry-agent | 2024-04-12T15:44:17Z I! Tags enabled: host=telemetry-agent
telemetry-agent | 2024-04-12T15:44:17Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"telemetry-agent", Flush Interval:10s
telemetry-agent | 2024-04-12T15:44:17Z I! [outputs.health] Listening on http://[::]:57001
telemetry-agent | 2024-04-12T15:44:17.998119+00:00 info telemetry Received connection request from Telemetry
telemetry-agent | 2024-04-12T15:44:17.999413+00:00 info telemetry Listener connected to Telemetry instance
telemetry-agent | 2024-04-12T15:44:18.026599+00:00 info data Connection to data broker established: Compression is Enabled
telemetry-agent | 2024-04-12T15:44:18.026641+00:00 info data Waiting batches: 0
telemetry-agent | 2024-04-12T15:44:24.924867+00:00 info data Authenticating data path: Attempt: 1
telemetry-agent | 2024-04-12T15:44:28.034204+00:00 info data Broker message chunck size changed to 3000.
telemetry-agent | 2024-04-12T15:44:28.039446+00:00 info data Sent authentication request, waiting for confirmation.
telemetry-agent | 2024-04-12T15:44:28.687240+00:00 info data Authentication successful!
These show the embedded Telemetry instance successfully starting:
telemetry-agent | 2024-04-12T15:44:17Z I! Loading config: /etc/Telemetry/Telemetry.conf
telemetry-agent | 2024-04-12T15:44:17Z I! Loading config: /etc/Telemetry/Telemetry.d/cisco_mdt_dialout.conf
telemetry-agent | 2024-04-12T15:44:17Z I! Starting Telemetry 1.29.4 brought to you by InfluxData the makers of InfluxDB
telemetry-agent | 2024-04-12T15:44:17Z I! Available plugins: 241 inputs, 9 aggregators, 30 processors, 24 parsers, 60 outputs, 6 secret-stores
telemetry-agent | 2024-04-12T15:44:17Z I! Loaded inputs: cisco_telemetry_mdt
telemetry-agent | 2024-04-12T15:44:17Z I! Loaded aggregators:
telemetry-agent | 2024-04-12T15:44:17Z I! Loaded processors: rename starlark tag_limit template (8x)
telemetry-agent | 2024-04-12T15:44:17Z I! Loaded secretstores:
telemetry-agent | 2024-04-12T15:44:17Z I! Loaded outputs: health websocket
telemetry-agent | 2024-04-12T15:44:17Z I! Tags enabled: host=telemetry-agent
telemetry-agent | 2024-04-12T15:44:17Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"telemetry-agent", Flush Interval:10s
telemetry-agent | 2024-04-12T15:44:17Z I! [outputs.health] Listening on http://[::]:57001
telemetry-agent | 2024-04-12T15:44:17.998119+00:00 info telemetry Received connection request from Telemetry
telemetry-agent | 2024-04-12T15:44:17.999413+00:00 info telemetry Listener connected to Telemetry instance
At this point, if all went as intended, your Telemetry Collector is running and connected to Provider Connectivity Assurance via the Sensor Collector instance that you have deployed.