Fault Management Configuration Guide

Provider Connectivity Assurance provides comprehensive Fault collection, alerting, visualization, and correlation. This article provides step-by-step instructions to configure the Fault Management feature within Provider Connectivity Assurance.

Solution Architecture

Fault management in Provider Connectivity Assurance consists of a collector that receives SNMP traps from network devices and forwards them to Provider Connectivity Assurance for correlation and visualization, and optionally to OSS systems.

The Fault feature is enabled within the Fault & Mobility PM Collector, which includes its own configuration UI in release 25.07 (to be integrated directly into Provider Connectivity Assurance in a future release). The architecture is depicted below:

Configuration Procedure

To configure Fault collection, alerting, and visualization in Provider Connectivity Assurance, and to deliver faults to northbound systems, follow these steps:

1. Enable the Observability Framework within the Fault & Mobility PM Collector.

2. Add SNMP device credentials to the configuration file in the Fault & Mobility PM Collector to enable device connectivity and trap collection.

3. Set up sending of faults to northbound integrated systems.

4. Enable the OTEL-based data flow from the Fault & Mobility PM Collector to Provider Connectivity Assurance.

Step 1: Enable the Observability Framework

The Observability Framework processes SNMP traps from devices, enriches the data, and sends Faults to Provider Connectivity Assurance.

To enable the Observability Framework plug-in using the configuration UI

At this stage, the Observability Framework is enabled.

Step 2: Add SNMP Device Credentials to snmptrapd.conf

Prerequisite: Ensure all monitored SNMP devices are configured and reachable from the Fault & Mobility PM Collector.
Add SNMP credentials to vmount/snmptrapd/snmptrapd.conf. See below for sample configurations.

For SNMPv2:

# Example SNMPv2 entry
rocommunity public

For SNMPv3:

# Example SNMPv3 entry
createUser username SHA "authpass" AES "privpass"
rouser username

After adding credentials, restart the SNMP trap daemon service:

systemctl restart matrix-of-snmptrapd

Step 3: Set Up Sending Faults to Northbound Integrated Systems

This section explains how to configure northbound integration to OSS via SNMP (SNMPv3 only).

Configure ITSM Plugin

To configure the ITSM plugin

1. From the side navigation bar, navigate to Admin ▶ Tenant ▶ App Plugins ▶ Add New Tenant Plugin.

2. Select Tenant, add ITSM.

3. Click Save.

4. Refresh the page.

Configure Handler in Northbound Integration Pipeline

To configure the new handler

1. From the side navigation bar, navigate to Admin ▶ Data Pipeline ▶ Handlers ▶ Add New Handler.

2. On the Add New Handler page, configure the parameters as shown below:
   

3. Click Save to apply the new Handler configuration.

4. Refresh the page.

Configure Consumers in Northbound Integration Pipeline

To configure consumers

1. From the side navigation bar, navigate to Admin ▶ Data Pipeline ▶ Consumers ▶ Add New Consumer.

2. On the Add New Consumer page, configure the parameters as shown below:

3. Click Save to apply the new consumer configuration.

4. Refresh the page.

Example Consumer Configuration:

Topic: <global>.snmp
Handler: SnmpHandler
Data Schema: SnmpSchema
Schema Mapping:
{
  "alert_end_ts": "alert_end_ts",
  "alert_type": "status",
  "clear_event": "clear_event",
  "clearing_required": "clearing_required",
  "source_ip": "ip_address"
}

Topic: <global>.processchain_snmp
Handler: ItsmSnmpHandler
Data Schema: SnmpSchema
Schema Mapping:
{
  "alert_end_ts": "alert_end_ts",
  "alert_type": "status",
  "clear_event": "clear_event",
  "clearing_required": "clearing_required",
  "source_ip": "ip_address"
}

Add Northbound Integration for SNMP

To add a new integration

1. From the Fault & Mobility PM Collector UI, navigate to Admin ▶ ITSM ▶ Northbound Integration (top panel).

2. Click the Add icon.

3. Select the type and complete required fields as shown below:

4. Click Save.

Alarm Resync

Use the Alarm Resync feature to resend failed alarms to specific or all ITSM API POST instances. Enable resync in the Northbound Rule section of ITSM. If not enabled, failed alerts will be ignored.
Note: Only one resync request can be in progress at a time.

To get the required cookie, visit https://<webapp-ip>/api/v1/iam/token and include it in your alarm resync API request.

POST https://<webapp-ip>/api/v1/itsm/alarm-resync/
{
  "instance":[]
}

{
  "instance":["instance_name_created_in_ui"]
}

{
  "instance":["instance_name_created_in_ui", "another_instance_name"]
}

{
  "instance":["instance_name_created_in_ui"],
  "force": true
}

Step 4: Enable the OTEL Pipeline to Deliver Faults to Provider Connectivity Assurance

Enable the OTEL data pipeline to send faults from the Fault & Mobility PM Collector to Provider Connectivity Assurance.

Prerequisites:

Ensure the following services are running:

• otel-transformer

• otel-load-balancer

• otel-collector-1

Make the configuration changes below to the respective services.

matrix-of-alertservice

Edit vmount/credentials.json to enable HTTPS connection forwarding to otel-transformer.

Example configuration:

{
  "services": {
    "kafka": {
      ...
      "connections": [
        {
          "topic": "tenantscale.snmp",
          "server": "<Kafka-HOST-IP>:9093",
          "cafile": "/app/ssl/ca-cert",
          "keyfile": "/app/ssl/cert.pem"
        }
      ]
    },
    "https": {
      "factory": "HttpConnectionFactory",
      "connections": [
        {
          "url": "http://<Observability framework-HOST-IP>:8087/api/alerts/process-json",
          "auth_type": "basic",
          "username": "<USER>",
          "password": "<PASSWORD>",
          "send_flag": true
        }
      ]
    }
  },
  "number_of_workers": 150
}

otel-transformer

Ensure the following environment variables are set to forward traffic to the internal otel-load-balancer:

OTEL_TRANSFORMER_USERNAME=<USER>
OTEL_TRANSFORMER_PASSWORD=<PASSWORD>
OTEL_LOAD_BALANCER_ENDPOINT=http://otel-load-balancer:8080

otel-load-balancer

Ensure vmount/otel_nginx_conf/nginx.conf includes the otel-collector-1 receiver endpoints.

otel-collector-1

In vmount/otel_collector/otel-collector-config.yaml, configure the Provider Connectivity Assurance endpoint and access token:

receivers:
  otlp:
    protocols:
      http:
        endpoint: "0.0.0.0:4318"

exporters:
  otlphttp:
    endpoint: "<PCA_ENDPOINT_URL>"
    headers:
      Authorization: "Bearer <ACCESS_TOKEN>"
      Content-Type: "application/json"

Access Token Creation

To create the access token

1. Use Zitadel Admin UI at auth.<deploymentURL> or by IP/port 3443.

2. Switch to the target tenant organization.

3. Under Users, create a new Service User.

4. Assign the “tenant-admin” role (required for deployments ≥ 25.2.232).

5. Generate a Personal Access Token (set an appropriate expiration date).

6. Use the generated token as the Authorization header in the OTEL exporter configuration.

For further details, see: HOWTO: PCA with OTEL collector

At this stage, faults are fully configured to flow to Provider Connectivity Assurance and can be viewed in dashboards.