Dynamic Metadata Mapping

Overview

Metadata is fundamental to the usefulness of an analytics platform. Without metadata, you're left with flat, uncorrelatable data. Skylight performance analytics offers the ability to provision metadata against sessions via APIs or via the web UI. Users leverage these methods to enrich performance data with added context. To learn more about metadata, see topics in Metadata.

The challenge is keeping metadata that changes more frequently up-to-date. This is usually information that is generated or discovered closer to the source of the performance data, rather than from a BSS or OSS system that runs higher in the IT stack.

To solve this problem, we've introduced the ability to map contextual data that comes in from the network alongside your performance metrics to your existing metadata categories, and enable it to be updated or changed automatically.

The Dynamic mappings settings page allows administrators to link contextual fields from the PM data sources to session metadata to allow the values to be auto-populated. These controls are on a per object type basis.

How it Works

When raw performance data comes in for each session, it usually contains some informational/non-PM columns that provide context to the measurements - this is essentially network metadata. Some of these informational columns are required to create the session (for example, ID name, type) but the other metadata columns, including OpenMetrics labels, are discarded.

This feature allows you to use these network-provided contextual data and map them to existing metadata categories.

Take note of these important points:

• With this mapping in place, metadata values for any current sessions, as well as any new sessions in the future, will get automatically provisioned with incoming data.
• Since the values are coming in with incoming PM data, over time, the mapped dynamic metadata values will be updated any time their value changes.
• Mapping also allows datasources (object types) with different field names to be mapped to the same metadata category, allowing for flexibility when datasource ingestion naming conventions do not align.

Mappings List

In the Dynamic mappings list, you can see fields that are mapped to the provisioned categories (→ icon). For a given datasource and field, you can map only one category.

To view the mappings list:

1. Go to Settings -> Metadata -> Session.
2. Click the Dynamic mappings tab.

Adding a New Mapping

To add a new mapping:

1. Click Add mapping in the Mappings column of the Datasource and Fields row that you’d like to map.
2. From the Categories dropdown, choose a category by clicking on it. You can search for a category by typing its name.

3. The mapping is added with *New mapping status on the right side.

4. Click the Submit button to save changes.

5. The mapping is saved. The modification time and date appears on the right side.

Edit a Mapping

To edit a mapping:

1. Click the mapping cell on the row you want to edit.
2. Choose a new category from the dropdown or choose None to clear the mapping for the given field.

3. To the right of the mapping, *New mapping (if you changed the category) or *Removed (if you chose None option for clear) displays.

4. Click the Submit button to save the changes.

Import/Export Mappings

Dynamic metadata mappings can be exported and imported within the mappings page.

The Import mappings and Export mappings menu options can be accessed by clicking the more (…) button at the top-right of the page.

Exports and imports are conducted using the following JSON format:

{
	"datasource": {
		"field": "category",
		"field2": "category2"
	},
	...
}

Active dynamic mappings that are currently displayed in the table can be exported to a JSON file (in the format above).

To export to a JSON file:

1. Click Export mappings in the menu.
2. With the use of a JSON file with the format specified above, the dynamic mappings to be imported can be reviewed and filtered before being imported into the dynamic mappings list.
3. Clicking Import mappings in the menu opens the file explorer.
4. Choose a JSON file (in the format described above) that contains dynamic mappings to be imported.
5. Choose dynamic mappings to be imported with selection of the import modal table’s rows.
6. Click the Import button on the modal.

The right-most column of the Import mappings modal table represents how the method that the corresponding row (the dynamic mapping) will be inserted or the manner in which it is invalid. Only valid mappings will be imported.

• No change: the dynamic mapping already exists. Consequently, importing the row will produce no changes to the existing dynamic mappings list.
• Replaced: the field is already linked to a category or the category is already linked to another field. Importing will unlink the existing dynamic mapping described above and link the new mapping specified by the corresponding row.
• Linked: the dynamic mapping specified in the row will be created.
• Unknown field(invalid): the field in the row is unknown.
• Unknown datasource(invalid): the datasource in the row is unknown.
• Duplicate category(invalid): the category is already linked to another (to be imported) row’s field of the same datasource.

Inventory Confirmation

Once a mapping is configured, go to the Inventory → Sessions page and select an active session that should be streaming the given field from the mapping with the same datasource as your mapping. Depending on ingestion and mapping update cadences, the new dynamic metadata values will begin to be populated for the given datasource sessions.