Data Aggregation

Rationale

The operations of statistical analysis require the storage of large amounts of data. Furthermore, that data must be stored over extended periods of time so as to expose overall trends. To minimize storage space while still making it possible to reveal trends over weeks or months, Cisco Provider Connectivity Assurance Sensors (previously SkyLIGHT PVX) automatically summarizes the collected data. The process of creating these summaries is called aggregation.

Process

Aggregation occurs automatically. Whenever your probe displays a chart or a table, this is based on already aggregated data. To display this, Cisco Provider Connectivity Assurance Sensors first decides on an aggregation granularity depending on the length of the time period you requested and how far back into the past it goes.

Note that because the higher aggregate levels summarize more data at once, they take up less disk space, and can be kept in storage much longer without filling out the hard drive. This strikes a good balance between data granularity and duration of retention: performance data with the best granularity will occupy much of your disk space, and long-lasting global trends can be exposed from far back (albeit with less detail), all from the same interface.

Requests

Generally, the chosen aggregate level is transparent for the user. The system will query the one that minimizes the computing effort for the same result. But when large intervals are requested, data can be degraded to guarantee a fast enough response latency.

Depending on your particular needs, it may be necessary to request more data points in graphs or to split a particular flow into smaller chunks in order to get a more precise timeline. The Mode option allows you to specify how the data granularity will automatically change depending on how far back you request, how many columns are requested (using themes) and how much filtering is applied to the page. Choosing the default Fast mode will report fewer data points or possible simpler data. The Precise mode will report more data points or more detailed data but may also extend your querying time. (See Precision mode for more details about the precision mode.)

If the data can be extended with more details, the option will be presented as a Refine button, that lets you ask the page to query again with a higher level of resources allocation. This will usually be slower (up to several minutes for some complex pages). This refinement is applied for the current page. When navigating to other pages, or when submitting a new search, this will default back to the current Mode (Fast or Precise). The same behavior applies when a page is added to a report. Note that increasing resource allocation will work with no guarantee and may trigger an error if there is not enough physical memory on the node that executes the query.