Skylight Analytics 20.06 Release Notes

Executive Summary

Another jam packed release with content that should make everyone happy!

• New portal user role and user group permission model that can enable you to start sharing access to your customers, or compartmentalize access on a need to know basis across internal teams
• Auto-refresh on dashboards is back enabling use of dashboards on NOC screens
• Counts on per packet intel data can now be used on dashboards so you can know the impact of poor performances (i.e. number of clients affected), enabling IT operations teams to better prioritize where to spend their effort
• Trend lines are back on our TopN lists, including a bump up to support Top 100s
• For deep investigations, we've improved how to pin Monitored Objects and bring them into our new Analysis module as a filter
• Along that same line, we've improved how you can use distributions within Analysis to find trends in metadata and quickly filter your investigation down to a root cause

Take it for a spin and let the team know via the Intercom chat if you have any questions!

Headline features

Portal User and User Group Permissions

• A Portal role has been added for users to limit user access to application areas and data

• User groups have been expanded to include Permissions. Users assigned to a user group with metadata permissions are restricted to the data that passes the metadata filter specified in the permissions (standard filtering of AND across keys and OR across values)

Analysis charting/distribution/filtering improvements

• Distributions have been improved in Analysis to update the metric distribution charts for the current metadata filters or currently hovered metadata value
• Metadata distribution value selections are gated and added to a filter control at the top of the Distribution. Gated filters can be applied to the active global analysis filters
• The header for each metadata distribution key has additional filter information and actions and can be used to quick-plot values
• Session-based metadata distributions show the filtered and total count of Monitored Objects per value
• Data from rows in the Tables section can also be used to quick-plot

Widgets in the Tables section have two parts:

• A TopN that supports quick filtering of TopN values (keeping other TopN values available for comparison and subsequent selection)
• A filtered list that explicitly lists the values present in the global analysis filter that correspond to that table's grouping

Global time controls, auto-refresh, step controls

• Global time controls have been added for the new Monitoring and Analysis areas
• Automatic refresh corresponding to the selected granularity can be enabled/disabled
• Step controls can be used to shift the current interval backward/forward; data within the interval will also be cached allowing for quick movement through previously visited intervals
• The state of the time controls is persisted to the URL and is shared between linked dashboards and analysis, including zoom intervals, so that the same context is retained when drilling down to move detail

Chart legend improvements / object counts

• If a grouping or filter split is present, the legend entries will be grouped under their associated metrics to save space
• Min/max will now appear in the chart when hovering the legend, better representing their values as points within the charted data
• Series can be disabled/enabled from the legend
• Unarchived object counts are present in the legend when series are grouped using metadata and are presented against the card according to the filters applied

Additional features - Monitoring

Dashboard export / import

• Primarily useful as a tool to transfer dashboards between tenants, but can also be used in a limited capacity to backup and restore dashboards
• Imports that fail model validation will display the validation failures
• If the validation failure is due to missing data dictionary entries (e.g. different metadata between tenants) the import tool can attempt a limited cleaning (removal) of the invalid portions of the model, allowing the dashboard import to complete. More appropriate data can then be selected

Dashboard section renaming

Drilldown trend lines and object selection for analysis

• Selected objects are transferred as filters into the new Analysis area

Heat map alerts

• Alerts raised and active alerts can be used as data for heat maps

Text widget

Additional Features: General

Table CSV export

Additional Features - Data Handling

Counts for categorical data (e.g. Server IP)

Utilization Computation

• As special computed metric that can be used in the same fashion as an observed metric (i.e. alert thresholds, heat maps)

Settings

Session filtering

• Ability to configure a filter to control what data gets sent up to Analytics using allow or drop filters.
• Session Filters can operate on exact or partial matching.
• One or more Session Filters can be added to a Session Filter Profile, which can then be assigned to one or more Connectors.
• By default, if a connector is not provided a Session Filter, everything is allowed through.
• Central managed via Analytics APIs - meant as a replacement for the CSV method of managing filters.

Settings navigation

• Things were getting a little crowded so we did some organization

Documentation

Capture documentation externally hosted

Change Log

[0.5]

Added new zone related fields:

• Added new field client.zone.id.
• Added new field server.zone.id.
• Added new field source.zone.id.
• Added new field dest.zone.id.
• Added new field client.error.zone.id.
• Added new field server.error.zone.id.
• Added new field source.error.zone.id.
• Added new field dest.error.zone.id.
• Added new field caller.zone.id.
• Added new field callee.zone.id.

Others:

• Added new field application.id.
• Add fields related to MD5 for HTTP Add new field request.payload.md5 Add new field response.payload.md5

Changed

• Fix using #(count) operator on complex fields.
• Add support for extended IP and MAC masks (<ip>/<ip>, <mac>/<mac>).

Zone related changes:

• Renamed field client.zone to client.zone.name.
• Renamed field server.zone to server.zone.name.
• Renamed field source.zone to source.zone.name.
• Renamed field dest.zone to dest.zone.name.
• Renamed field client.error.zone to client.error.zone.name.
• Renamed field server.error.zone to server.error.zone.name.
• Renamed field source.error.zone to source.error.zone.name.
• Renamed field dest.error.zone to dest.error.zone.name.
• Renamed field caller.zone to caller.zone.name.
• Renamed field callee.zone to callee.zone.name.

Other:

• Renamed field applicationto application.name.

[0.4] - 2020-05-27

Added

• Added new field capture.hostname.
• Added new field caller.label.
• Added new field callee.label.
• Added new field client.ja3.
• Added new field server.ja3.
• Added new field source.ja3.
• Added new field dest.ja3.

Changed

• Renamed field captureto capture.id.
• Clause FROMis now mandatory.
• Improve support for mac address querying:
  • Able to match a mac address using both a continuous and non continuous mask.
  • Create non continuous mask filtering for mac address.
  • Implement INoperation for mac address using a continuous mask.
• Order results in PVQL distinct sets.
• Add PVQL setting limit_size_set = 100.
• Implement glob/iglobfunctions for applications.
• Ignoring case when sorting by a string field.
• Fix can't query dicts as values.

Removed

• Removed field storage from public API.

Notable bug fixes

• alerts : Better guards against misconfiguration
• alerts : Query improvements (e.g. geospatial queries for historical/active alerts, metric filters)
• analytics: Display missing metadata in groupBys as '---'
• analytics: Filtering metadata used in a grouping on a dashboard impacts the grouping
• analytics: Normalize multi-unit charts on a per-metric basis instead of per-series
• analytics: Show per-row timeseries failures in grouped timeseries
• analytics: Reduce decimal places to one decimal for numbers above 0
• analytics: Clamp time to the display unit provisioned in settings (e.g. ms)
• analytics: Allow multiple data entries to be added to numeric widgets
• analytics: Adjust the look of sparklines for better trend clarity
• analytics: Increase the timeout for displaying a loading state
• analytics: Guard the additional of data for advanced computations
• analytics: Use save-as dialog when saving a custom analysis
• analytics: Better resize for widgets at the bottom of the screen
• analytics: Don't immediately author cloned cards for easier re-arrangement
• analytics: Don't immediately open the dictionary when adding multiple widgets
• analytics: Better resizing for widgets/cards
• analytics: Handle. in dashboard categories/names when linking dashboards
• analytics: Sort chart legends by descending average
• analytics: Show date/time on x-axis in full
• analytics: Fix NaN values in metric filters, allow adding free-form values from filter dropdown
• analytics: Improve table performance, better text measurements
• auth: Redirect to intended target post-login
• data: Don't automatically average traffic metrics in grouped capture data
• data: Support multi-source grouped capture data
• data : Better default source for capture data
• data: Support source selection for categorical capture data
• data: Pre-filter object types in session data groupBy requests
• data: Better display of more session data unit types (e.g. bandwidth)
• data: Better management of cached data requests, don't use stale entries
• data: Convert capture data metric filters to the source unit prior to querying
• data: Consolidate monitored object groupBy timeseries requests for better performance
• object-details: Display data cleaning history (incorrectly cached data)
• network: Restricted archiving to admins
• network: Handle network search input overflow
• settings: Lowercase session metadata when added/renamed
• settings: Upgrade alert policies using application names to id, use id for BCA rawValue
• legacy-dashboard: Keep metadata columns on card expansion/collapse