NetFlow

Introduction

NetFlow collects various statistics about the network traffic passing through network equipment. Not all network equipment allows exporting statistics as NetFlow.

The major NetFlow versions in use are:

• NetFlow v5
• NetFlow v9
• IPFIX

NetFlow protocols were first established by Cisco® but are not actual standards.

NetFlow v5 is the most common version, providing some basic statistics export but is inherently limited to IPv4.

NetFlow v9 is the next major version; its strength and complexity rely on its template-based approach to delivering statistics. It has been described in an informational RFC3954.

IPFIX, also known under the "NetFlow v10" name, is largely inspired by NetFlow v9 but the protocol itself isn’t quite compatible. IPFIX has been standardized in RFC5101.

Skylight capture: sensor and Equipment Configuration

The equipment should aggregate flows based on time rather than sending per-packet or per-event flows (as capture: sensor has no reliable way to understand the event that caused the flow to be sent in the case of NetFlow v9 and IPFIX).

Skylight capture: sensor uses the timestamps from the NetFlow packets, it is required that those are accurate. Configuring NTP on Skylight capture: sensor and your equipment is a reliable way to ensure everything is synchronized.

Moreover, the equipment’s NetFlow configuration should ensure that flows are sent, at least, every minute. The equipment should not be configured to send flows less frequently (for example, every 10 minutes). Higher rates than 1 minute are supported, like 30 seconds, Skylight capture: sensor will then aggregate them as it does for sniffed flows.

The sniffer is configured to listen for other ports on the Nodes Management page. By default, it listens on the standard UDP port 2055 and 4739. For completeness, it also listens on UDP ports 9555 and 9995.

NetFlow v5

Skylight capture: sensor can easily retrieve and integrate some of the statistics reported by NetFlow v5 because they are static. Netflow v5 does not support IPv6.

The following fields are used by Skylight capture: sensor.

Fields used by Skylight capture: sensor from the NetFlow v5 header to export statistics:

Fields used by Skylight capture: sensor from the NetFlow v5 flow data to export statistics:

ICMP changes the meaning of the ports; the destination port encodes both the ICMP type and codes (as ICMP type * 256 + ICMP code).

NetFlow v9 and IPFIX

The statistics reported by NetFlow v9 and IPFIX are more numerous and more complex compared to NetFlow v5; approximately 500 are standardized and vendors may extend them.

The complete list of standardized IPFIX entities is available at IANA. NetFlow v9 entities declared in the RFC are compatible with IPFIX entities and, as such, their IANA or RFC name will be used in this article.

Templates, permitting extensibility of NetFlow v9 and IPFIX, are special flows that describe which entities other flows can contain. The sniffer keeps templates in memory for each NetFlow sender; flows will be dropped if their template has not been seen. The NetFlow sender could either send templates regularly and separately or with other flows.

You should configure your templates to include entities described below.

Fields used by Skylight capture: sensor from the NetFlow v9 header to export statistics:

Fields used by Skylight capture: sensor from the IPFIX header to export statistics:

Required time fields used by Skylight capture: sensor from the NetFlow v9 or IPFIX flow data to export statistics:

The sniffer supports having the flowStartSysUpTime sent within an optional record and the other timestamps as part of the flow data when in IPFIX.

Skylight capture: sensor handles different fields from the NetFlow v9 or IPFIX flow data to export statistics:

Multiple NetFlow v9 or IPFIX entities may map to the same or similar information, as shown in the previous tables.

All of the standardized entities are not implemented due to a lack of examples from a production site or the impossibility to integrate into the design. If a missing entity’s information could be used by the sniffer, it could be later added in exchange of a PCAP file.

Most of the required entities are similar to the ones present in NetFlow v5.

The base information required for any flow is:

• Start & end of the flow
• Ingress & egress interfaces
• IP protocol & ToS
• Source & destination addresses
• Number of bytes
• Number of packets

Skylight capture: sensor will not ingest any flow that doesn't contain the base information.

The following layers require more information, specific to each:

• TCP: source & destination port
• UDP: source & destination port
• ICMP: type & code

Examples

This is a flow that may induce problems due to its excessively long duration.

$ tshark -V -r some.pcap
...
        Flow 5
          SrcAddr: 1.2.3.4 (1.2.3.4)
          DstAddr: 6.7.8.9 (6.7.8.9)
          SrcPort: 3544
          DstPort: 51655
          IPv4 ICMP Type: 0
          IPv4 ICMP Code: 0
          IP ToS: 0x00
          Protocol: 17
          InputInt: 43
          TCP Flags: 0x00
          Octets: 548
          Packets: 4
          [Duration: 75.000000000 seconds]
              StartTime: May  5, 2017 17:06:28.056000000 CEST
              EndTime: May  5, 2017 17:07:43.056000000 CEST
          OutputInt: 0
...

Below is a flow that will not be ingested due to several missing required fields. Note that NAT entities won’t be used by Skylight capture: sensor.

$ tshark -d 'udp.port==9995,cflow' -V -r some.pcap
...
      Flow 14
          SrcAddr: 1.2.3.4 (1.2.3.4)
          Post NAT Source IPv4 Address: 10.20.30.40 (10.20.30.40)
          DstAddr: 6.7.8.9 (6.7.8.9)
          Post NAT Destination IPv4 Address: 60.70.80.90 (60.70.80.90)
          SrcPort: 64507
          Post NAPT Source Transport Port: 23644
          DstPort: 22652
          Post NAPT Destination Transport Port: 22652
          Ingress VRFID: 0
          Protocol: 6
          Nat Event: 2
          Observation Time Milliseconds: Aug  5, 2014 13:18:46.246000000 CEST
...

The following excerpts of commands were successfully used on Cisco switches and may be of interest.

collect timestamp absolute first
collect timestamp absolute last

match interface input
collect interface output

match ipv4 protocol
match ipv4 tos

match ipv4 source address
match ipv4 destination address

match transport source-port
match transport destination-port

match transport icmp ipv4 type
match transport icmp ipv4 code

collect counter bytes long
collect counter packets long

cache timeout inactive 60
cache timeout active 60
cache timeout update 60