NetFlow is a handy tool in the daily work of Network Admins and Analysts. It can be used to measure traffic in networks from an End-to-End perspective, and allows the filtering by several types of data:
- Time
- Source/destination IP
- Source/destination port numbers (at least for UDP and TCP)
- Ingress interface SNMP ifIndex TOS information
- AS numbers
- TCP flags
- Protocol type
How does it work?
NetFlow information is exposed by compatible routers or switches. These can be configured to send their NetFlow data in a well-defined format to a dedicated collector that is responsible for storing the data. A reporting engine can then access the data storage and perform analysis, create a statistic, plot charts for the end-user, etc. But, when drilling down into the details, it is unfortunately not as easy as it sounds.
Flow fragmentation
Originally developed by Cisco, NetFlow has developed into an open standard (https://tools.ietf.org/html/rfc3954), but has also spawned other types of Flow implementations over time, like sFlow, cFlow, jFlow, IPFIX, only to name a few. Some of them are compatible with each other, others are not – like NetFlow and sFlow. Furthermore, also the orginal NetFlow specifications have experienced multiple revisions, and in the wild, there are different versions. Maintaining support for all those versions is not trivial.
Data storage
As NetFlow is a passive monitoring tool, you cannot simply query the Flow emitters. Quite the contrary, data is sent once – via UDP. What’s worse is that the data volume, which might be considerably high, is likely to be sent in bursts. That means the Flow collector must be able to handle all the incoming packets with sufficient speed. If there are packet drops or I/O problems, the data is lost. To perform in a Telco-grade network, the monitoring solution must be well-designed and decouple data collection from necessary data preprocessing and storage.
Correlating the data
Once the data is collected, you would want to make use of it via reports, for example. As you likely have multiple Flow emitters in your network, it is also likely that you have to handle multiple Flow versions. Therefore, it is crucial to preprocess and transform the Flows into a unified data format to provide version-spanning reporting. If you have a large network, you will need multiple Flow collectors to handle the vast amount of data. In order to provide also collector-spanning reporting, you will eventually need a central database where it all comes together.
However, bringing it all together is often not enough. Flows only contain “raw” information about source and target: IP addresses, ports, and protocol information. Most of this only becomes valuable in a bigger scope – when you correlate the IP addresses to actual devices and their related services and topology. This enables you to not only track top talkers in your network, but also keep an eye on the impact, which should be done with a method that incorporates fault management.
Integration
Most likely, there are multiple monitoring tools in place. Maybe that is because no tool has covered it all yet, or because of different departments and different preferences of the people in charge. As such, NetFlow tools will presumably never be isolated, but always part of a bigger picture. That’s why it is important to have a rich set of northbound interfaces. A relay functionality can be a big help if certain Flows should be also redirected to legacy systems due to some existing business process, or just temporarily while the new solution is evaluated without touching the daily business of the NOC.
Another integration aspect is the support for multi-tenancy. Service providers often offer statistics and other report data to their customers to give them an overview of their purchased services’ performance. As Flow packets do not carry any customer information, correlation between Flows and customers is not a trivial task.
How to choose the right tool?
Over the years, many companies and people have realized the benefits of NetFlow. As a result, you now have dozens of free and commercial tools to choose from. As you will have noticed, a good NetFlow solution faces lots of requirements, and it is hard to cover them all.
Many free solutions do not have many contributors and consequently have to focus on different parts of the NetFlow eco system – which is often about collecting, storing, and reporting. Whereas coverage of the different Flow protocols is often good, there is still room for improvement especially on the integration side. It is also the case that those free tools often do not scale well enough to be used in a Telco environment.
Besides community-driven solutions, there are also some free professional solutions developed by companies. Most of them have a business model that allows free usage of the basic functionalities, but limits the integration, correlation, or storage functionalities. To unlock those features, you have to buy the paid version, which is reasonable, as it is made by a company that needs to pay its employees.
The unified approach
Eventually, it is likely that you will end up using a proprietary solution for your NetFlow requirements if you want to implement a solution in an advanced environment and make use of all NetFlow has to offer. That is, in a nutshell, End-to-End visibility of traffic in your network, which can only be achieved if your NetFlow tool can integrate very well in your inventory, performance, and fault management setup. As integration costs can be considerably high, a unified monitoring tool, even if it only provides basic Flow analysis and reporting capabilities, might provide the best ROI in this scenario.
Whatever road you take, make sure to keep the big picture in mind and try not to waste time implementing a solution that leads to a dead end. More information is available here on a Netflow monitoring technology solution.
Thanks to Infosim for this article.
