Update network flow visibility document #1677
Conversation
c23b72e to
bc81084
Compare
Codecov Report
@@ Coverage Diff @@
## master #1677 +/- ##
===========================================
- Coverage 63.31% 40.89% -22.43%
===========================================
Files 170 106 -64
Lines 14250 13113 -1137
===========================================
- Hits 9023 5362 -3661
- Misses 4292 7275 +2983
+ Partials 935 476 -459
Flags with carried forward coverage won't be shown. Click here to find out more.
|
94b03a3 to
0bed060
Compare
docs/network-flow-visibility.md
Outdated
| parameters are set to 5s and 12, respectively. `flowCollectorAddr` is a required | ||
| parameter that is necessary for the Flow Exporter feature to work. | ||
| Please note that the default value for `flowCollectorAddr` is `"flow-aggregator.flow-aggregator.svc:4739:tcp"`, | ||
| which is based of DNS name, if Flow Aggregator service is deployed with the Name |
There was a problem hiding this comment.
"which uses the DNS name of the flow aggregator Service"?
docs/network-flow-visibility.md
Outdated
| ### Configuration | ||
|
|
||
| The following configuration parameters have to be provided through the Flow Aggregator | ||
| configMap. `externalFlowCollectorAddr` is a required parameter that is necessary |
docs/network-flow-visibility.md
Outdated
|
|
||
| ### Configuration | ||
|
|
||
| To enable the Flow Exporter feature at the Antrea Agent, the following config | ||
| parameters have to be set in the Antrea Agent ConfigMap as shown below. We provide | ||
| some examples for the parameter values in the following snippet. | ||
| parameters have to be set in the Antrea Agent ConfigMap as shown below. |
There was a problem hiding this comment.
nit: a bit redundant considering "the following" at the beginning of the sentence
docs/network-flow-visibility.md
Outdated
| Currently, the Flow Exporter feature provides visibility for Pod-to-Pod, Pod-to-Service | ||
| and Pod-to-External network flows along with the associated statistics such as data | ||
| throughput (bits per second), packet throughput (packets per second), cumulative byte | ||
| count, cumulative packet count etc. Pod-To-Service flow visibility is supported |
There was a problem hiding this comment.
I would prefer we specify the exact set of stats that are supported if not done elsewhere; then we can remove "etc".
docs/network-flow-visibility.md
Outdated
|
|
||
| # Provide the IPFIX collector address as a string with format <HOST>:[<PORT>][:<PROTO>]. | ||
| # HOST can either be the DNS name or the IP of the Flow Collector. For example, | ||
| # "flow-aggregator.flow-aggregator.svc" can be provided as DNS name to connect |
docs/network-flow-visibility.md
Outdated
|
|
||
| ```yaml | ||
| flow-aggregator.conf: | | ||
| # Provide the flow collector address as string with format <IP>:<port>[:<proto>], where proto is tcp or udp. |
0bed060 to
8f9b034
Compare
8f9b034 to
d9f8f8f
Compare
docs/network-flow-visibility.md
Outdated
| and Pod-to-External network flows along with the associated statistics such as data | ||
| throughput (bits per second), packet throughput (packets per second), cumulative byte | ||
| count, cumulative packet count etc. Pod-To-Service flow visibility is supported | ||
| only [when Antrea Proxy enabled](feature-gates.md). In the future, we will extend |
There was a problem hiding this comment.
not addressed yet
maybe specify that this is now the default: Pod-To-Service flow visibility is supported only [when Antrea Proxy is enabled](feature-gates.md), which is the case by default starting with Antrea v0.11.
docs/network-flow-visibility.md
Outdated
| IPFIX protocol, and for this purpose we use the [go-ipfix](https://github.com/vmware/go-ipfix) library. | ||
| Connections from the connection store are exported to the [Flow Aggregator | ||
| service](#flow-aggregator) using the IPFIX protocol, and for this purpose we use | ||
| the IPFIX exporter process from [go-ipfix](https://github.com/vmware/go-ipfix) library. |
docs/network-flow-visibility.md
Outdated
| @@ -67,25 +77,34 @@ some examples for the parameter values in the following snippet. | |||
| # Service traffic. | |||
| AntreaProxy: true | |||
There was a problem hiding this comment.
this is slightly outdated now that AntreaProxy is enabled by default
docs/network-flow-visibility.md
Outdated
| flow records that are exported from any given Antrea Agent, the Flow Exporter only | ||
| provides the information of Kubernetes entities that are local to the Antrea Agent. | ||
| In other words, flow records are only complete for intra-Node flows, but incomplete | ||
| for inter-Node flows. |
There was a problem hiding this comment.
Maybe add an extra sentence at the end here: It is the responsibility of the (Flow Aggregator)[#flow-aggregator] to correlate flows from source and destination Nodes and produce complete flow records.`
docs/network-flow-visibility.md
Outdated
| for inter-Node flows. | ||
|
|
||
| Flow Exporter is supported in IPv4 clusters, IPv6 clusters and dual-stack clusters. | ||
| Please note that Flow Aggregator is only supported for IPv4 clusters. We plan to |
There was a problem hiding this comment.
"in IPv4 clusters" to match the previous sentence?
docs/network-flow-visibility.md
Outdated
| ### IPFIX Information Elements (IEs) in an Aggregated Flow Record | ||
|
|
||
| In addition to IPFIX information elements provided in the [above section](#ipfix-information-elements-ies-in-a-flow-record), | ||
| the Flow Aggregator adds following fields to the flow records. |
There was a problem hiding this comment.
adds the following fields
| | IPFIX Information Element | Enterprise ID | Field ID | Type | | ||
| |-------------------------------------------|---------------|----------|-------------| | ||
| | packetTotalCountFromSourceNode | 56506 | 120 | unsigned64 | | ||
| | octetTotalCountFromSourceNode | 56506 | 121 | unsigned64 | | ||
| | packetDeltaCountFromSourceNode | 56506 | 122 | unsigned64 | | ||
| | octetDeltaCountFromSourceNode | 56506 | 123 | unsigned64 | | ||
| | reversePacketTotalCountFromSourceNode | 56506 | 124 | unsigned64 | | ||
| | reverseOctetTotalCountFromSourceNode | 56506 | 125 | unsigned64 | | ||
| | reversePacketDeltaCountFromSourceNode | 56506 | 126 | unsigned64 | | ||
| | reverseOctetDeltaCountFromSourceNode | 56506 | 127 | unsigned64 | | ||
| | packetTotalCountFromDestinationNode | 56506 | 128 | unsigned64 | | ||
| | octetTotalCountFromDestinationNode | 56506 | 129 | unsigned64 | | ||
| | packetDeltaCountFromDestinationNode | 56506 | 130 | unsigned64 | | ||
| | octetDeltaCountFromDestinationNode | 56506 | 131 | unsigned64 | | ||
| | reversePacketTotalCountFromDestinationNode| 56506 | 132 | unsigned64 | | ||
| | reverseOctetTotalCountFromDestinationNode | 56506 | 133 | unsigned64 | | ||
| | reversePacketDeltaCountFromDestinationNode| 56506 | 134 | unsigned64 | | ||
| | reverseOctetDeltaCountFromDestinationNode | 56506 | 135 | unsigned64 | |
There was a problem hiding this comment.
This is actually the first time I am looking at this, and I am actually struggling a bit to understand what these do?
How are they different from existing IEs (packetTotalCount, reversePacketTotalCount, etc)? What does reversePacketTotalCountFromSourceNode mean?
There was a problem hiding this comment.
For inter-Node flows, there is one stream of flow records from the source Node and another stream from the destination node. Aggregating stats across these two streams is not straight forward and they may not be similar in some scenarios, where there are intermittent losses or some sort of processing bottleneck at receiver.
To avoid loosing information,*SourceNode fields have aggregated statistics from the first stream of flow records and *DestinationNode fields have aggregated statistics from the second stream of flow records.
There was a problem hiding this comment.
Is this standard in IPFIX mediation? If not, are we sure this is the best way to handle this case?
Anyway, it is independent of this PR so we can discuss further later.
docs/network-flow-visibility.md
Outdated
| #### Storage of Flow Records | ||
|
|
||
| Flow Aggregator stores the received flow records from Antrea Agents in a hash map, | ||
| where the flow key is five-tuple of a network connection. Five-tuple consists of Source IP, |
docs/network-flow-visibility.md
Outdated
| #### Storage of Flow Records | ||
|
|
||
| Flow Aggregator stores the received flow records from Antrea Agents in a hash map, | ||
| where the flow key is five-tuple of a network connection. Five-tuple consists of Source IP, |
There was a problem hiding this comment.
I have never seen it spelled "five-tuple" :)
There was a problem hiding this comment.
Thought that would be more formal. Just did a search with "five-tuple" on google scholar, and there are 10K results :)
Changed to "5-tuple".. the easier convention.
docs/network-flow-visibility.md
Outdated
| from the source Node, where the flow originates from, and another one from the destination | ||
| Node, where the destination Pod resides. Both the flow records contain incomplete | ||
| information as mentioned [here](#types-of-flows-and-associated-information). Flow | ||
| Aggregator provides the support for the correlation of the flow records from the |
docs/network-flow-visibility.md
Outdated
| and Pod-to-External network flows along with the associated statistics such as data | ||
| throughput (bits per second), packet throughput (packets per second), cumulative byte | ||
| count, cumulative packet count etc. Pod-To-Service flow visibility is supported | ||
| only [when Antrea Proxy enabled](feature-gates.md). In the future, we will extend |
There was a problem hiding this comment.
Missed it here. There was extended in the context of IPv6 cluster support. Changed it there. Now done.
docs/network-flow-visibility.md
Outdated
| If you are deploying the Flow Aggregator Service on a [vagrant setup](../test/e2e/README.md), | ||
| you can use following command that deploys Antrea and Flow Aggregator in one go | ||
| with the required configuration: | ||
|
|
||
| ```shell | ||
| ./infra/vagrant/push_antrea.sh -fc <externalFlowCollectorAddr> | ||
| ``` |
There was a problem hiding this comment.
I added the manifest deployment steps here.
As there are a few steps involved in trying this feature, I thought pointing to a single script could be appealing for users, even though it is a vagrant setup. Moved these to a different section, which I feel more appropriate.
| | IPFIX Information Element | Enterprise ID | Field ID | Type | | ||
| |-------------------------------------------|---------------|----------|-------------| | ||
| | packetTotalCountFromSourceNode | 56506 | 120 | unsigned64 | | ||
| | octetTotalCountFromSourceNode | 56506 | 121 | unsigned64 | | ||
| | packetDeltaCountFromSourceNode | 56506 | 122 | unsigned64 | | ||
| | octetDeltaCountFromSourceNode | 56506 | 123 | unsigned64 | | ||
| | reversePacketTotalCountFromSourceNode | 56506 | 124 | unsigned64 | | ||
| | reverseOctetTotalCountFromSourceNode | 56506 | 125 | unsigned64 | | ||
| | reversePacketDeltaCountFromSourceNode | 56506 | 126 | unsigned64 | | ||
| | reverseOctetDeltaCountFromSourceNode | 56506 | 127 | unsigned64 | | ||
| | packetTotalCountFromDestinationNode | 56506 | 128 | unsigned64 | | ||
| | octetTotalCountFromDestinationNode | 56506 | 129 | unsigned64 | | ||
| | packetDeltaCountFromDestinationNode | 56506 | 130 | unsigned64 | | ||
| | octetDeltaCountFromDestinationNode | 56506 | 131 | unsigned64 | | ||
| | reversePacketTotalCountFromDestinationNode| 56506 | 132 | unsigned64 | | ||
| | reverseOctetTotalCountFromDestinationNode | 56506 | 133 | unsigned64 | | ||
| | reversePacketDeltaCountFromDestinationNode| 56506 | 134 | unsigned64 | | ||
| | reverseOctetDeltaCountFromDestinationNode | 56506 | 135 | unsigned64 | |
There was a problem hiding this comment.
For inter-Node flows, there is one stream of flow records from the source Node and another stream from the destination node. Aggregating stats across these two streams is not straight forward and they may not be similar in some scenarios, where there are intermittent losses or some sort of processing bottleneck at receiver.
To avoid loosing information,*SourceNode fields have aggregated statistics from the first stream of flow records and *DestinationNode fields have aggregated statistics from the second stream of flow records.
docs/network-flow-visibility.md
Outdated
| #### Storage of Flow Records | ||
|
|
||
| Flow Aggregator stores the received flow records from Antrea Agents in a hash map, | ||
| where the flow key is five-tuple of a network connection. Five-tuple consists of Source IP, |
There was a problem hiding this comment.
Thought that would be more formal. Just did a search with "five-tuple" on google scholar, and there are 10K results :)
Changed to "5-tuple".. the easier convention.
8ad2365 to
5a05eda
Compare
docs/network-flow-visibility.md
Outdated
| Connections from the connection store are exported to a flow collector using the | ||
| IPFIX protocol, and for this purpose we use the [go-ipfix](https://github.com/vmware/go-ipfix) library. | ||
| Connections from the connection store are exported to the [Flow Aggregator | ||
| service](#flow-aggregator) using the IPFIX protocol, and for this purpose we use |
5a05eda to
b50271d
Compare
b50271d to
10d21d1
Compare
| | IPFIX Information Element | Enterprise ID | Field ID | Type | | ||
| |-------------------------------------------|---------------|----------|-------------| | ||
| | packetTotalCountFromSourceNode | 56506 | 120 | unsigned64 | | ||
| | octetTotalCountFromSourceNode | 56506 | 121 | unsigned64 | | ||
| | packetDeltaCountFromSourceNode | 56506 | 122 | unsigned64 | | ||
| | octetDeltaCountFromSourceNode | 56506 | 123 | unsigned64 | | ||
| | reversePacketTotalCountFromSourceNode | 56506 | 124 | unsigned64 | | ||
| | reverseOctetTotalCountFromSourceNode | 56506 | 125 | unsigned64 | | ||
| | reversePacketDeltaCountFromSourceNode | 56506 | 126 | unsigned64 | | ||
| | reverseOctetDeltaCountFromSourceNode | 56506 | 127 | unsigned64 | | ||
| | packetTotalCountFromDestinationNode | 56506 | 128 | unsigned64 | | ||
| | octetTotalCountFromDestinationNode | 56506 | 129 | unsigned64 | | ||
| | packetDeltaCountFromDestinationNode | 56506 | 130 | unsigned64 | | ||
| | octetDeltaCountFromDestinationNode | 56506 | 131 | unsigned64 | | ||
| | reversePacketTotalCountFromDestinationNode| 56506 | 132 | unsigned64 | | ||
| | reverseOctetTotalCountFromDestinationNode | 56506 | 133 | unsigned64 | | ||
| | reversePacketDeltaCountFromDestinationNode| 56506 | 134 | unsigned64 | | ||
| | reverseOctetDeltaCountFromDestinationNode | 56506 | 135 | unsigned64 | |
There was a problem hiding this comment.
Is this standard in IPFIX mediation? If not, are we sure this is the best way to handle this case?
Anyway, it is independent of this PR so we can discuss further later.
|
/skip-all |
|
|
||
| | IPFIX Information Element | Enterprise ID | Field ID | Type | | ||
| |----------------------------|---------------|----------|-------------| | ||
| | originalExporterIPv4Address| 0 | 403 | ipv4Address | |
There was a problem hiding this comment.
we also support originalExporterIPv6Address here.
There was a problem hiding this comment.
Let's add it when we support IPv6?
There was a problem hiding this comment.
Sure. Just realized even though it is the exporter address, it is still in flow aggregator scope. (sorry for the last minute review :))
|
|
||
|  | ||
|  |
There was a problem hiding this comment.
what does the arrow direction between flow exporter and conntrack mean? I would suppose flow exporter to get data from conntrack
There was a problem hiding this comment.
Yes, it signifies that flow exporter is polling conntrack module to get flow data.
Document flow aggregator feature details.
This will be checked in after PR #1671