VMware Cloud Community
erikjohnsen
Enthusiast
Enthusiast
Jump to solution

Bandwidth requirements for Remote Collector Nodes

Hello

Is there some sort of calculator available that details how much bandwidth should be available between the Master Node and Remote Collector Nodes, depending on number of objects/VMs/Hosts/clusters etc, or any documentation regarding how much bandwidth vROps uses?

Erik

EDIT: What I'm trying to determine is whether it's feasible for low-bandwidth sites/locations to send data to the Master Node or whether vROps will saturate the link completely. There should be some documentation on this right?

0 Kudos
1 Solution

Accepted Solutions
mark_j
Virtuoso
Virtuoso
Jump to solution

Network calculator for vR Ops, not right this moment. It will depend on the object count and collection volume (# metrics). But it also depending what, exactly, each MP is actually pulling. Some MP are heavier than other with regarding to API calls and what they're gobbling up from the data source. However, if we're talking just the data we're keeping, call it 24 bytes per metric. Assuming you're collecting 1-million metrics, it'll end up being about 22.8MB per 5 min. Once you throw an RC in there, it'll handle the dirty part of the data collection and normalize the data down to something efficient for transfer to the Analytic nodes. So, an RC certainly can't hurt at a remote site that's got a so-so WAN link between Analytic cluster nodes and data source. Here are some guidelines -

Between the Analytics cluster nodes and Remote Collectors, make sure your WAN latency is less than 200ms.

From a throughput perspective, the Analytics Node-to-RC network utilization will be a bit lower than if you're running Analytics Nodes-to-data sources. This is because the RC performs the raw data collections from a data source, then normalized and cleans up the data before it sends it back to the Analytics node.

Another consideration is the queuing.. the remote collectors will queue data and retries for 2min if it loses connectivity to the Analytics nodes. After that 2 min, it'll drop the queued data and pull fresh from data source on next collection interval and repeat the cycle. This gives you a little buffer on those unreliable WAN links.

So, no downside for RCs in WAN scenarios, we do them as  rule of thumb if you've got anything other than a high quality, fat WAN link.

If you find this or any other answer useful please mark the answer as correct or helpful.

View solution in original post

0 Kudos
2 Replies
mark_j
Virtuoso
Virtuoso
Jump to solution

Network calculator for vR Ops, not right this moment. It will depend on the object count and collection volume (# metrics). But it also depending what, exactly, each MP is actually pulling. Some MP are heavier than other with regarding to API calls and what they're gobbling up from the data source. However, if we're talking just the data we're keeping, call it 24 bytes per metric. Assuming you're collecting 1-million metrics, it'll end up being about 22.8MB per 5 min. Once you throw an RC in there, it'll handle the dirty part of the data collection and normalize the data down to something efficient for transfer to the Analytic nodes. So, an RC certainly can't hurt at a remote site that's got a so-so WAN link between Analytic cluster nodes and data source. Here are some guidelines -

Between the Analytics cluster nodes and Remote Collectors, make sure your WAN latency is less than 200ms.

From a throughput perspective, the Analytics Node-to-RC network utilization will be a bit lower than if you're running Analytics Nodes-to-data sources. This is because the RC performs the raw data collections from a data source, then normalized and cleans up the data before it sends it back to the Analytics node.

Another consideration is the queuing.. the remote collectors will queue data and retries for 2min if it loses connectivity to the Analytics nodes. After that 2 min, it'll drop the queued data and pull fresh from data source on next collection interval and repeat the cycle. This gives you a little buffer on those unreliable WAN links.

So, no downside for RCs in WAN scenarios, we do them as  rule of thumb if you've got anything other than a high quality, fat WAN link.

If you find this or any other answer useful please mark the answer as correct or helpful.
0 Kudos
erikjohnsen
Enthusiast
Enthusiast
Jump to solution

Thank you so much for answering Mark! Was losing hope here..

For my design the only important traffic is the one traversing RC and the Analytics cluster (where bandwidth may be scarce or the node appearing "offline" at times). Is there a source for the 24 bytes per metric and the other tidbits in your answer? I need to document this somehow, alas..

Thanks again

Erik

0 Kudos