One of the sales bullets for vDSes is that they offer ingress traffic shaping. I started thinking about what that really means, and came to the tentative conclusion that ingress shaping only pays off for TCP traffic. Let me know if you agree or disagree with the following reasoning.
My mental model for what ESXi does when it is tasked with shaping ingress traffic to VM1 is this. Some distant machine S (lets assume one source machine for now) is shooting frames at VM1 at a high rate. If the rate exceeds the max or average limits, ESXi either buffers some of the incoming frames in RAM, or just drops them. (I'm guessing the frames are dropped, as this seems like a Layer 2 switch kind of thing; when in doubt, drop it out). So lets imagine what happens when machine S is shooting UDP frames, versus TCP frames.
When machine S is shooting a high rate of UDP frames, its going to turn on the "UDP firehose" and its up to the intermediate network devices and destination machine VM1 to simply absorb all that is sent. If S sends frames so fast that ESXi runs out of buffering then frames start getting dropped to satisfy the ingress traffic shaping "equation" and the connection is borked. But source machine S has no idea this is going on, so it keeps hosing out UDP frames until it finishes or maybe receives some kind of reset from Layer 7 on VM1. So the network from the backbone thru the pSwitches into the vDS still has the uncontrolled traffic and the other VMs on the vDS are starved of bandwidth, its just that VM1 isn't allowed to receive all the UDP frames.
On the other hand, when machine S is shooting a high rate of TCP frames, after it shoots a window full of frames, its supposed to wait for an ACK before shooting the next windowfull. When ingress traffic shaping is happening, if frames are dropped then VM1's ACK will shrink expected window size until a complete window can pass through the filters without droppage. In other words, what makes ingress shaping useful is TCP's PAR (Positive Acknowledgment and Retransmission). Machine S is forced to throttle its "TCP firehose" until it is only sending what the vDS will allow VM1 to receive.
So, is any of this correct?
