I have 3 servers, 2 are Dell 1425's running ESXi 3 (box A and B), another is a SuperMicro running ESXi 4 (box C). Everything was running fine until about 4 days ago. I noticed that one of the guests on box A was getting a mind blowing 40kb/s file transfer rate when doing a yum update against a repository on box B. Testing showed that any instance on box B or C can access the repo just fine and get various speeds well above the 5MB range when doing file transfers.
Box A had both Windows and Linux guests on it. It doesn't seem to matter which guest OS is installed, I still get this slowness. Sometimes on the Windows box, it's fine. Other times, it's not. I suspect that the same is true for the Linux boxes.
The only change I made about 10 days ago was to add a virtual switch on a seperate VLAN and put a single server on that one. I created this VLAN on all ESXi hosts though (in the same way with the same VLAN ID's). I would suspect that if this change is the root cause that I would be seeing this problem across the board. scp'ing a file from box B to/from box A is about 5mb/s so the physical network seems fine.
Realtime:
CPU on the affected host is about 30% (10 guests), memory is about 73% (6GB of 8GB allocated and used), disk is averaging 100kb/sec, network 17kb/sec.
Any ideas on how to approach this problem?
