What sort of network thruput to expect between 2K3 hosts on the same vSwitch

[Brijn]

Hello,

I'm very new to ESX and trying to feel my way around. I've put two W2K3 VM's on a HP Proliant BL465c G5 blade (8 Cores @ 2.1G, 16G RAM) attached to a HP EVA SAN (R5 vDisk).

The two VM's are connected to the same vSwitch. VM configuration shows NIC as Flexible. Inside the VM they show as "VMware Accelerated AMD PCnet adapter", driver is vmxnet.sys

When I copy a large ISO (3GB) between the two hosts I only get ~ 250Mbit, CPU load goes up to about 50%

Since they are connected to the same vSwitch I was expecting basically full wire speed.

Am I to optimistic Or are there things to tweak/check?

Tx

Bas

[Texiwill]

Hello,

Moved to Virtual Machine and Guest OS forum.

Performance of network transfers depend entirely on what is running on the host at this time. Remember that the vSwitch is a software device that has to use CPU as well as the VMs, etc. So while it can perform very well at times, depending on the workload on your VMware ESX host your transfer rates could suffer.

Could you explain what you were using to show '50%' utilization? Windows tools or ESX based tools?

Also what else is on the vSwitch and are you overcommitting memory or CPU?

Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.
Blue Gears and SearchVMware Pro Blogs -- Top Virtualization Security Links -- Virtualization Security Round Table Podcast

[Brijn]

Hi,

That was inside the Guest. I repeated the test using esxtop.

-

---

Idle situation (I took the lines I think are relevant)

-

---

10:34:10am up 4 days 2:13, 76 worlds; CPU load average: 0.06, 0.05, 0.05

PCPU(%): 1.28, 0.33, 1.63, 0.29, 6.38, 0.57, 5.49, 37.17 ; used total: 6.64

CCPU(%): 0 us, 0 sy, 100 id, 0 wa ; cs/sec: 302

1 1 idle 8 749.77 751.71 0.00 0.00 45.97 0.00 0.14 0.00 0.00

35 35 CAGSW008 5 6.14 6.10 0.07 495.04 0.49 94.68 0.32 0.00 0.00

41 41 CAGSW001 5 8.95 8.93 0.06 492.02 0.64 91.11 0.17 0.00 0.00

-

---

3G ISO copy

-

---

10:36:16am up 4 days 2:15, 76 worlds; CPU load average: 0.08, 0.06, 0.05

PCPU(%): 2.16, 1.74, 0.35, 0.32, 56.57, 16.20, 17.88, 50.99 ; used total: 18.28

CCPU(%): 0 us, 1 sy, 99 id, 0 wa ; cs/sec: 322

1 1 idle 8 655.98 658.31 0.00 0.00 144.25 0.00 0.40 0.00 0.00

35 35 CAGSW008 5 71.71 71.96 0.50 426.65 3.01 26.06 0.95 0.00 0.00

41 41 CAGSW001 5 66.83 66.53 1.05 430.91 0.70 32.73 0.90 0.00 0.00

The values obviously jump around a bit, but these where fairly average. Network usage average around 20% as measured by Win task Man, some jumps to 25%

Is this useful data? Anything helpful in there?

TX!

Bas

[Cooldude09]

When using esxtop hit alphabet "n" and paste the results.

Regards

Anil

Save the planet, Go Green

if you found my answer to be useful, feel free to mark it as Helpful or Correct.

[Brijn]

Hi,

OK, I trust the networking stack of Linux a bit more then Windows. Also, my tests systems where P2V'ed, so I created to new Ubuntu x32 server install from scratch (1 vCPU, 512MB ram). Almost no services other then SSHd and VSFTPd. Test file is a 2.6G iso.

first test is connected to the normal production lan vSwitch, Both scp and ftp show about 20-22 MB/s transfer speeds. Esxtop results:

PORT ID UPLINK USED BY DTYP DNAME PKTTX/s MbTX/s PKTRX/s MbRX/s %DRPTX %DRPRX

16777217 Y vmnic0 H vSwitch0 5.98 0.01 10.57 0.01 0.00 0.00

16777218 N 0:NCP H vSwitch0 0.00 0.00 0.00 0.00 0.00 0.00

16777219 Y vmnic2 H vSwitch0 0.60 0.00 5.78 0.01 0.00 0.00

16777220 N 0:CDP H vSwitch0 0.00 0.00 0.00 0.00 0.00 0.00

16777221 N 0:vswif0 H vSwitch0 5.58 0.01 9.37 0.01 0.00 0.00

16777263 N 1174:Test Linux 2 H vSwitch0 2278.45 1.15 17467.02 196.79 0.00 0.00

16777265 N 1179:Test Linux 1 H vSwitch0 17461.44 196.78 2284.23 1.15 0.00 0.00

33554433 Y vmnic1 H vSwitch1 0.00 0.00 0.00 0.00 0.00 0.00

33554434 N 0:NCP H vSwitch1 0.00 0.00 0.00 0.00 0.00 0.00

33554436 N 0:CDP H vSwitch1 0.00 0.00 0.00 0.00 0.00 0.00

33554437 N 0:vmk-tcpip-192.168. H vSwitch1 0.00 0.00 0.00 0.00 0.00 0.00

50331649 N 0:CDP H vSwitch2 0.00 0.00 0.00 0.00 0.00 0.00

50331650 Y vmnic3 H vSwitch2 0.00 0.00 0.00 0.00 0.00 0.00

50331651 N 0:NCP H vSwitch2 0.00 0.00 0.00 0.00 0.00 0.00

67108865 N 0:CDP H vSwitch3 0.00 0.00 0.00 0.00 0.00 0.00

On a new vSwitch with no actual cards attached:

PORT ID UPLINK USED BY DTYP DNAME PKTTX/s MbTX/s PKTRX/s MbRX/s %DRPTX %DRPRX

16777217 Y vmnic0 H vSwitch0 11.77 0.04 14.56 0.01 0.00 0.00

16777218 N 0:NCP H vSwitch0 0.00 0.00 0.00 0.00 0.00 0.00

16777219 Y vmnic2 H vSwitch0 0.60 0.00 6.18 0.01 0.00 0.00

16777220 N 0:CDP H vSwitch0 0.00 0.00 0.00 0.00 0.00 0.00

16777221 N 0:vswif0 H vSwitch0 11.37 0.04 13.56 0.01 0.00 0.00

33554433 Y vmnic1 H vSwitch1 0.00 0.00 0.00 0.00 0.00 0.00

33554434 N 0:NCP H vSwitch1 0.00 0.00 0.00 0.00 0.00 0.00

33554436 N 0:CDP H vSwitch1 0.00 0.00 0.00 0.00 0.00 0.00

33554437 N 0:vmk-tcpip-192.168. H vSwitch1 0.00 0.00 0.00 0.00 0.00 0.00

50331649 N 0:CDP H vSwitch2 0.00 0.00 0.00 0.00 0.00 0.00

50331650 Y vmnic3 H vSwitch2 0.00 0.00 0.00 0.00 0.00 0.00

50331651 N 0:NCP H vSwitch2 0.00 0.00 0.00 0.00 0.00 0.00

67108865 N 0:CDP H vSwitch3 0.00 0.00 0.00 0.00 0.00 0.00

67108868 N 1184:Test Linux 1 H vSwitch3 2347.86 1.18 12244.53 137.19 0.00 0.00

67108869 N 1189:Test Linux 2 H vSwitch3 12244.93 137.20 2347.86 1.18 0.00 0.00

So the same results

Main screen essxtop results during this run:

10:04:37am up 5 days 22:02, 65 worlds; CPU load average: 0.10, 0.05, 0.04

PCPU(%): 1.26, 49.29, 0.26, 0.12, 51.54, 0.12, 0.06, 0.67 ; used total: 12.91

CCPU(%): 0 us, 0 sy, 100 id, 0 wa ; cs/sec: 316

ID GID NAME NWLD %USED %RUN %SYS %WAIT %RDY %IDLE %OVRLP %CSTP %MLMTD

1 1 idle 8 692.58 693.48 0.00 0.00 101.70 0.00 0.08 0.00 0.00

2 2 system 7 0.03 0.03 0.00 695.76 0.01 0.00 0.00 0.00 0.00

6 6 helper 23 0.01 0.01 0.00 2286.38 0.00 0.00 0.00 0.00 0.00

7 7 drivers 14 0.01 0.01 0.00 1391.64 0.00 0.00 0.00 0.00 0.00

8 8 vmotion 1 0.00 0.00 0.00 99.40 0.00 0.00 0.00 0.00 0.00

9 9 console 1 1.37 1.02 0.01 98.38 0.00 98.34 0.13 0.00 0.00

15 15 vmware-vmkauthd 1 0.00 0.00 0.00 99.40 0.00 0.00 0.00 0.00 0.00

43 43 Test Linux 1 5 51.87 51.76 0.19 445.24 0.01 40.52 0.75 0.00 0.00

44 44 Test Linux 2 5 49.25 49.35 0.29 447.67 0.01 50.16 0.65 0.00 0.00

I surely should be able to get more then 200 Mbit???

Bas