VMware Cloud Community
AntonioFel
Contributor
Contributor

Esxi 6.0 slow network performance

Hi, i have two esxi 6.0 whit gb connection, but when i tranfer a file between the two server, the speed is vary bad ( average 8Mbyte/s).

I have the same issue in both direction. I have also perform a nic teaming, but the result is the same.

Enviroment:

ProLiant DL360e Gen8Vmware ESXi 6.0.0, 3380124
ProLiant DL120 Gen9Vmware ESXi 6.0.0, 3380124

Both the server are unloaded in Cpu and memory, and are connected at the same GB switch (with cat6 cables Smiley Happy )

Tell me if you need others info.

Thank you for the support!!

Reply
0 Kudos
18 Replies
J1mbo
Virtuoso
Virtuoso

What's the storage subsystem?

Reply
0 Kudos
AntonioFel
Contributor
Contributor

Local datastore, configured in raid 5 (both)

Reply
0 Kudos
J1mbo
Virtuoso
Virtuoso

Do your controllers have battery or flash-backed write cache?

Reply
0 Kudos
AntonioFel
Contributor
Contributor

I don't know. How can i chack it?

P.S In a previosus installation with the same enviroment but with esxi 5.5, the speed was regular.

Reply
0 Kudos
J1mbo
Virtuoso
Virtuoso

Look up the controller, B140 or H240 controller have no cache options so aren't suitable for ESXi. But these services might have P440 controller with FBWC.

Reply
0 Kudos
thecloudxpert
Enthusiast
Enthusiast

is anything set to auto-negotiate network speeds? that has in the past, caused me problems. (i.e. switch port set to auto, server set to auto and then it decides to run at 10Mb/s)

Reply
0 Kudos
AntonioFel
Contributor
Contributor

I have these controller:

[root@SRV-ESXI-2:/opt/hp/hpssacli/bin] ./hpssacli ctrl all show detail

Dynamic Smart Array B140i in Slot 0b

   Bus Interface: PCI

   Slot: 0b

   Controller Status: OK

   Hardware Revision: C

   RAID Stack Version: 3.02

   Option ROM Version:3.49-10

   Rebuild Priority: High

   Surface Scan Delay: 3 secs

   Surface Scan Mode: Idle

   Parallel Surface Scan Supported: Yes

   Current Parallel Surface Scan Count: 1

   Max Parallel Surface Scan Count: 16

   Wait for Cache Room: Disabled

   Surface Analysis Inconsistency Notification: Disabled

   Post Prompt Timeout: 15 secs

   Cache Board Present: False

   Cache Ratio: 100% Read / 0% Write

   Read Cache Size: 64 MB

   Write Cache Size: 0 MB

   Drive Write Cache: Disabled

   Total Cache Size: 0 MB

   SATA NCQ Supported: True

   Spare Activation Mode: Activate on physical drive failure (default)

   Number of Ports: 4 Internal only

   Driver Name: HP HPDS

   Driver Version: 0.0.5

   PCI Address (Domain:Bus:Device.Function): 0000:01:00.7

   Controller Mode: RAID

   Controller Mode Reboot: Not Required

   Latency Scheduler Setting: Disabled

   Host Serial Number: CZ2611052K

   Sanitize Erase Supported: False

   Primary Boot Volume: None

   Secondary Boot Volume: None

[root@SRV-ESXi-1:/opt/hp/hpssacli/bin] ./hpssacli ctrl all show detail

Dynamic Smart Array B320i RAID in Slot 0 (Embedded)

   Bus Interface: PCI

   Slot: 0

   Cache Serial Number: PBKUB0CRH6H2O7

   Controller Status: OK

   Hardware Revision: C

   RAID Stack Version: 5.00

   HBA Firmware Version: 15.10.05.00

   HBA Mode Option ROM Version: 07.25.05.00

   Option ROM Version:3.54-0

   Rebuild Priority: Medium

   Surface Scan Delay: 3 secs

   Surface Scan Mode: Idle

   Parallel Surface Scan Supported: No

   Wait for Cache Room: Disabled

   Surface Analysis Inconsistency Notification: Disabled

   Post Prompt Timeout: 15 secs

   Cache Board Present: True

   Cache Status: OK

   Read Cache Size: 64 MB

   Write Cache Size: 512 MB

   Drive Write Cache: Disabled

   Total Cache Size: 512 MB

   No-Battery Write Cache: Disabled

   Cache Backup Power Source: Capacitors

   Battery/Capacitor Count: 1

   Battery/Capacitor Status: OK

   SATA NCQ Supported: True

   Spare Activation Mode: Activate on physical drive failure (default)

   Number of Ports: 2 Internal only

   Driver Name: HP HPVS

   Driver Version: 0.0.1

   PCI Address (Domain:Bus:Device.Function): 0000:07:00.0

   Host Serial Number: CZ342064VB

   Sanitize Erase Supported: False

   Primary Boot Volume: None

   Secondary Boot Volume: None

Reply
0 Kudos
AntonioFel
Contributor
Contributor

All the interface on the switch are in auto.

I have force it to 1000-full but the issue is the same.

Reply
0 Kudos
J1mbo
Virtuoso
Virtuoso

‌you need to enable the write cache of the b320 and upgrade the b140.

Reply
0 Kudos
AntonioFel
Contributor
Contributor

Ok, but as writed before, in the old enviroment with:


ProLiant DL360e Gen8


ProLiant DL180e Gen6


equiped with esxi 5.5, the speed between the two host was normal (wire speed). Nothing has changed, the controller b320 is the same.


Can you explain it?


Have the previuous version of esxi, a different behavior?


Thank you!!

Reply
0 Kudos
J1mbo
Virtuoso
Virtuoso

If you want to gain some assurance over the network side of things try running iperf both ways, eg:

within a guest on one machine (I'll call it 1.1.1.1): iperf -s -w 256K

within a guest on the other machine: iperf -c 1.1.1.1 -w 256k and iperf -c 1.1.1.1 -w 256k -d

This will show raw IP throughput across the physical network between the two guests. You should see 110MB/s (about 940Mbps) or more.

Re caching - I've just edited this post as I miss-read the output from the b320 controller. It looks that the write cache is enabled on that. Do you see slow copy both from and to that host? Also, check all firmware is up-to-date and are you using the HP customised build?

Reply
0 Kudos
AntonioFel
Contributor
Contributor

Yes, the slow copy it's from and to the host, and the image it's an HP custom on both hosts.

But, sorry for the repeat, but i't importanto for me undestand, why in the old enviroment all works fine?

Reply
0 Kudos
J1mbo
Virtuoso
Virtuoso

Check the driver versions on both environments - SSH to the hosts then run

esxcli storage adapter core list

to find the driver name, then

esxcli software vib list

To see the version. Others have reported driver v88 on 5.5 being OK but v100 being slow and a solution being to revert to the older driver (this thread).

Hope that helps.

Reply
0 Kudos
AntonioFel
Contributor
Contributor

Ok i have tried but the commands are refused:

[root@SRV-ESXI-2:~] esxcli storage adapter core list

Error: Unknown command or namespace storage adapter core list

what is the right syntax ?

Reply
0 Kudos
J1mbo
Virtuoso
Virtuoso

Sorry  esxcli storage core adapter list

Reply
0 Kudos
AntonioFel
Contributor
Contributor

[root@SRV-ESXI-2:~] esxcli storage core adapter list

HBA Name  Driver  Link State  UID            Capabilities  Description                                                 

--------  ------  ----------  --------------  ------------  -----------------------------------------------------------------------------------

vmhba0    hpdsa  link-n/a    unknown.vmhba0                (0000:01:00.7) Hewlett-Packard Company HP|Dynamic Smart Array B140i RAID Controller

[root@SRV-ESXI-2:~] esxcli software vib list

Name                          Version                                Vendor          Acceptance Level  Install Date

-----------------------------  -------------------------------------  ---------------  ----------------  ------------

net-tg3                        3.137l.v60.1-1OEM.600.0.0.2494585      BRCM            VMwareCertified  2016-04-19

elxnet                        10.5.121.7-1OEM.600.0.0.2159203        EMU              VMwareCertified  2016-04-19

ima-be2iscsi                  10.5.101.0-1OEM.600.0.0.2159203        EMU              VMwareCertified  2016-04-19

lpfc                          10.5.70.0-1OEM.600.0.0.2159203        EMU              VMwareCertified  2016-04-19

scsi-be2iscsi                  10.5.101.0-1OEM.600.0.0.2159203        EMU              VMwareCertified  2016-04-19

hpe-build                      600.9.4.5.11-2494585                  HPE              PartnerSupported  2016-04-19

char-hpcru                    6.0.6.14-1OEM.600.0.0.2159203          Hewlett-Packard  PartnerSupported  2016-04-19

char-hpilo                    600.9.0.2.8-1OEM.600.0.0.2159203      Hewlett-Packard  PartnerSupported  2016-04-19

hp-ams                        600.10.3.0-15.2494585                  Hewlett-Packard  PartnerSupported  2016-04-19

hp-conrep                      6.0.0.1-0.0.13.2159203                Hewlett-Packard  PartnerSupported  2016-04-19

hp-esxi-fc-enablement          600.2.4.6-2494585                      Hewlett-Packard  PartnerSupported  2016-04-19

hp-smx-provider                600.03.09.00.15-2768847                Hewlett-Packard  VMwareAccepted    2016-04-19

hpbootcfg                      6.0.0.02-01.00.11.2159203              Hewlett-Packard  PartnerSupported  2016-04-19

hpnmi                          600.2.3.14-2159203                    Hewlett-Packard  PartnerSupported  2016-04-19

hponcfg                        6.0.0.04-00.13.17.2159203              Hewlett-Packard  PartnerSupported  2016-04-19

hpssacli                      2.30.6.0-6.0.0.2159203                Hewlett-Packard  PartnerSupported  2016-04-19

hptestevent                    6.0.0.01-00.00.8.2159203              Hewlett-Packard  PartnerSupported  2016-04-19

scsi-hpdsa                    5.5.0.46-1OEM.550.0.0.1331820          Hewlett-Packard  PartnerSupported  2016-04-19

scsi-hpsa                      6.0.0.114-1OEM.600.0.0.2494585        Hewlett-Packard  VMwareCertified  2016-04-19

scsi-hpvsa                    5.5.0.100-1OEM.550.0.0.1331820        Hewlett-Packard  PartnerSupported  2016-04-19

intelcim-provider              0.5-1.4                                Intel            VMwareAccepted    2016-04-19

net-i40e                      1.2.48-1OEM.550.0.0.1331820            Intel            VMwareCertified  2016-04-19

net-igb                        5.2.10-1OEM.550.0.0.1331820            Intel            VMwareCertified  2016-04-19

net-ixgbe                      3.21.4.3-1OEM.550.0.0.1331820          Intel            VMwareCertified  2016-04-19

scsi-mpt2sas                  15.10.06.00.1vmw-1OEM.550.0.0.1198610  LSI              VMwareCertified  2016-04-19

nmlx4-core                    3.1.0.0-1OEM.600.0.0.2348722          MEL              VMwareCertified  2016-04-19

nmlx4-en                      3.1.0.0-1OEM.600.0.0.2348722          MEL              VMwareCertified  2016-04-19

nmst                          4.0.0.20-1OEM.600.0.0.2295424          MEL              PartnerSupported  2016-04-19

misc-cnic-register            1.712.50.v60.1-1OEM.600.0.0.2494585    QLogic          VMwareCertified  2016-04-19

net-bnx2                      2.2.5j.v60.3-1OEM.600.0.0.2494585      QLogic          VMwareCertified  2016-04-19

net-bnx2x                      2.712.50.v60.6-1OEM.600.0.0.2494585    QLogic          VMwareCertified  2016-04-19

net-cnic                      2.712.50.v60.6-1OEM.600.0.0.2494585    QLogic          VMwareCertified  2016-04-19

net-nx-nic                    6.0.643-1OEM.600.0.0.2494585          QLogic          VMwareCertified  2016-04-19

net-qlcnic                    6.1.191-1OEM.600.0.0.2494585          QLogic          VMwareCertified  2016-04-19

qlnativefc                    2.1.27.0-1OEM.600.0.0.2768847          QLogic          VMwareCertified  2016-04-19

scsi-bfa                      3.2.5.0-1OEM.550.0.0.1331820          QLogic          VMwareCertified  2016-04-19

scsi-bnx2fc                    1.712.50.v60.7-1OEM.600.0.0.2494585    QLogic          VMwareCertified  2016-04-19

scsi-bnx2i                    2.712.50.v60.4-1OEM.600.0.0.2494585    QLogic          VMwareCertified  2016-04-19

scsi-qla4xxx                  644.6.04.0-1OEM.600.0.0.2159203        QLogic          VMwareCertified  2016-04-19

mtip32xx-native                3.8.5-1vmw.600.0.0.2494585            VMWARE          VMwareCertified  2016-04-19

ata-pata-amd                  0.3.10-3vmw.600.0.0.2494585            VMware          VMwareCertified  2016-04-19

ata-pata-atiixp                0.4.6-4vmw.600.0.0.2494585            VMware          VMwareCertified  2016-04-19

ata-pata-cmd64x                0.2.5-3vmw.600.0.0.2494585            VMware          VMwareCertified  2016-04-19

ata-pata-hpt3x2n              0.3.4-3vmw.600.0.0.2494585            VMware          VMwareCertified  2016-04-19

ata-pata-pdc2027x              1.0-3vmw.600.0.0.2494585              VMware          VMwareCertified  2016-04-19

ata-pata-serverworks          0.4.3-3vmw.600.0.0.2494585            VMware          VMwareCertified  2016-04-19

ata-pata-sil680                0.4.8-3vmw.600.0.0.2494585            VMware          VMwareCertified  2016-04-19

ata-pata-via                  0.3.3-2vmw.600.0.0.2494585            VMware          VMwareCertified  2016-04-19

block-cciss                    3.6.14-10vmw.600.0.0.2494585          VMware          VMwareCertified  2016-04-19

cpu-microcode                  6.0.0-0.0.2494585                      VMware          VMwareCertified  2016-04-19

ehci-ehci-hcd                  1.0-3vmw.600.1.26.3380124              VMware          VMwareCertified  2016-04-19

emulex-esx-elxnetcli          10.2.309.6v-0.0.2494585                VMware          VMwareCertified  2016-04-19

esx-base                      6.0.0-1.26.3380124                    VMware          VMwareCertified  2016-04-19

esx-dvfilter-generic-fastpath  6.0.0-0.0.2494585                      VMware          VMwareCertified  2016-04-19

esx-tboot                      6.0.0-0.0.2494585                      VMware          VMwareCertified  2016-04-19

esx-xserver                    6.0.0-0.0.2494585                      VMware          VMwareCertified  2016-04-19

ima-qla4xxx                    2.02.18-1vmw.600.0.0.2494585          VMware          VMwareCertified  2016-04-19

ipmi-ipmi-devintf              39.1-4vmw.600.0.0.2494585              VMware          VMwareCertified  2016-04-19

ipmi-ipmi-msghandler          39.1-4vmw.600.0.0.2494585              VMware          VMwareCertified  2016-04-19

ipmi-ipmi-si-drv              39.1-4vmw.600.0.0.2494585              VMware          VMwareCertified  2016-04-19

lsi-mr3                        6.605.08.00-7vmw.600.1.17.3029758      VMware          VMwareCertified  2016-04-19

lsi-msgpt3                    06.255.12.00-8vmw.600.1.17.3029758    VMware          VMwareCertified  2016-04-19

lsu-hp-hpsa-plugin            1.0.0-1vmw.600.0.0.2494585            VMware          VMwareCertified  2016-04-19

lsu-lsi-lsi-mr3-plugin        1.0.0-2vmw.600.0.11.2809209            VMware          VMwareCertified  2016-04-19

lsu-lsi-lsi-msgpt3-plugin      1.0.0-1vmw.600.0.0.2494585            VMware          VMwareCertified  2016-04-19

lsu-lsi-megaraid-sas-plugin    1.0.0-2vmw.600.0.11.2809209            VMware          VMwareCertified  2016-04-19

lsu-lsi-mpt2sas-plugin        1.0.0-4vmw.600.1.17.3029758            VMware          VMwareCertified  2016-04-19

lsu-lsi-mptsas-plugin          1.0.0-1vmw.600.0.0.2494585            VMware          VMwareCertified  2016-04-19

misc-drivers                  6.0.0-1.26.3380124                    VMware          VMwareCertified  2016-04-19

net-e1000                      8.0.3.1-5vmw.600.0.0.2494585          VMware          VMwareCertified  2016-04-19

net-e1000e                    3.2.2.1-1vmw.600.1.26.3380124          VMware          VMwareCertified  2016-04-19

net-enic                      2.1.2.38-2vmw.600.0.0.2494585          VMware          VMwareCertified  2016-04-19

net-forcedeth                  0.61-2vmw.600.0.0.2494585              VMware          VMwareCertified  2016-04-19

net-mlx4-core                  1.9.7.0-1vmw.600.0.0.2494585          VMware          VMwareCertified  2016-04-19

net-mlx4-en                    1.9.7.0-1vmw.600.0.0.2494585          VMware          VMwareCertified  2016-04-19

net-vmxnet3                    1.1.3.0-3vmw.600.0.0.2494585          VMware          VMwareCertified  2016-04-19

nmlx4-rdma                    3.0.0.0-1vmw.600.0.0.2494585          VMware          VMwareCertified  2016-04-19

nvme                          1.0e.0.35-1vmw.600.1.17.3029758        VMware          VMwareCertified  2016-04-19

ohci-usb-ohci                  1.0-3vmw.600.0.0.2494585              VMware          VMwareCertified  2016-04-19

rste                          2.0.2.0088-4vmw.600.0.0.2494585        VMware          VMwareCertified  2016-04-19

sata-ahci                      3.0-22vmw.600.1.17.3029758            VMware          VMwareCertified  2016-04-19

sata-ata-piix                  2.12-10vmw.600.0.0.2494585            VMware          VMwareCertified  2016-04-19

sata-sata-nv                  3.5-4vmw.600.0.0.2494585              VMware          VMwareCertified  2016-04-19

sata-sata-promise              2.12-3vmw.600.0.0.2494585              VMware          VMwareCertified  2016-04-19

sata-sata-sil24                1.1-1vmw.600.0.0.2494585              VMware          VMwareCertified  2016-04-19

sata-sata-sil                  2.3-4vmw.600.0.0.2494585              VMware          VMwareCertified  2016-04-19

sata-sata-svw                  2.3-3vmw.600.0.0.2494585              VMware          VMwareCertified  2016-04-19

scsi-aacraid                  1.1.5.1-9vmw.600.0.0.2494585          VMware          VMwareCertified  2016-04-19

scsi-adp94xx                  1.0.8.12-6vmw.600.0.0.2494585          VMware          VMwareCertified  2016-04-19

scsi-aic79xx                  3.1-5vmw.600.0.0.2494585              VMware          VMwareCertified  2016-04-19

scsi-fnic                      1.5.0.45-3vmw.600.0.0.2494585          VMware          VMwareCertified  2016-04-19

scsi-ips                      7.12.05-4vmw.600.0.0.2494585          VMware          VMwareCertified  2016-04-19

scsi-megaraid-mbox            2.20.5.1-6vmw.600.0.0.2494585          VMware          VMwareCertified  2016-04-19

scsi-megaraid-sas              6.603.55.00-2vmw.600.0.0.2494585      VMware          VMwareCertified  2016-04-19

scsi-megaraid2                2.00.4-9vmw.600.0.0.2494585            VMware          VMwareCertified  2016-04-19

scsi-mptsas                    4.23.01.00-9vmw.600.0.0.2494585        VMware          VMwareCertified  2016-04-19

scsi-mptspi                    4.23.01.00-9vmw.600.0.0.2494585        VMware          VMwareCertified  2016-04-19

uhci-usb-uhci                  1.0-3vmw.600.0.0.2494585              VMware          VMwareCertified  2016-04-19

vsanhealth                    6.0.0-3000000.2.0.1.17.2972216        VMware          VMwareCertified  2016-04-19

xhci-xhci                      1.0-3vmw.600.1.26.3380124              VMware          VMwareCertified  2016-04-19

tools-light                    6.0.0-1.26.3380124                    VMware          VMwareCertified  2016-04-19

[root@SRV-ESXI-2:~]



Ok i can't find it...you?

Reply
0 Kudos
J1mbo
Virtuoso
Virtuoso

Driver appears to be scsi-hpdsa 5.5.0.46-1OEM.550.0.0.1331820.

You could try the 5.5 driver (here) but I'd suggest opening a case with HP in this case first.

Reply
0 Kudos
AntonioFel
Contributor
Contributor

Mybe the better soloution is the downgrade to the 5.5....

Reply
0 Kudos