I have come across an interesting issue with a new HPE platform. The system is running within a C7000 BladeSystem, with BL460c Gen9 blades.
We have noticed some degradation in performance on iSCSI connection (using the Software iSCSI initiator), this traffic runs over vmnic1 and vmnic2 details from the NIC list are below.
vmnic1 0000:06:00.1 elxnet Up | Up | 10000 Full | 32:a6:05:e0:00:be 1500 Emulex Corporation HPE FlexFabric 20Gb 2-port 650FLB Adapter |
vmnic2 0000:06:00.2 elxnet Up | Up | 10000 Full | 32:a6:05:e0:00:bd 1500 Emulex Corporation HPE FlexFabric 20Gb 2-port 650FLB Adapter |
Each NIC is reporting at 10000 Mb full, however I am not able to set the speed on the ESXi server. vmnic1 reports the following for advertised link modes;
[root@ESX:~] esxcli network nic get -n vmnic1
Advertised Auto Negotiation: true
Advertised Link Modes: 1000BaseKR2/Full, 10000BaseKR2/Full, 20000BaseKR2/Full, Auto
Auto Negotiation: true
Where as vmnic2 reports the following modes
[root@ESXi2b-14:~] esxcli network nic get -n vmnic2
Advertised Auto Negotiation: false
Advertised Link Modes: 20000None/Full
Auto Negotiation: false
Confused, the settings are identical for these within OneView. Both NIC's are using firmware - 12.0.1110.11 from SPP 2018.06.0. The HPE ESXi image has been used including driver version 12.0.1115.0 which shows as being compatible on the comparability guide VMware Compatibility Guide - I/O Device Search.
Has anyone else seen this issue? If I try and manually set the speed/duplex settings via esxcli it fails with the following error in the vmkernel.log
2018-08-14T23:49:41.361Z cpu20:65677)WARNING: elxnet: elxnet_linkStatusSet:7471: [vmnic2] Device is not privileged to do speed changes
As a result of this when using HCIBench to test the storage throughput the 95%tile_LAT value is reading excessively when traversing vmnic2 - 95%tile_LAT = 3111.7403 ms
Any thoughts??
Did you ever find a resolution for this issue? I'm moving some BL460c Gen9 hosts with the 650FLB to enclosures with the VCFF 20/40 F8 modules and am curious if I'll start seeing the same connectivity issues. I'll be defining (3) connections for Management, vMotion & VM traffic and (2) for FC SAN.
Not OP but we did end up resolving the issue by changing the port speed on our enclosure uplinks to auto negotiate instead of statically assigned. This was a change by our network team on the virtual port channel configuration. Haven't had any connectivity issues since.
We started seeing PSODs with the BL460c Gen9 running 6.5 Update 1 and HPE identified an issue with the 650FLB driver in the HPE custom image. We had only just started migrating to G3 enclosures and leveraging the 650FLB as a CNA. We got the PSOD with the hosts under a relatively heavy load. HPE recommended we upgrade to 6.5 Update 2, but confirmed the driver issue was resolved with 6.7 Update 1, so we rebuilt our BL460 Gen9s with HPE custom image VMware-ESXi-6.7.0-Update1-10302608-HPE-Gen9plus-670.U1.10.3.5.12-Oct2018.
We also fully patched them afterwards, which dropped down additional 9 patches, so our hosts are showing hypervisor VMware ESXi, 6.7.0, 11675023. Knock on wood, no PSOD since then and I'm just about done migrating the BL460c Gen9 hosts off my original G1 enclosures.