VMware Cloud Community
ben_turner_
Contributor
Contributor

HP FlexFabric 20Gb 2-port 650FLB - Gen9 networking inconsistency

I have come across an interesting issue with a new HPE platform. The system is running within a C7000 BladeSystem, with BL460c Gen9 blades.

We have noticed some degradation in performance on iSCSI connection (using the Software iSCSI initiator), this traffic runs over vmnic1 and vmnic2 details from the NIC list are below.

vmnic1  0000:06:00.1  elxnet  Up        Up       10000  Full32:a6:05:e0:00:be  1500  Emulex Corporation HPE FlexFabric 20Gb 2-port 650FLB Adapter
vmnic2  0000:06:00.2  elxnet  Up        Up       10000  Full32:a6:05:e0:00:bd  1500  Emulex Corporation HPE FlexFabric 20Gb 2-port 650FLB Adapter

Each NIC is reporting at 10000 Mb full, however I am not able to set the speed on the ESXi server. vmnic1 reports the following for advertised link modes;

[root@ESX:~] esxcli network nic get -n vmnic1

   Advertised Auto Negotiation: true

   Advertised Link Modes: 1000BaseKR2/Full, 10000BaseKR2/Full, 20000BaseKR2/Full, Auto

   Auto Negotiation: true

Where as vmnic2 reports the following modes

[root@ESXi2b-14:~] esxcli network nic get -n vmnic2

   Advertised Auto Negotiation: false

   Advertised Link Modes: 20000None/Full

   Auto Negotiation: false

Confused, the settings are identical for these within OneView. Both NIC's are using firmware - 12.0.1110.11 from SPP 2018.06.0. The HPE ESXi image has been used including driver version 12.0.1115.0 which shows as being compatible on the comparability guide VMware Compatibility Guide - I/O Device Search.

Has anyone else seen this issue? If I try and manually set the speed/duplex settings via esxcli it fails with the following error in the vmkernel.log

2018-08-14T23:49:41.361Z cpu20:65677)WARNING: elxnet: elxnet_linkStatusSet:7471: [vmnic2] Device is not privileged to do speed changes

As a result of this when using HCIBench to test the storage throughput the 95%tile_LAT value is reading excessively when traversing vmnic2 - 95%tile_LAT = 3111.7403 ms

Any thoughts??

Reply
0 Kudos
22 Replies
sgunelius
Hot Shot
Hot Shot

Did you ever find a resolution for this issue?  I'm moving some BL460c Gen9 hosts with the 650FLB to enclosures with the VCFF 20/40 F8 modules and am curious if I'll start seeing the same connectivity issues.  I'll be defining (3) connections for Management, vMotion & VM traffic and (2) for FC SAN.

Reply
0 Kudos
tvanholland
Contributor
Contributor

Not OP but we did end up resolving the issue by changing the port speed on our enclosure uplinks to auto negotiate instead of statically assigned. This was a change by our network team on the virtual port channel configuration. Haven't had any connectivity issues since.

Reply
0 Kudos
sgunelius
Hot Shot
Hot Shot

We started seeing PSODs with the BL460c Gen9 running 6.5 Update 1 and HPE identified an issue with the 650FLB driver in the HPE custom image.  We had only just started migrating to G3 enclosures and leveraging the 650FLB as a CNA.  We got the PSOD with the hosts under a relatively heavy load.  HPE recommended we upgrade to 6.5 Update 2, but confirmed the driver issue was resolved with 6.7 Update 1, so we rebuilt our BL460 Gen9s with HPE custom image VMware-ESXi-6.7.0-Update1-10302608-HPE-Gen9plus-670.U1.10.3.5.12-Oct2018.

We also fully patched them afterwards, which dropped down additional 9 patches, so our hosts are showing hypervisor VMware ESXi, 6.7.0, 11675023.  Knock on wood, no PSOD since then and I'm just about done migrating the BL460c Gen9 hosts off my original G1 enclosures.

Reply
0 Kudos