VMware

This Question is Possibly Answered

1 "correct" answer available (10 pts)
10 Replies Last post: Nov 16, 2009 12:30 PM by d3aj  

ESX4 with HP NC510F 10Gbit adapter posted: Nov 6, 2009 1:16 AM

Click to view Poort443's profile Enthusiast
Poort443
25 posts since
Mar 12, 2009
I have a number of issues with the HP NC510F adapter. NFS datastores remain in "Inactive" state permanently. The datastores can be browsed, but that's all. VM's can't be started off it, and VMotions fail. The issues are resolved when I use a different adapter on the same vSwitch.

I've also had nothing but trouble with this adapter on ESX3.5. The NFS stores were properly connected, but VM's would freeze or be otherwise unreachable after a few hours or maybe a day. This is why I wanted to upgrade to ESX4, hoping that these problems would be resolved. All these issues happen on several servers, but all with the NC510F.

I'm working with VMware support to resolve these problems, but my question really is: Is anybody using this NC510F succesfully on ESX4?

This adapter is on the I/O Compatibilty Guide, and I'm using the driver that's stated there, the nx_nic 4.0.301

Tags: esx_4, networking, vsphere_4

Re: ESX4 with HP NC510F 10Gbit adapter

1. Nov 11, 2009 12:46 AM in response to: Poort443
Click to view Poort443's profile Enthusiast
Poort443
25 posts since
Mar 12, 2009
Anybody using the NC510F? Don't tell me I'm the only one..:|

Re: ESX4 with HP NC510F 10Gbit adapter

2. Nov 12, 2009 10:09 PM in response to: Poort443
Click to view d3aj's profile Enthusiast
d3aj
13 posts since
Apr 20, 2009
Please paste the vmkernel log output when loading the nx_nic driver module

Re: ESX4 with HP NC510F 10Gbit adapter

3. Nov 13, 2009 12:55 AM in response to: d3aj
Click to view Poort443's profile Enthusiast
Poort443
25 posts since
Mar 12, 2009
Attached the vmkernel log. There's actually nothing in it related to loading nx_nic. There also isn't anything in it about loading bnx2 (the broadcom adapter that's working fine). esxcfg-module -l does show it as loaded.

Could you please tell me if you're using the NC510F? That's as important to me as resolving the issue itself. I'm still working with VMware support for that.

Thanks.
Attachments:

Re: ESX4 with HP NC510F 10Gbit adapter

4. Nov 13, 2009 1:03 AM in response to: Poort443
Click to view d3aj's profile Enthusiast
d3aj
13 posts since
Apr 20, 2009

WARNING: NFS: 898: RPC error 12 (RPC failed) trying to get port for Mount Program (100005) Version (3) Protocol (TCP) on Server

That's not right...

Anyhow, could you check if you have backed up logs (vmkernel.1 perhaps?)

If not then reload the driver and there should be nx_nic related output in the vmkernel.

Re: ESX4 with HP NC510F 10Gbit adapter

5. Nov 13, 2009 1:49 AM in response to: d3aj
Click to view Poort443's profile Enthusiast
Poort443
25 posts since
Mar 12, 2009
I rebooted the host to make sure the module was loaded again. It does show some messages:

<4>nx_nicvmnic2*: Unable to destroy the rx context, code 17 Error: Timed Out***

The message is repeated for my other nc510f (vmnic3).
Attachments:

Re: ESX4 with HP NC510F 10Gbit adapter

6. Nov 13, 2009 2:14 AM in response to: Poort443
Click to view d3aj's profile Enthusiast
d3aj
13 posts since
Apr 20, 2009
You have not provided the logs that show the nx_nic module being loaded. I need to look at that to make an assessment. You mentioned you rebooted the system but looking at the attached logs I do not see the system being rebooted. Watchdog is kicking, and there are tx timeouts.. There is certainly something wrong with the netxen card/driver. Until I see the logs with the module being loaded I wont be able to help you.

Re: ESX4 with HP NC510F 10Gbit adapter

7. Nov 13, 2009 6:06 AM in response to: d3aj
Click to view Poort443's profile Enthusiast
Poort443
25 posts since
Mar 12, 2009

The only reference in a vmkernel log to loading the nx_nic is from a week ago, in vmkernel.27 The other more recent logs show nothing for nx_nic. If I run esxcfg-modules -l the nx_nic does however show up as loaded..

In the meantime I installed a vmware driver CD, the VMware ESX/ESXi 4.0 Driver CD for QLogic Intelligent Ethernet Adapters. This changed my vmware driver and firmware versions. The vmware driver is newer (now: 4.0.404 before: 4.0.301), and oddly enough the firmware seems older (now: 3.4.339 before:4.0.222. HP however reports that firmware and driver are matched: I quote from another thread:

Information
The HP website is not containing any firmware for the NetXen based HP 10 GB network cards (NIC) like the NC510 and the NC512.

Reports have been submitted to HP support where people where trying to use the Redhat or SLES firmware package from the HP website in combination with the NetXen boot ISO /CD. The HP NetXen based 10 GB network card firmware was placed on a USB key or was added to the NetXen ISO / CD and was used for trying to flash and upgrade the HP 10 GB NIC firmware

NetXen based 10 GB network cards are using host-based firmware. This means that the Vmware driver consits of a driver part and a firmware image part. The moment the Vmware driver is loaded the host based firmware on teh NetXen card is automaticcally running the firmware level as supplied in the Vmware driver.

Due to the above it is not required to update the HP 10 GB based NetXen cards with a firmware package meant for Redhat or SLES

It all makes no difference unfortunately, the card still does not work at all.

Attachments:

Re: ESX4 with HP NC510F 10Gbit adapter

8. Nov 13, 2009 11:28 AM in response to: Poort443
Click to view d3aj's profile Enthusiast
d3aj
13 posts since
Apr 20, 2009
The discrepancy with the firmware versions happens due to the firmware being dynamically loaded onto the card. Part of the firmware resides in the driver.

Re: ESX4 with HP NC510F 10Gbit adapter

9. Nov 15, 2009 2:49 AM in response to: d3aj
Click to view Poort443's profile Enthusiast
Poort443
25 posts since
Mar 12, 2009
I've loaded the driver manually again, there are now some entries in the vmkernel log. Loading the driver was done at november 15, 11:40. I'd appreciate if you could look at these.
Attachments:

Re: ESX4 with HP NC510F 10Gbit adapter

10. Nov 16, 2009 12:30 PM in response to: Poort443
Click to view d3aj's profile Enthusiast
d3aj
13 posts since
Apr 20, 2009

Thanks Poort. I will look into this.

Here's a more technical explanation of the firmware versions:

"The vmware driver is newer (now: 4.0.404 before: 4.0.301), and oddly
enough the firmware seems older (now: 3.4.339 before:4.0.222."

Since your NX2031 card is flashed with an
older firmware 3.4.336 the firmware is loaded from the driver (fw file). The older
driver 4.0.301 has version NX2031 firmware file 4.0.222 and the newer
driver 4.0.404 has firmware version 3.4.339. This is not an issue as it behaves correctly, only a difference in versions.

Actions

VMware Developer

SDKs, APIs, Videos, Learn and much more in the Developer community.

Learn More

Developer Sample Code

Increase your developer productivity with VMware API sample code.

Learn More

VMworld Sessions & Labs

Online access to the latest VMworld Sessions & Labs and online services.

Learn more

Purchase PSO Credits Online

Purchase credits to redeem training and consulting services online.

Buy Now

Community Hardware Software

View reported configurations or report your own.

Learn More

VMware vSphere

Come witness the next giant leap in virtualization.

Register Today

Communities