10 Gibabits Ethernet NIC on ESX 3.0.1 (HP NC510F) - Does it work ?

[spilette]

Hi,

My company recently bought HP DL380 with 10Gbits Ethernet Cards (NC510F). We are nowplanning to deploy VI3 on these servers (ESX + VC), but can't find any reliable info on the fact that 10Gb ethernet NIC actually work on ESX 3 ...

I know 10gigabits ehternet is not really necessary for the moment, as nic teaming can do the trick for high bandwidth. But the fact is that those cards are already bought, no way to make the provider change them for a bunch of quad 1000 cards, so ...

Has anyone one of you made the test ? What's the result ?

If it's not currently supported, is there a way to port a linux driver to make it work with ESX (console + vmkernel) ? and if so, what's the process to follow ?

Also, does the support contract features something like RedHat ones, i.e porting drivers on demand from customers, with/without full support ?

Thanks a lot !

Regards,

[mwheeler1982]

As you can probably guess, that NIC isn't supported on the VMware HCL. So, as far as using it in production, I wouldn't recommend it.

However, If you have some good linux people in your enterprise, you could try doing the following:

This PDF says that ESX 3.0.1 is based on RHEL 3 Update 6. So, if you can get ahold of this version of RHEL, install it somewhere and build the driver you need, there's a possibility it may work. That should get it working with the service console..

I'm not sure if that will also get you going with vmkernel. There is some pretty tight interactions between it and the COS. It would probably add too much bloat to the vmkernel to compile in all the drivers for all the possible hardware, so it may get its drivers from the COS. But i'm only speculating...

From what I've heard, VMware is pretty strict about adhering to the HCL. I doubt you'd get much help from them.

[spilette]

Thanks for answer.

Well, in fact, I already thought of building en RHEL3U6 driver, but the problem is that drivers provided by HP and NIC chip manufacturer are designed for 2.6 kernels, so I really doubt it can work easily ... I'll give a try anyway, I need to be sure !

Does anyone know if VMware is planning to include support for 10giga etherne in the next few months ? Or did anyone heard of 10giga nics working with ESX ?

Thanks !

Regards,

[Jae_Ellers]

Unless you're Paul LaLonde (or equiv. wetware) mucking about with drivers in ESX isn't recommended.

Submit a VMware SR requesting supported 10G adapters. Until one appears in this document: http://www.vmware.com/pdf/vi3_io_guide.pdf forget it unless you want to reverse engineer a lot more than a RHEL3U6 system. In addition to service console (which IS based on RHEL3U6) you will need vmkernel (which isn't RHEL3U6) drivers for your device.

Submit a feature request here http://www.vmware.com/community/forum.jspa?forumID=26 as well. If there are hundreds more chiming in for the same feature VMware will probably still ignore it, but at least you'll feel better knowing you're not the only one missing out.

Alternatively, if you require this kind of BW, use a HP bladecenter (or equivalent) and I think you should be able to get 8 Gb of BW into a 680c or 685c via virtualconnect. Virtualconnect blade enclosure i/o modules support 10G connections. Massimo, if IBM has similar capabilities feel free to chime in.

Of course, it will cost you at least $16k for 4 VC modules and another $4k for an enclosure, but at least you'll have something to plug your 10G ports into.

I think it would be cheaper to team up a couple of the 4x e1000 boards in your DL and eat the GbE ports.

[jasonboche]

Don't forget this resource: Community supported HCL

http://www.vmware.com/vmtn/resources/communitysupport/

Use it to find out if someone has gotten these NIC cards to work.

Or blaze a new trail, get the NIC cards to work, then update the Community supported HCL to provide others with the benefits of your labor.

[meistermn]

Hi,

in this whitepaper http://www.vmware.com/pdf/vi_performance_tuning.pdf

is writen on page 8 over 10Gig Ethernet network adapters.

"In addition to the PCI and PCI-X architectures, we now have the PCIe architecture.

10Gig Ethernet network adapters should ideally use PCIe-8x or PCI-X 266.

Preferably, there is no "bridge chip" (e.g. PCI-X to PCIe, or PCIe to PCI-X) in the

path to the actual Ethernet device (including any embedded bridge chip on the

device itself) that can lower performance."

Maybe it will work with an intel 10 gigcard, if you use the e1000 driver in the virtual machine.

Or ask if it will work in the current esx beta 3.0.X .

[spilette]

Hello,

Thanks for your answers ! It's really helpful !

Something more in this paper (performance tuning) is this :

"With the more widespread adoption of 10 Gigabit Ethernet

(support for it exists in ESX 3.0), the performance scales may shift towards NAS

and iSCSI with these 10G deployments"

Does "support for it exists in ESX 3.0" means what I think ?

I'll do the install next week, and see if this damned nic can get recongnized ... crossing fingers !

I'll let you know !

Thanks again, your answers are very helpful.

Regards,

[Jae_Ellers]

File and SR and ask. That's the best way to confirm or deny any hopes you've had. If I'm out on the fringes I usually do this when I know suspicious hardware is on the way to save time beating on something that will simply not work.

I do support and appreciate user supported hardware, I just wanted to point out as a reality check that you shouldn't expect to copy the RHEL3U6 drivers in and do a build for full support.

Let us know how it goes.

[mwheeler1982]

I was just reading over an e-mail I got regarding VMWorld 2007. Of note (emphasis added by me):

"

As a preview, the following sessions will be offered in the agenda:

• Tips for Troubleshooting VI3 Faults

• VI3 Advanced Diagnostics Logs Analysis

• Exchange and SQL on ESX – Best Practices on Virtualization on

Transactional Application Workloads

* High Performance Virtualized I/O in 10Gigabit Ethernet Era[/b]

• NFS & iSCSI: Performance Characterization and Best Practices

"

[wila]

Hello,

Thanks for your answers ! It's really helpful !

Something more in this paper (performance tuning) is

this :

"With the more widespread adoption of 10 Gigabit

Ethernet

(support for it exists in ESX 3.0), the performance

scales may shift towards NAS

and iSCSI with these 10G deployments"

Does "support for it exists in ESX 3.0" means what I

think ?

Yip, it means that the marketing/documentation department is ahead of the released versions and talk about in house stuff. As in "a slip of the tongue", like that some little bird told me that SATA support comes with 3.2 (or 3.02 whatever)... woops

[meistermn]

You have to wait till VMWARE World 2007

Agenda: http://www.vmware.com/vmworld/agenda/tracks_sessions.html#c5725

Technology & Architecture

Exploring Virtual Infrastructure Technology & Architecture in Depth

Top Support Issues and How to Solve Them

The Next Generation Virtual Network

High Performance Virtualized I/O in 10 Gigabit Ethernet Era

NFS & iSCSI: Performance Characterization and Best Practices

Using the Secure Technical Implementation Guide (STIG)

with Virtual Infrastructure 3