Anyone using Intel based10gbe with ESX 3.5?

[Cloneranger]

I dont think the cards are in VMs HCL, but intel say they support the card under ESX 3.5,

So I assume it will work,

I am having difficulty loading drivers for the card, I am not that well versed in linux,

Can anyone out there give me a few pointers?

Thanks all!

PS: the card is based on the 82598 controller,

[kams]

Hello, I'm currently an Intel EXPX9502CX4 with ESX 3.5 and it's not detecting - I've installed all patches, but no luck. I'm assuming that I need the 'ixgbe' driver, but so far I haven't found instrutions on how to install this under VMWare - do instructions exist?

Thank you,

Kevin

[azn2kew]

Have anyone look at Neterion product. It is made for 10GB bandwidth network and it has built in management where you can control bandwidth for each ESX host or machines. You have to check them out for details.

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

Regards,

Stefan Nguyen

iGeek Systems Inc.

VMware, Citrix, Microsoft Consultant

[kams]

actually, I'm using the Neterion cards as well - they detect right away, and are working great. However, they're considerably more expensive than the Intel, only have one port where the Intel has two, and the Intel card is also supposed to support ISCSI. My plan was to use the Intel cards to add a bit of flexibility to my setup, but if they're not gonna detect I may have a problem . . .

[rmrobert]

Can you run lspci and return the results which match this card? I can check and see for certain which driver supports this card.

[kams]

Here you go - and thank you for your help! I'm including more verbose output for thenics at the end.

00:00.0 Host bridge: Intel Corporation: Unknown device 25d8 (rev b1)

00:02.0 PCI bridge: Intel Corporation: Unknown device 25e2 (rev b1)

00:03.0 PCI bridge: Intel Corporation: Unknown device 25e3 (rev b1)

00:04.0 PCI bridge: Intel Corporation: Unknown device 25f8 (rev b1)

00:05.0 PCI bridge: Intel Corporation: Unknown device 25e5 (rev b1)

00:06.0 PCI bridge: Intel Corporation: Unknown device 25f9 (rev b1)

00:07.0 PCI bridge: Intel Corporation: Unknown device 25e7 (rev b1)

00:10.0 Host bridge: Intel Corporation: Unknown device 25f0 (rev b1)

00:10.1 Host bridge: Intel Corporation: Unknown device 25f0 (rev b1)

00:10.2 Host bridge: Intel Corporation: Unknown device 25f0 (rev b1)

00:11.0 Host bridge: Intel Corporation: Unknown device 25f1 (rev b1)

00:13.0 Host bridge: Intel Corporation: Unknown device 25f3 (rev b1)

00:15.0 Host bridge: Intel Corporation: Unknown device 25f5 (rev b1)

00:16.0 Host bridge: Intel Corporation: Unknown device 25f6 (rev b1)

00:1c.0 PCI bridge: Intel Corporation: Unknown device 2690 (rev 09)

00:1c.1 PCI bridge: Intel Corporation: Unknown device 2692 (rev 09)

00:1d.0 USB Controller: Intel Corporation: Unknown device 2688 (rev 09)

00:1d.1 USB Controller: Intel Corporation: Unknown device 2689 (rev 09)

00:1d.2 USB Controller: Intel Corporation: Unknown device 268a (rev 09)

00:1d.3 USB Controller: Intel Corporation: Unknown device 268b (rev 09)

00:1d.7 USB Controller: Intel Corporation: Unknown device 268c (rev 09)

00:1e.0 PCI bridge: Intel Corporation 82801BA/CA/DB/EB PCI Bridge (rev d9)

00:1f.0 ISA bridge: Intel Corporation: Unknown device 2670 (rev 09)

00:1f.1 IDE interface: Intel Corporation: Unknown device 269e (rev 09)

01:03.0 VGA compatible controller: ATI Technologies Inc: Unknown device 515e (rev 02)

01:04.0 System peripheral: Compaq Computer Corporation: Unknown device b203 (rev 03)

01:04.2 System peripheral: Compaq Computer Corporation: Unknown device b204 (rev 03)

01:04.4 USB Controller: Hewlett-Packard Company: Unknown device 3300

01:04.6 Class 0c07: Hewlett-Packard Company: Unknown device 3302

02:00.0 PCI bridge: ServerWorks: Unknown device 0103 (rev c3)

03:00.0 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T (rev 12)

04:00.0 PCI bridge: ServerWorks: Unknown device 0103 (rev c3)

05:00.0 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T (rev 12)

06:00.0 RAID bus controller: Hewlett-Packard Company Smart Array P400 (rev 04)

09:00.0 PCI bridge: Intel Corporation: Unknown device 3500 (rev 01)

09:00.3 PCI bridge: Intel Corporation: Unknown device 350c (rev 01)

0a:00.0 PCI bridge: Intel Corporation: Unknown device 3510 (rev 01)

0a:01.0 PCI bridge: Intel Corporation: Unknown device 3514 (rev 01)

0a:02.0 PCI bridge: Intel Corporation: Unknown device 3518 (rev 01)

0b:00.0 Ethernet controller: Intel Corporation: Unknown device 10ec (rev 01)

0b:00.1 Ethernet controller: Intel Corporation: Unknown device 10ec (rev 01)

0e:00.0 RAID bus controller: Hewlett-Packard Company Smart Array P400 (rev 03)

________________________________________

0b:00.0 Ethernet controller: Intel Corporation: Unknown device 10ec (rev 01)

Subsystem: Intel Corporation: Unknown device a01f

Flags: bus master, fast devsel, latency 0, IRQ 17

Memory at fdde0000 (32-bit, non-prefetchable)

Memory at fdd80000 (32-bit, non-prefetchable)

I/O ports at 5000

Memory at fdd70000 (32-bit, non-prefetchable)

Capabilities: Power Management version 3

Capabilities: Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-

Capabilities: #11

Capabilities: #10

0b:00.1 Ethernet controller: Intel Corporation: Unknown device 10ec (rev 01)

Subsystem: Intel Corporation: Unknown device a01f

Flags: bus master, fast devsel, latency 0, IRQ 16

Memory at fdd40000 (32-bit, non-prefetchable)

Memory at fdd00000 (32-bit, non-prefetchable)

I/O ports at 5020

Memory at fdcf0000 (32-bit, non-prefetchable)

Capabilities: Power Management version 3

Capabilities: Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-

Capabilities: #11

Capabilities: #10

Thanks again,

Kevin

[rmrobert]

Don't quote me on this, but it looks like we support

#define IXGBE_DEV_ID_82598AF_DUAL_PORT 0x10C6

#define IXGBE_DEV_ID_82598AF_SINGLE_PORT 0x10C7

#define IXGBE_DEV_ID_82598EB_CX4 0x10DD

Your device is:

10ec 10 Gigabit AT CX4 Network Connection .

It doesn't look like our ixgbe supports that yet. I think it was very recently added to the linux driver. I would expect that a patch or update release may enable this in the future, but I'm sorry I don't know for sure about that, or when.

[coltex-support]

I am running into the same problem, I have requested the same fix for the ixgbe driver.

The ixgbe driver in vmware 3i U2 is 1.3.7.4, the driver shipped with the cx4 cards is 1.3.20.x. That one atleast supports the card normally. This driver is from march 2008, which is 6 months ago.

It's the same as the dual port SR optic card, but just with copper connectors.

The current driver on the intel site is 1.3.33.x

Please update the driver so we can use our investment, please.

Kind regards,

Seth Mos

[kams]

Hello, I've still not seen any movement on drivers for this card - anyone have any word? Very strange - the Intel product brief indicates VMWare support for the card: http://www.intel.com/Assets/PDF/prodbrief/10-GbE-CX4-DualPort-product-brief.pdf

[rmrobert]

I looked at that document and it doesn't list VMware ESX as a supported operating system. It mentions it was designed for virtualization and has Intel's VMDq technology, which obviously means that they intend for it to be used with virtualization systems such as ESX. I will ask around internally to see if there is any plan to support this in a patch or update release for ESX35. You may try asking through other channels- support request or customer representative as well.

[coltex-support]

Hi,

The current state of affairs is that VMware support has accepted my new Feature Request by SR #1137261431. Karen Egan is currently handling this support request. This support request was made on 22nd september 2008, it was escalated to Karen Egan on the 29th september 2008. The current date is 21st october 2008. So it's been almost a month since.

Repeated requests by voicemail for a status update have been left unanswered (3). The whole issue revolves around updating the ixgbe driver which is now shipped as part of ESX 3.5 since the update 1 release. The strangest thing is that when I asked around to update this driver so that it supports al 10 something models it was denied. Meanwhile I see updates for other network cards like the tg3 driver to support new cards and include bugfixes. Why can this same procedure not be followed for the ixgbe driver?

What I have gathered so far is that supposedly Intel and VMware would be certifying the cards. Although all I need is a updated ixgbe driver to support the other cards based on the same 82598 chipset.

I suspect someone is just making life hard for the sake of it. Something as entirely mundane as a driver update appears to require moving mountains.

Kind regards,

Coltex Support

[iviemeister]

For what it's worth, we just ran into this as well -- bad assumption on our part that since the same chipset was used in the EXPX9502CX4 as the other supported Intel 10-GigE cards on the HCL.....

We have six of these just waiting for the driver to be updated.

[Dave_Mishchenko]

Your post has been moved to the VI: ESX 3.5 forum.

Dave Mishchenko

VMware Communities User Moderator

[coltex-support]

I have good and bad news, the bad news is that my voicemail to our support representitive has still been unanswered. The good news is that our support request is currently displaying a "Waiting on Engineering".

There is hope on the horizon!

If all goes well this should imply the cards will be supported within a reasonable timeframe.

Fingers crossed!

[jonmckinney]

I've picked up the same cards only to find they don't work. Just wanted to see if you had any updates with your request coltex?

[coltex-support]

Hi,

Vmware has release a driver disk with a newer ixgbe driver that should support pretty much all models available.

After

installing this driver disk the system no longer boots. There is a

version discrepancy in the modules. This does however mean that it

fails at the initrd stage and you can't load the storage driver. And

without storage it can't mount it's root filesystem.

There is a

open prio 1 SR for this but the 1st person I got did not understand the

nature of the issue and thus we wasted a work day.

Cheers

[jonmckinney]

I'm getting the same doing an install booting from the driver disk as well. Hope they fix it soon.

[coltex-support]

What you can do is install the VMware-drivers-ixgbe only.

rpm -ivh --force VMware-esx-drivers-net-ixgbe-350.1.3.31.5-73033.i386.rpm

Preparing... ###########################################

1:VMware-esx-drivers-net-###########################################

# vmkload_mod ixgbe

Using /usr/lib/vmware/vmkmod/ixgbe.o

succeeded

#

Now it works!

And it doesn't break your machine.