Only one nic avaliable for ESX?

[obleak]

I was wondering if someone could help me out here, I'm a little stuck.

I have a HP DL360 with a Dual Broadcom Gigabit NIC's running a fully patched version of ESX 3.0.1.

For some unknown reason only one NIC is available...

From my investigation of the issue it looks like the 2nd nic has no kernel module associated with it.

Some info:

lspci[/u]

02:02.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 10)

02:02.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 10)

/etc/vmware/esx.conf[/u]

/device/002:02.0/class = "020000"

/device/002:02.0/devID = "1648"

/device/002:02.0/irq = "24"

/device/002:02.0/name = "Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 10)"

/device/002:02.0/options = ""

/device/002:02.0/owner = "vmkernel"

/device/002:02.0/subsysDevID = "00d0"

/device/002:02.0/subsysVendor = "0e11"

/device/002:02.0/vendor = "14e4"

/device/002:02.0/vmkname = "vmnic0"

/device/002:02.1/class = "020000"

/device/002:02.1/devID = "1648"

/device/002:02.1/irq = "25"

/device/002:02.1/name = "Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 10)"

/device/002:02.1/options = ""

/device/002:02.1/owner = "vmkernel"

/device/002:02.1/subsysDevID = "00d0"

/device/002:02.1/subsysVendor = "0e11"

/device/002:02.1/vendor = "14e4"

/device/002:02.1/vmkname = "vmnic1"

esxcfg-module -l[/u]

Device Driver Modules

Module Type Enabled Loaded

vmkapimod vmkapimod true true

vmklinux linux true true

cciss scsi true true

tg3 nic true true

/proc/vmware/pci[/u] (note vmnic1 has no module)

002:02.0 14e4:1648 0e11:00d0 Ethernet Broadcom 5/ 24/0xb9 A V tg3 vmnic0

002:02.1 14e4:1648 0e11:00d0 Ethernet Broadcom 255/ 25/0xc1 B V vmnic1

esxcfg-info[/u] (again vmnic1 has no module)

\==+PciDeviceImpl :

\ |----Bus.............................................0x02

\ |----Slot............................................0x02

\ |----Function........................................0x00

\ |----Vendor Id.......................................0x14e4

\ |----Device Id.......................................0x1648

\ |----Sub-Vendor Id...................................0x0e11

\ |----Sub-Device Id...................................0x00d0

\ |----Vendor Name.....................................Broadcom Corporation

\ |----Device Name.....................................NC7782 Gigabit Server Adapter (PCI-X, 10,100,1000-T)

\ |----Device Class....................................512

\ |----PIC Line........................................5

\ |----Service Console IRQ.............................24

\ |----Vector..........................................185

\ |----PCI Pin.........................................0

\ |----Spawned Bus.....................................0

\ |----Flags...........................................1

\ |----Module Id.......................................4

\ |----Chassis.........................................0

\ |----Physical Slot...................................0

\ |----VmKernel Device Name............................vmnic0

\==+PciDeviceImpl :

\ |----Bus.............................................0x02

\ |----Slot............................................0x02

\ |----Function........................................0x01

\ |----Vendor Id.......................................0x14e4

\ |----Device Id.......................................0x1648

\ |----Sub-Vendor Id...................................0x0e11

\ |----Sub-Device Id...................................0x00d0

\ |----Vendor Name.....................................Broadcom Corporation

\ |----Device Name.....................................NC7782 Gigabit Server Adapter (PCI-X, 10,100,1000-T)

\ |----Device Class....................................512

\ |----PIC Line........................................255

\ |----Service Console IRQ.............................25

\ |----Vector..........................................193

\ |----PCI Pin.........................................0

\ |----Spawned Bus.....................................0

\ |----Flags...........................................1

\ |----Module Id.......................................0

\ |----Chassis.........................................0

\ |----Physical Slot...................................0

\ |----VmKernel Device Name............................vmnic1

/var/log/vmkernel[/u] (bootup)

Apr 30 23:24:06 hostname vmkernel: 0:00:00:06.804 cpu2:1033)PCI: 1325: 002:02.0 named 'vmnic0' (was '')

Apr 30 23:24:06 hostname vmkernel: 0:00:00:06.804 cpu2:1033)LinPCI: 47: 002:02.0 vmnic0

Apr 30 23:24:06 hostname vmkernel: 0:00:00:06.804 cpu3:1034)PCI: 1325: 002:02.1 named 'vmnic1' (was '')

Apr 30 23:24:06 hostname vmkernel: 0:00:00:06.804 cpu3:1034)LinPCI: 47: 002:02.1 vmnic1

Apr 30 23:24:06 hostname vmkernel: tg3: vmnic0: Flow control is on for TX and on for RX.

Apr 30 23:24:06 hostname vmkernel: 0:00:00:35.248 cpu0:1024)Uplink: 1291: Setting capabilities 0xb for device vmnic0

Apr 30 13:27:14 hostname vmkernel: 0:00:03:57.466 cpu3:1056)Uplink: 1291: Setting capabilities 0x0 for device vmnic0

Apr 30 13:27:46 hostname vmkernel: 0:00:04:28.805 cpu3:1061)Uplink: 1291: Setting capabilities 0x0 for device vmnic0

So how does one associate the tg3 module against vmnic1?

I can't find any esxcfg-* command that will do it

[bister]

What information does "esxcfg-nics -l" give?

[Dave_Mishchenko]

Looks like both drivers are running OK. By default the install won't configure the 2nd or other NIC. In the VI client if you go to configuration \network adapters, do you see both vmnic0 and vmnic1. If so you can then

1) Add vmnic1 to your existing vswitch - you'll want to look at the implications of network teaming before you do.

2) You could create a second vswitch and assign vmnic1 to it. You could then add a vmkernel port to it and start to run VMs over the NIC. Adding a second vswitch might be helpful if you want to put the NIC in a DMZ.

[obleak]

What information does "esxcfg-nics -l" give?

esxcfg-nics -l[/u]

Name PCI Driver Link Speed Duplex Description

vmnic0 02:02.00 tg3 Up 100Mbps Full Broadcom Corporation NC7782 Gigabit Server Adapter (PCI-X, 10,100,1000-T)

[obleak]

Looks like both drivers are running OK. By default

the install won't configure the 2nd or other NIC. In

the VI client if you go to configuration \network

adapters, do you see both vmnic0 and vmnic1.

I just see vmnic0

[bister]

Do you have HP SIM agents installed? If so: Is there any information? Is the 2nd NIC enabled in the BIOS? Normally I'd think only both NICs of a dual-port adapter will fail but not a single one, but you never know: Is the hardware OK?

[obleak]

Do you have HP SIM agents installed? If so: Is there

any information? Is the 2nd NIC enabled in the BIOS?

Normally I'd think only both NICs of a dual-port

adapter will fail but not a single one, but you never

know: Is the hardware OK?

No HPSIM installed, but it's on the list to do.

Good point about the BIOS - hadn't though of looking in there. When I loaded ESX I was asked which nic to configure for the console so just naturally assumed both were enabled

The hardware is OK AFAIK, the box was a windows 2k3 server and didn't have any issues.

I won't be able to bounce the box for a few hours (nearly 4pm here ATM) but will let you know what I find. Gives me the chance to load HPSIM at the same time.

regards,

Fraser

[bister]

HPSIM agents do not need a reboot. What generation of DL360 is it what you are using?

When the ESX installer prompts for two NIC I assume the hardware is OK and also enabled in BIOS.... So I'd install the agents and see what they sey...

Regards,

Christian

[obleak]

HPSIM agents do not need a reboot. What generation of

DL360 is it what you are using?

When the ESX installer prompts for two NIC I assume

the hardware is OK and also enabled in BIOS.... So

I'd install the agents and see what they sey...

It's a ProLiant DL360 G4p

Ok will give the agents a shot.

thanks

[obleak]

I'd install the agents and see what they sey...

Well the agents just reported 1 nic too.

so I bounced the box and checked the BIOS

and guess what - you were right, the nic was disabled in the bios.

beats me how a disabled device is detectable & configurable (in that it had a vmnic assigned to it by esx)

esxcfg-nics -l[/u]

Name PCI Driver Link Speed Duplex Description

vmnic0 02:02.00 tg3 Up 100Mbps Full Broadcom Corporation NC7782 Gigabit Server Adapter (PCI-X, 10,100,1000-T)

vmnic1 02:02.01 tg3 Up 100Mbps Full Broadcom Corporation NC7782 Gigabit Server Adapter (PCI-X, 10,100,1000-T)

anyway happy now, thanks for your help

[obleak]

HP bios seems to 'fake' disabling devices.

They still get assigned resources (IRQ's etc) and are detectable by the OS.

[bister]

You're welcome!