Intel PRO/1000 NICs not working

[MarcBouchard]

New installation of ESX4 in our lab on "unsupported" hardware. The system has 3 NICs. 2 Intel PRO/1000 and an onboard Broadcom NIC. At installation I noticed something strange but figured I'd run thru it anyway. To summarize the issue, only one NIC works (vmnic2, one of the Intel adapters). On the physical switch, I see the link is up on all 3, but ESX says two of them are down

I need those extra adapters for iSCSI support etc...

Any help would be much appreciated!

[MarcBouchard]

.

[Dave_Mishchenko]

What sort of hardware do you have and would it be an Intel Pro / 1000 MT?

[MarcBouchard]

Intel motherboard. The NICs are Gigabit Intel PRO 1000 PT cards, PCI-E if I recall correctly. This setup worked A1 with 3.5 (I have a 2nd identical server still running with it right now)... ESX 4 supports the NIC as one of them works...

This is the NIC...

[MarcBouchard]

Nobody has any idea? Not even a suggestion?

[Dave_Mishchenko]

I'd start with running lspci -p to see if the driver is loaded OK for the NIC. Also take a look at this thread and it's why I asked about it being an MT - http://www.vm-help.com/forum/viewtopic.php?f=17&t=283.

[HughBorg707]

I don't see where you specified that the Broadcom was also gigabit. If its 10/100 it won't be supported under ESX 4.

Could you try pulling the Intel card out, boot the system and see if the Broadcom will detect. If it does, shut down again and then put the Intel card back in and see how that goes. Sounds kinda fishy like a device conflict of some kind that would cause you to only see 1 port of the 2 port card. Also you may, just to troubleshoot, try disabling the onboard card in the BIOS to see what effect that has on the Intel.

[AndreTheGiant]

Have you tryed to remove a card, and see what's happen?

You say that one of your card is recongnized as vmnic2, but where are vmnic0 and 1?

What's the output of ifconfig command?

Andre

**if you found this or any other answer useful please consider allocating points for helpful or correct answers

[MarcBouchard]

Ok. I didn't post the right information regarding the 3rd adapter (it's not a broadcom).

Firstly, I doubt there is any conflict as this system has been working for over a year with no issues whatsoever on ESX 3.5

Here are the specs of the system (remember it's a test lab).

Intel DQ35JOE Motherboard

- built in Gigabit (10/100/1000 Mbits/sec) LAN subsystem using the Intel® 82566DM Gigabit Ethernet Controller

- 2 Intel PRO/1000PT NICs

During installation, the 3 adapters are visible, but only one is seen as 'Connected' by ESX 4. On ESX3.5, only the 2 PRO/1000PT worked.

I can't disable the onboard NIC in the BIOS (No idea why, I can't even select the option, it jumps over it!)

vmnic0 and 1 are there, everything looks just fine, it just won't see that it's connected. On the physical switch, I see the link is up and I tried different ports to rule out defective ports.

Here is the dump of ifconfig

-

---

# ifconfig

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

UP LOOPBACK RUNNING MTU:16436 Metric:1

RX packets:404488 errors:0 dropped:0 overruns:0 frame:0

TX packets:404488 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:456538479 (435.3 MiB) TX bytes:456538479 (435.3 MiB)

vmnic0 Link encap:Ethernet HWaddr 00:1C:C0:82:3A:D1

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:0 errors:0 dropped:0 overruns:0 frame:0

TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

Interrupt:33

vmnic1 Link encap:Ethernet HWaddr 00:1B:21:29:7A:E0

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:0 errors:0 dropped:0 overruns:0 frame:0

TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

Interrupt:41

vmnic2 Link encap:Ethernet HWaddr 00:1B:21:29:7A:98

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:2421457 errors:0 dropped:0 overruns:0 frame:0

TX packets:1693382 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:3091778760 (2.8 GiB) TX bytes:369939234 (352.8 MiB)

Interrupt:49

vswif0 Link encap:Ethernet HWaddr 00:50:56:4E:B9:25

inet addr:10.0.0.10 Bcast:10.0.0.255 Mask:255.255.255.0

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:104176 errors:0 dropped:0 overruns:0 frame:0

TX packets:56236 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:32290234 (30.7 MiB) TX bytes:21763283 (20.7 MiB)

#

[MarcBouchard]

And here's the lspci dump (verbose)

# lspci -v

00:00.0 Host bridge: Intel Corporation Unknown device 29b0 (rev 02)

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, fast devsel, latency 0

Capabilities: Vendor Specific Information

00:02.0 VGA compatible controller: Intel Corporation Unknown device 29b2 (rev 02) (prog-if 00 )

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, fast devsel, latency 0, IRQ 11

Memory at e0580000 (32-bit, non-prefetchable)

I/O ports at 4430

Memory at d0000000 (32-bit, prefetchable)

Memory at e0400000 (32-bit, non-prefetchable)

Capabilities: Message Signalled Interrupts: 64bit- Queue=0/0 Enable-

Capabilities: Power Management version 2

00:02.1 Display controller: Intel Corporation Unknown device 29b3 (rev 02)

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, fast devsel, latency 0

Memory at e0500000 (32-bit, non-prefetchable)

Capabilities: Power Management version 2

00:03.0 Communication controller: Intel Corporation Unknown device 29b4 (rev 02)

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, fast devsel, latency 0, IRQ 11

Memory at e0626100 (64-bit, non-prefetchable)

Capabilities: Power Management version 3

Capabilities: Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-

00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network Connection (rev 02)

Subsystem: Intel Corporation Unknown device 0001

Flags: bus master, fast devsel, latency 0, IRQ 9

Memory at e0600000 (32-bit, non-prefetchable)

Memory at e0624000 (32-bit, non-prefetchable)

I/O ports at 4400

Capabilities: Power Management version 2

Capabilities: Message Signalled Interrupts: 64bit+ Queue=0/0 Enable+

Capabilities: Vendor Specific Information

00:1a.0 USB Controller: Intel Corporation Unknown device 2937 (rev 02) (prog-if 00 )

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, medium devsel, latency 0, IRQ 10

I/O ports at 40e0

Capabilities: Vendor Specific Information

00:1a.1 USB Controller: Intel Corporation Unknown device 2938 (rev 02) (prog-if 00 )

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, medium devsel, latency 0, IRQ 11

I/O ports at 40c0

Capabilities: Vendor Specific Information

00:1a.2 USB Controller: Intel Corporation Unknown device 2939 (rev 02) (prog-if 00 )

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, medium devsel, latency 0, IRQ 9

I/O ports at 40a0

Capabilities: Vendor Specific Information

00:1a.7 USB Controller: Intel Corporation Unknown device 293c (rev 02) (prog-if 20 )

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, medium devsel, latency 0, IRQ 9

Memory at e0625c00 (32-bit, non-prefetchable)

Capabilities: Power Management version 2

Capabilities: Debug port

Capabilities: Vendor Specific Information

00:1b.0 Class 0403: Intel Corporation Unknown device 293e (rev 02)

Subsystem: Intel Corporation Unknown device 0012

Flags: bus master, fast devsel, latency 0, IRQ 10

Memory at e0620000 (64-bit, non-prefetchable)

Capabilities: Power Management version 2

Capabilities: Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-

Capabilities: Express Unknown type IRQ 0

00:1c.0 PCI bridge: Intel Corporation Unknown device 2940 (rev 02) (prog-if 00 )

Flags: bus master, fast devsel, latency 0

Bus: primary=00, secondary=01, subordinate=01, sec-latency=0

I/O behind bridge: 00003000-00003fff

Memory behind bridge: e0300000-e03fffff

Prefetchable memory behind bridge: 00000000e0700000-00000000e0700000

Capabilities: Express Root Port (Slot+) IRQ 0

Capabilities: Message Signalled Interrupts: 64bit- Queue=0/0 Enable-

Capabilities: #0d

Capabilities: Power Management version 2

00:1c.1 PCI bridge: Intel Corporation Unknown device 2942 (rev 02) (prog-if 00 )

Flags: bus master, fast devsel, latency 0

Bus: primary=00, secondary=02, subordinate=02, sec-latency=0

I/O behind bridge: 00002000-00002fff

Memory behind bridge: e0200000-e02fffff

Prefetchable memory behind bridge: 00000000e0800000-00000000e0800000

Capabilities: Express Root Port (Slot+) IRQ 0

Capabilities: Message Signalled Interrupts: 64bit- Queue=0/0 Enable-

Capabilities: #0d

Capabilities: Power Management version 2

00:1c.2 PCI bridge: Intel Corporation Unknown device 2944 (rev 02) (prog-if 00 )

Flags: bus master, fast devsel, latency 0

Bus: primary=00, secondary=03, subordinate=03, sec-latency=0

I/O behind bridge: 00001000-00001fff

Memory behind bridge: e0100000-e01fffff

Capabilities: Express Root Port (Slot+) IRQ 0

Capabilities: Message Signalled Interrupts: 64bit- Queue=0/0 Enable-

Capabilities: #0d

Capabilities: Power Management version 2

00:1c.3 PCI bridge: Intel Corporation Unknown device 2946 (rev 02) (prog-if 00 )

Flags: bus master, fast devsel, latency 0

Bus: primary=00, secondary=04, subordinate=04, sec-latency=0

Capabilities: Express Root Port (Slot+) IRQ 0

Capabilities: Message Signalled Interrupts: 64bit- Queue=0/0 Enable-

Capabilities: #0d

Capabilities: Power Management version 2

00:1c.4 PCI bridge: Intel Corporation Unknown device 2948 (rev 02) (prog-if 00 )

Flags: bus master, fast devsel, latency 0

Bus: primary=00, secondary=05, subordinate=05, sec-latency=0

Capabilities: Express Root Port (Slot+) IRQ 0

Capabilities: Message Signalled Interrupts: 64bit- Queue=0/0 Enable-

Capabilities: #0d

Capabilities: Power Management version 2

00:1d.0 USB Controller: Intel Corporation Unknown device 2934 (rev 02) (prog-if 00 )

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, medium devsel, latency 0, IRQ 11

I/O ports at 4080

Capabilities: Vendor Specific Information

00:1d.1 USB Controller: Intel Corporation Unknown device 2935 (rev 02) (prog-if 00 )

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, medium devsel, latency 0, IRQ 11

I/O ports at 4060

Capabilities: Vendor Specific Information

00:1d.2 USB Controller: Intel Corporation Unknown device 2936 (rev 02) (prog-if 00 )

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, medium devsel, latency 0, IRQ 10

I/O ports at 4040

Capabilities: Vendor Specific Information

00:1d.7 USB Controller: Intel Corporation Unknown device 293a (rev 02) (prog-if 20 )

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, medium devsel, latency 0, IRQ 11

Memory at e0625800 (32-bit, non-prefetchable)

Capabilities: Power Management version 2

Capabilities: Debug port

Capabilities: Vendor Specific Information

00:1e.0 PCI bridge: Intel Corporation 82801BA/CA/DB/EB PCI Bridge (rev 92) (prog-if 01 )

Flags: bus master, fast devsel, latency 0

Bus: primary=00, secondary=06, subordinate=06, sec-latency=32

Memory behind bridge: e0000000-e00fffff

Capabilities: #0d

00:1f.0 ISA bridge: Intel Corporation Unknown device 2914 (rev 02)

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, medium devsel, latency 0

Capabilities: Vendor Specific Information

00:1f.2 Class 0106: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA AHCI Controller (rev 02) (prog-if 01)

Subsystem: Intel Corporation Unknown device 4f4a

Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 11

I/O ports at 4428

I/O ports at 443c

I/O ports at 4420

I/O ports at 4438

I/O ports at 4020

Memory at e0625000 (32-bit, non-prefetchable)

Capabilities: Message Signalled Interrupts: 64bit- Queue=0/4 Enable-

Capabilities: Power Management version 3

Capabilities: #12

Capabilities: Vendor Specific Information

00:1f.3 SMBus: Intel Corporation Unknown device 2930 (rev 02)

Subsystem: Intel Corporation Unknown device 4f4a

Flags: medium devsel, IRQ 10

Memory at e0626000 (64-bit, non-prefetchable)

I/O ports at 4000

01:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (rev 06)

Subsystem: Intel Corporation Unknown device 1083

Flags: bus master, fast devsel, latency 0, IRQ 11

Memory at e0320000 (32-bit, non-prefetchable)

Memory at e0300000 (32-bit, non-prefetchable)

I/O ports at 3000

Expansion ROM at e0700000

Capabilities: Power Management version 2

Capabilities: Message Signalled Interrupts: 64bit+ Queue=0/0 Enable+

Capabilities: Express Endpoint IRQ 0

02:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (rev 06)

Subsystem: Intel Corporation Unknown device 1083

Flags: bus master, fast devsel, latency 0, IRQ 9

Memory at e0220000 (32-bit, non-prefetchable)

Memory at e0200000 (32-bit, non-prefetchable)

I/O ports at 2000

Expansion ROM at e0800000

Capabilities: Power Management version 2

Capabilities: Message Signalled Interrupts: 64bit+ Queue=0/0 Enable+

Capabilities: Express Endpoint IRQ 0

03:00.0 IDE interface: Galileo Technology Ltd. Unknown device 6101 (rev b2) (prog-if 8f )

Subsystem: Galileo Technology Ltd. Unknown device 6101

Flags: bus master, fast devsel, latency 0, IRQ 10

I/O ports at 1018

I/O ports at 1024

I/O ports at 1010

I/O ports at 1020

I/O ports at 1000

Memory at e0100000 (32-bit, non-prefetchable)

Capabilities: Power Management version 2

Capabilities: Message Signalled Interrupts: 64bit- Queue=0/0 Enable-

Capabilities: Express Legacy Endpoint IRQ 0

06:03.0 FireWire (IEEE 1394): Lucent Microelectronics FW323 (rev 70) (prog-if 10 )

Subsystem: Lucent Microelectronics FW323

Flags: bus master, fast Back2Back, medium devsel, latency 32, IRQ 11

Memory at e0000000 (32-bit, non-prefetchable)

Capabilities: Power Management version 2

#

[AndreTheGiant]

The NIC are correcly recognized (ifconfig confirm that you have 3 NIC).

Under ESX / Configuration / Network adapters can you see all of them?

Andre

**if you found this or any other answer useful please consider allocating points for helpful or correct answers

[MarcBouchard]

Yes, I see all 3 NICs. Everything looks just fine, I can even go in and set speed etc... The other 2 NICs just wont show a link... I reinstalled ESX twice to check if I had the same issue during installation and I do.

[AndreTheGiant]

The other 2 NICs just wont show a link...

I'm not sure to understand.

Have you add the other 2 NICs to your vSwitch?

Andre

**if you found this or any other answer useful please consider allocating points for helpful or correct answers

[MarcBouchard]

The other 2 NICs just wont show a link...I'm not sure to understand.

Have you add the other 2 NICs to your vSwitch?

Andre

**if you found this or any other answer useful please consider allocating points for helpful or correct answers

The problem is not even that far. ESX 4 does not see that the NICs are connected i.e. as if the cable was physically unplugged. Only one NIC sees when I plug in the cable, the vmnic2. That's what I mean by won't show a link.

Recap:

ESX sees all 3 adapters. vSphere client allows me to do whatever I want with them, add them to vswitch, change parameters etc.. But no matter what I do, 2 of them don't see the network (as if they were disconnected).

I have replaced cables, physical switch ports etc with no result.

Thanks for your assistance!

Marc

The other 2 NICs just wont show a link...I'm not sure to understand.

Have you add the other 2 NICs to your vSwitch?

Andre

**if you found this or any other answer useful please consider allocating points for helpful or correct answers

[AndreTheGiant]

If you can see the NIC in configuration / network adapter than it's all fine.

You have only to go in configuration / networking, click properties near the vSwitch, go in network adapters tab, and add the other NIC.

Or create new vSwitches.

Andre

**if you found this or any other answer useful please consider allocating points for helpful or correct answers

[MarcBouchard]

If you can see the NIC in configuration / network adapter than it's all fine.

You have only to go in configuration / networking, click properties near the vSwitch, go in network adapters tab, and add the other NIC.

Or create new vSwitches.

Andre

**if you found this or any other answer useful please consider allocating points for helpful or correct answers

That's what I figured however the links are listed as DOWN, so no network connectivity. I can create vswitches add the nics to them but they dont see the outside world as the system thinks the link is down as if unplugged...

[Cruicer]

Are your ports turned on at the Physcial switch end?

[AndreTheGiant]

Have you tried to change NIC speed? Or change the cable or change the switch port?

Andre

**if you found this or any other answer useful please consider allocating points for helpful or correct answers

[AndreTheGiant]

What's the output of this command:

ethtool vmnic0

Andre

**if you found this or any other answer useful please consider allocating points for helpful or correct answers

[MarcBouchard]

Have you tried to change NIC speed? Or change the cable or change the switch port?

Andre

**if you found this or any other answer useful please consider allocating points for helpful or correct answers

Indeed I did. I mentionned that several times already... This makes no sense...