VMware Cloud Community
Symbion_Tech
Contributor
Contributor

Issue with Intel Pro 1000 MT Quad Port NIC

Anyone got any ideas on this? I replaced my dual port Pro 1000 MT with quad port Pro 1000 MT in my whitebox running ESX 4 Update 1 and only ports 2 & 3 show up. Ports 0 & 1 are not available and there is no connectivity light on either port. When i boot the esx host from cd to mini-xp and start the network all four ports are available and connected so the card is working. The only similar issue i could find was an article where on certain Dell and SuperMicro servers (i'm running a whitebox on an ASUS P5WDG2 WS Pro) the same IRQ may be assigned to more than one port on the NIC rendering one or more ports unavailable. Is there anyway to check IRQ assignments in any of the esx logs?

0 Kudos
3 Replies
conyards
Expert
Expert

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=100371...

Excellent article above describing how to fix this in 3.5, still valid for 4

"Determining if IRQ Sharing Issues Affect Your System's Performance

+The tell-tale sign of IRQ sharing

between VMkernel and the service console is a high number of interrupts

being serviced by PCPU0 (CPU #0 on the physical host) while the other

CPUs are relatively lightly loaded. The high interrupt rates might

sometimes render the service console unusable and cause a high

variation in ESX Server performance.+

+The most common service console device

that causes IRQ sharing is the USB controller. On the VMkernel side,

the network and storage controllers are susceptible to IRQ sharing. IRQ

sharing is more common in dual- and quad-port NICs and storage HBAs

than in single port controllers. However, IRQ sharing is not restricted

only to these devices. The effect is more visible under I/O intensive

loads (with high interrupt rates). The problem might manifest itself as

variation in performance or as an absolute drop in ESX Server

performance.+

+To determine if your setup suffers

from IRQ sharing, list the IRQ assignment in VMkernel by typing the

following at the command-line of the service console:+

> cat /proc/vmware/interrupts

This lists the interrupt usage. The output looks similar to the following example:

+Vector PCPU 0 PCPU 1 PCPU 2 PCPU 3

     0x21: 0 0 0 0 VMK ACPI Interrupt

     0x29: 1 0 0 0 <COS irq 1 (ISA edge)>, VMK keyboard

     0x31: 4 0 0 0 <COS irq 3 (ISA edge)>

     0x39: 4 0 0 0 <COS irq 4 (ISA edge)>

     0x41: 0 0 0 0 <COS irq 6 (ISA edge)>

     0x49: 0 0 0 0 <COS irq 7 (ISA edge)>

     0x51: 0 0 0 0 <COS irq 8 (ISA edge)>

     0x59: 0 0 0 0 <COS irq 12 (ISA edge)>

     0x61: 0 0 0 0 <COS irq 13 (ISA edge)>

     0x69: 43762 0 0 0 COS irq 14 (ISA edge)

     0x71: 0 0 0 0 <COS irq 15 (ISA edge)>

     0x79: 7917 542 3583 8292 <COS irq 16 (PCI level)>, VMK aic79xx

     0x81: 1544 0 0 0 COS irq 17 (PCI level), VMK aic79xx

     0x89: 1212177 0 0 0 COS irq 19 (PCI level), VMK vmnic1

     0x91: 90997 0 0 0 COS irq 18 (PCI level), VMK vmnic0

     0x99: 152 0 0 0 <COS irq 20 (PCI level)>, VMK qla2300

      
    0xdf: 8904447 10869326 11006262 10844861 VMK timer

       0xe1: 74 5582 12007 16005 VMK monitor

       0xe9: 60854 443843 477389 509724 VMK resched

       0xec: 0 0 0 0 VMK ucodeUpdate

       0xf1: 3 40 68 100 VMK tlb

       0xf9: 243265 0 0 0 VMK noop

       0xfc: 0 0 0 0 VMK thermal

     0xfd: 0 0 0 0 VMK lint1

     0xfe: 0 0 0 0 VMK error

     0xff: 0 0 0 0 VMK spurious 
+

+This output lists the interrupt vectors with the number of interrupts fielded by each physical CPU ( PCPU).

The last column lists the device or devices associated with the

particular IRQs. Along with the device name, the owner of the device is

also listed. COS refers to service console ownership, and VMK refers to VMkernel ownership. (In older versions of ESX Server, the service console was named the console operating system, or COS.)+

+For

the sake of analysis, you can ignore devices owned by the service

console that have their names enclosed in angle brackets (<>),

because the service console does not load a driver for them. All other

interrupt vectors that have a device with COS and VMK next to them

indicate the IRQs are shared for that device. Again, devices with low

interrupt rates (for example, 0x81 and 0x99) can be ignored because the

performance impact due to them is minimal.+

+In the above example, IRQ 0x81, 0x89,

0x91, and 0x99 are shared between service-console and VMK devices. You

don't need to analyze the interrupt counts for 0x81 and 0x99, because

the counts are reasonably low. Also note that all interrupts for these

four IRQs are being fielded by PCPU 0, while all other PCPUs show zero

interrupt count. In contrast, interrupt vectors 0x89 and 0x91 (for

vmnic1 and vmnic0) show high interrupt counts. The shared IRQs in the

service console for vmnic1 and vmnic0 are IRQ 19 and IRQ 18

respectively.+

+In this example, you need to remove

the device conflicting with vmnic0 and vmnic1 from the service console.

To find the offending device in the service console, list the interrupt

vectors:+

> cat /proc/interrupts

This lists the IRQ lines assigned to the devices in the service console.

+CPU0

      0: 1744046 vmnix-edge timer

      1: 3 vmnix-edge keyboard

      2: 163071 vmnix-edge VMnix interrupt

      14: 44022 vmnix-edge ide0

      17: 1499 vmnix-level usb-uhci, ehci-hcd

      18: 91236 vmnix-level usb-uhci

      19: 1303228 vmnix-level usb-uhci

      NMI: 0

      LOC: 0

      ERR: 0

      MIS: 0 
+

You can see that both IRQ 18 and IRQ 19 are in use by usb-uhci, a USB Universal Host Controller device."

https://virtual-simon.co.uk/
Symbion_Tech
Contributor
Contributor

Many thanks for taking the time. I'll check this out this evening.

0 Kudos
Symbion_Tech
Contributor
Contributor

Having checked as you suggested and other tests i've done I think it's a limitation of the motherboard and irq's with ESX. When i had 2k8 r2 loaded on the box a while back i could see all four ports but that'll be due to Windows handling the IRQ assignments. ESX with this mobo only see's ports 2 &3 (0 & 1) missing.

0 Kudos