Re: esx.conf format

ScottHolman · ‎04-24-2007

I'm trying to understand the esx.conf file and cannot find a document that explains how this file is structured.

The reason I'm trying to understand this is the ESX appears to have picked up my 6 NIC (2 integrated (Broadcom), 2 dual NIC cards (Intel)) incorrectly.

My first question is where can I find a doc that explains the structure of this file?

My second is does /device/nnn:n.n bear any relation to the lspci output?

scott

masaki · ‎04-24-2007

As written in the name esx.conf is the ESX configuration.

the /device section tells you information on a particular device (id inside ESX, name, vendor, type, owner, etc.)

the /firewall tells you which ports are opened

the /net about the network parameters

/resourcegroups (info about resource pooling)

/vmkmodule info on modules

etc.

lspci gives much less info than esx.conf. It gives the name and the device id of each pci device.

ScottHolman · ‎04-24-2007

masaki, thanks for your reply.

I think you've misunderstood what I need to know or maybe I didn't explain myself well enough. I already understand what you've explained. I need a little more detail.

For instance, my lspci shows one of my dual nic card as follows:

0d:00.0 Intel Ethernet controller

0d:00.1 Intel Ethernet controller

Is it possible to match up the pci id with the devices shown in my esx.conf?

masaki · ‎04-24-2007

Yes,

it's the same.

Look into esx.conf cat esx.conf|more

0d:00:0 should be /device00d:00:0/.....

Post here to check

ScottHolman · ‎04-24-2007

If it was that obvious I wouldn't be posting here 🐵

Section from lspci output:

0d:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06)

0d:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06)

16:02.0 Ethernet controller: Broadcom Corporation NetXtreme II 5706 Gigabit Ethernet (rev 02)

16:03.0 Ethernet controller: Broadcom Corporation NetXtreme II 5706 Gigabit Ethernet (rev 02)

21:01.0 Ethernet controller: Intel Corporation 8254NXX Gigabit Ethernet Controller (rev 03)

21:01.1 Ethernet controller: Intel Corporation 8254NXX Gigabit Ethernet Controller (rev 03)

Relevent sections from esx.conf:

/device/000:01.0/class = "020000"

/device/000:01.0/devID = "1229"

/device/000:01.0/irq = "17"

/device/000:01.0/name = "Intel Corporation 82557/8/9 \[Ethernet Pro 100] (rev 0c)"

/device/000:01.0/options = ""

/device/000:01.0/owner = "vmkernel"

/device/000:01.0/subsysDevID = "01f1"

/device/000:01.0/subsysVendor = "1014"

/device/000:01.0/vendor = "8086"

/device/000:01.0/vmkname = "vmnic5"

/device/002:01.0/class = "020000"

/device/002:01.0/devID = "1229"

/device/002:01.0/irq = "17"

/device/002:01.0/name = "Intel Corporation 82557/8/9 \[Ethernet Pro 100] (rev 0c)"

/device/002:01.0/options = ""

/device/002:01.0/owner = "vmkernel"

/device/002:01.0/subsysDevID = "01f1"

/device/002:01.0/subsysVendor = "1014"

/device/002:01.0/vendor = "8086"

/device/002:01.0/vmkname = "vmnic1"

/device/002:02.0/class = "020000"

/device/002:02.0/devID = "1229"

/device/002:02.0/irq = "18"

/device/002:02.0/name = "Intel Corporation 82557/8/9 \[Ethernet Pro 100] (rev 0c)"

/device/002:02.0/options = ""

/device/002:02.0/owner = "vmkernel"

/device/002:02.0/subsysDevID = "01f1"

/device/002:02.0/subsysVendor = "1014"

/device/002:02.0/vendor = "8086"

/device/002:02.0/vmkname = "vmnic2"

/device/002:08.0/class = "020000"

/device/002:08.0/devID = "16a7"

/device/002:08.0/irq = "23"

/device/002:08.0/name = "Broadcom Corporation NetXtreme BCM5703 Gigabit Ethernet (rev 02)"

/device/002:08.0/options = ""

/device/002:08.0/owner = "vmkernel"

/device/002:08.0/subsysDevID = "026f"

/device/002:08.0/subsysVendor = "1014"

/device/002:08.0/vendor = "14e4"

/device/002:08.0/vmkname = "vmnic0"

/device/007:05.0/class = "020000"

/device/007:05.0/devID = "1229"

/device/007:05.0/irq = "20"

/device/007:05.0/name = "Intel Corporation 82557/8/9 \[Ethernet Pro 100] (rev 0c)"

/device/007:05.0/options = ""

/device/007:05.0/owner = "vmkernel"

/device/007:05.0/subsysDevID = "01f1"

/device/007:05.0/subsysVendor = "1014"

/device/007:05.0/vendor = "8086"

/device/007:05.0/vmkname = "vmnic3"

/device/009:06.0/class = "020000"

/device/009:06.0/devID = "1008"

/device/009:06.0/irq = "25"

/device/009:06.0/name = "Intel Corporation 82544EI Gigabit Ethernet Controller (Copper) (rev 02)"

/device/009:06.0/options = ""

/device/009:06.0/owner = "vmkernel"

/device/009:06.0/subsysDevID = "1107"

/device/009:06.0/subsysVendor = "8086"

/device/009:06.0/vendor = "8086"

/device/009:06.0/vmkname = "vmnic4"

I don't see how they are related!

It's also weird that esx.conf shows 4 Intel Pro100, 1 Intel Gigbit, & 1 Broadcom; when in fact it has 4 Intel Pro1000 & 2 Broadcom gigabit, but lspci picks them all up correctly.

Now you can understand my confusion! By the way this server is working. I just want to be able to rename the vmnics to the physical nics.

Scott

masaki · ‎04-24-2007

It's strange I have a perfect match (I posted only a part of esx.conf and lspci output here):

/device/001:00.0/class = "0c0300"

/device/001:00.0/devID = "7464"

/device/001:00.0/irq = "17"

/device/001:00.0/name = "Advanced Micro Devices \[AMD] AMD-8111 USB (rev 0b) (pro

g-if 10 \[OHCI])"

/device/001:00.0/options = ""

/device/001:00.0/owner = "serviceConsole"

/device/001:00.0/subsysDevID = "0020"

/device/001:00.0/subsysVendor = "17c2"

/device/001:00.0/vendor = "1022"

/device/001:00.1/class = "0c0300"

/device/001:00.1/devID = "7464"

/device/001:00.1/irq = "17"

/device/001:00.1/name = "Advanced Micro Devices \[AMD] AMD-8111 USB (rev 0b) (pro

g-if 10 \[OHCI])"

/device/001:00.1/options = ""

/device/001:00.1/owner = "serviceConsole"

/device/001:00.1/subsysDevID = "0020"

/device/001:00.1/subsysVendor = "17c2"

/device/001:00.1/vendor = "1022"

/device/001:05.0/class = "030000"

/device/001:05.0/devID = "9880"

/device/001:05.0/irq = "16"

/device/001:05.0/name = "Trident Microsystems Blade 3D PCI/AGP (rev 3a) (prog-if

00 \[VGA])"

/device/001:05.0/options = ""

/device/001:05.0/owner = "serviceConsole"

/device/001:05.0/subsysDevID = "0020"

/device/001:05.0/subsysVendor = "17c2"

/device/001:05.0/vendor = "1023"

/device/002:02.0/class = "020000"

/device/002:02.0/devID = "1648"

/device/002:02.0/irq = "18"

/device/002:02.0/name = "Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet

(rev 03)"

/device/002:02.0/options = ""

/device/002:02.0/owner = "vmkernel"

/device/002:02.0/subsysDevID = "0010"

/device/002:02.0/subsysVendor = "17c2"

/device/002:02.0/vendor = "14e4"

\[root@esx1 vmware]# /sbin/lspci

00:06.0 PCI bridge: Advanced Micro Devices \[AMD] AMD-8111 PCI (rev 07)

00:07.0 ISA bridge: Advanced Micro Devices \[AMD] AMD-8111 LPC (rev 05)

00:07.1 IDE interface: Advanced Micro Devices \[AMD] AMD-8111 IDE (rev 03)

00:07.3 Bridge: Advanced Micro Devices \[AMD] AMD-8111 ACPI (rev 05)

00:0a.0 PCI bridge: Advanced Micro Devices \[AMD] AMD-8131 PCI-X Bridge (rev 12)

00:0a.1 PIC: Advanced Micro Devices \[AMD] AMD-8131 PCI-X APIC (rev 01)

00:0b.0 PCI bridge: Advanced Micro Devices \[AMD] AMD-8131 PCI-X Bridge (rev 12)

00:0b.1 PIC: Advanced Micro Devices \[AMD] AMD-8131 PCI-X APIC (rev 01)

00:18.0 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] HyperTransport Technology Configuration

00:18.1 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] Address MapK8 NorthBridge

00:18.2 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] DRAM Controller

00:18.3 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] Miscellaneous Control

00:19.0 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] HyperTransport Technology Configuration

00:19.1 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] Address MapK8 NorthBridge

00:19.2 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] DRAM Controller

00:19.3 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] Miscellaneous Control

00:1a.0 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] HyperTransport Technology Configuration

00:1a.1 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] Address MapK8 NorthBridge

00:1a.2 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] DRAM Controller

00:1a.3 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] Miscellaneous Control

00:1b.0 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] HyperTransport Technology Configuration

00:1b.1 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] Address MapK8 NorthBridge

00:1b.2 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] DRAM Controller

00:1b.3 Host bridge: Advanced Micro Devices \[AMD] K8 \[Athlon64/Opteron] Miscellaneous Control

01:00.0 USB Controller: Advanced Micro Devices \[AMD] AMD-8111 USB (rev 0b)

01:00.1 USB Controller: Advanced Micro Devices \[AMD] AMD-8111 USB (rev 0b)

01:05.0 VGA compatible controller: Trident Microsystems Blade 3D PCI/AGP (rev 3a)

02:02.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 03)

02:02.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 03)

02:04.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 08)

masaki · ‎04-24-2007

Did you upgrade this server from ESX 2.X?

Did you change any network card (there was an intel pro 100 before)?

Which Hardware are you using?

If not I suggest you to reinstall the system especially if you can vmotion vm on another HOST.

ScottHolman · ‎04-26-2007

Thanks masaki

I've freshly installed 3.0.1 on 3 brand new IBM xSeries 3655's. Nothing has changed since the installation. I don't think reinstallation is going to help me here.

Anyone else have an idea of what's going on?

skearney · ‎04-26-2007

That looks odd. I would expect them to be the same. Just keep in mind that lspci output is in HEX, but the esx.conf is in decimal.

masaki · ‎04-26-2007

In the compatibility guide:

Do you have dual core processors?

Do you have 2000 Series REV F processors?

Remember that : Not supported with IBM 40K7427 PCI-X 133MHz (FH/FL) Riser for IBM System 3655.

admin · ‎04-26-2007

Let me jump in here and make a few comments:

1) I saw you mention that you want to rename the devices in some way. I'd strongly recommend against doing that, the vmnicX format is enforced in code and renaming the devices in esx.conf may not update all the places that use that info and produce inconsistent results when you reboot.

2) The format of esx.conf is not documented as its not intended to be modified directly by users.

3) The lspci output and esx.conf /device lines are indeed oddly out of sync. masaki's esx.conf looks much more like what I'd expect. There are a few different tools that interact with this code, so it could be nothing and after a reboot it will look normal.

Just a few questions though.

a) Is the system running normally? I noted that esx.conf thinks you have e100s where clearly you've got e1000s. You can verify the right module is loaded with vmkload_mod -l

b) What does the output of esxcfg-nics -l look like?

c) What renaming did you want to do?

ScottHolman · ‎04-27-2007

I don't to change the vmnicX format I just want to rearrange the numbers assigned to each of the nics. For instance, I have 6 nic ports, two integrated and two dual nic adapters that I want to number left to right top to bottom vmnic1, vmnic2, etc so its's easier for my 2nd line guys to identitfy. At the moment they go 2, 1, 5, 6, 3, 4, which is confusing for the 2nd line guys, of which we have quite a turnover of. It all got a bit confusing with the esx.conf didn't match lspci. It also got a little confusing when I was trying to assign sc, vmk, and vm networks to specific physical nics.

vmkload_mod -l has verified that e1000 is loaded and e100 is not, and esxcfg-nics has confirmed that the e1000 driver is loaded against 4 of the nics and the bnx2 driver loaded for the broadcoms.

All

esx.conf format