Request for ESX network config advice

[MattJoyce]

Hi,

We have ESX30.1 on a Dell PE2950, both onboard nics are connected to a 100mb building switch (due to be a upgraded soon).

I'm not particularly experienced with ESX under the hood.

Gather some info...

From ifconfig....

vmnic0 Link encap:Ethernet HWaddr 00:1A:A0:0B:53:4C

UP BROADCAST MULTICAST MTU:1500 Metric:1

RX packets:205224730 errors:0 dropped:0 overruns:0 frame:0

TX packets:212108266 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:3380017347 (3223.4 Mb) TX bytes:73945130 (70.5 Mb)

Interrupt:129 Memory:f8000000-f8011100

vmnic1 Link encap:Ethernet HWaddr 00:1A:A0:0B:53:4E

UP BROADCAST MULTICAST MTU:1500 Metric:1

RX packets:3122475 errors:0 dropped:0 overruns:0 frame:0

TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:231822168 (221.0 Mb) TX bytes:0 (0.0 b)

Interrupt:129 Memory:f4000000-f4011100

\# /usr/sbin/esxcfg-nics -l

Name PCI Driver Link Speed Duplex Description

vmnic0 05:00.00 bnx2 Up 100Mbps Full Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T

vmnic1 09:00.00 bnx2 Up 100Mbps Full Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T

\# /usr/sbin/esxcfg-vswitch -l

Switch Name Num Ports Used Ports Configured Ports Uplinks

vSwitch0 32 14 32 vmnic0

PortGroup Name Internal ID VLAN ID Used Ports Uplinks

VM Network portgroup1 0 11 vmnic0

Service Console portgroup0 0 1 vmnic0

Both the esxcfg command took over a minute to respond is this typical ?

To me this looks like only one nic vmnic0 is being used, does that look right ?

Perhaps these are configure as fail-over rather than load-balanced+fail-over.

How do I check/change this ?

Also, I have a second pair of dual nics to install, and I'm not sure of the process.

I guess broadly speaking it would go something like...

Install hardware

Configure OS to recognise hardware

Configure ESX to use new nics

Assign ESX vnics to vswitch

These are Intel nics, and I'm hoping this will be straight forward.

Should I assign this pair to a second vswitch or just add them all to the existing vswitch ?

Any advice welcomed.

Thanks

Matt

[RParker]

Our company is 100% committed to Dell. Mainly because management knows we needed a good company, and Dell has a very good reputation (despite those people that claim to the contrary).

I love Dell and Microsoft. Did I say they were perfect, NO.

That being said, we have 2950 machines for our ESX cluster, and I can tell you.. Broadcom Sucks!

"We have ESX30.1 on a Dell PE2950, both onboard nics are connected to a 100mb building switch (due to be a upgraded soon"

Use Intel PCIe add-on NICs. You will be much better off.

"Both the esxcfg command took over a minute to respond is this typical ?"

Nope, but then we have Intel, and everytime there is a problem with hardware, It's almost certainly due to Broadcom drivers/or onboard CPU problems as a result of Broadcom NIC (WE have 1750, 1850, 1950 as well).

"To me this looks like only one nic vmnic0 is being used, does that look right ?

Perhaps these are configure as fail-over rather than load-balanced+fail-over.

How do I check/change this ? "

In networking, you can see the Nics configured for the switch. And you can visibly see which NICs are assigned (in the configuration tab from the VI)

"Also, I have a second pair of dual nics to install, and I'm not sure of the process.

I guess broadly speaking it would go something like... "

DO NOT \*WALK, \RUN* to the location of the Intel NICs. Install them DISABLE Internal NICS, turn \*ON* VT Extensions.

Upgrade the Firmware for BIOS, Backplane, controller while you are at it.

Frankly, I haven't tried plug and play in ESX, but I don't think there will be a problem adding hardware. If you need to, upgrade to ESX 3.02 at the same time (The reinstall may help with the plug and play). You can do this from the CD.

After that, you should be MUCH better situation.

[esiebert7625]

100%?

Have you tried anything else? Unless you try the competition it's hard to compare against other companies. Imagine Rocky saying "I am the Greatest" and never having actually fought anyone. I've heard HP is halfway decent also.

Sorry, had to give you a hard time. Never really used Dell but have never really heard much bad about them. We just replaced all the IBM systems in our stores with Dells. We're still a HP server shop though, they've always been good for us and I've used them for 15 years. I guess you can't go wrong with either HP or Dell servers.

[BUGCHK]

Both the esxcfg command took over a minute to respond is this typical ?

I don't remember which thread it was, but there was a discussion of a similar problem and I think it was a bad network configuration. I've seen something similar when I accidently created a domain name inconsistency between /etc/hosts[/i] and /etc/resolv.conf[/i]

[MattJoyce]

I did discover the primary dns server was pointing to a decommissioned server,

the secondary was fine.

I've updated this and bounced the network service.

Before I made that change (and still) using VI if I click through to localhost\configuration\networking

I only see vmnic0, if I click properties I eventually get a error message "Error - Timeout waiting for server response."

the next dialog show no network cards.

Also, click through to localchost\configuration\networkadaptors, I get the same error and no adaptors are show in the dialog when it appears.

Clearly something is wrong.

I'm not sure how to proceed.

[MattJoyce]

I think I fixed it.

While I was getting info from esxcfg-nics -l, I got a wierd error about esx.conf being locked.

I investigatred that and ended up running :

service mgmt-vmware restart

Then i was able to see the network configuration ain VI and the performance data was working again.

One of my nics was not being used by the vswitch, so I added it back and it seems fine.

[MattJoyce]

Everything worked fine after reassigning nics.