Kenn213
Enthusiast
Enthusiast

Can't get management network working on fresh install

Jump to solution

I'm trying to connect to my fresh ESXi 6 install but I'm having troubles.

Background: Previously I had 5.1 running on this machine. I'm using the same network configuration that I was before. My 3 nics in the machine were Realtek 8111f and so initially I couldn't install  ESXi 6. I purchased a Broadcom BCM5751 which I believe is on the hcl but I have had a hard time navigating that at times. ESXi 6 installed with no troubles once this card was present.

Problem: I can't ping my ESXi fresh install. The network configuration hasn't changed from the previous server, just the physical nic is different. The ESXi server can ping it's own IP and localhost. The router can ping it's own IP (the gateway). Neither can ping each other. There is a static route set in the router (unchanged from previous), and the router says the link is active. Router is Mikrotik Routerboard RB2011UiAS-2HnD-IN.

Network Configuration:

IP: 10.0.0.6

Mask: 255.255.255.252 (30 bit)

Gateway: 10.0.0.5

IPv6: Disabled

Any advice is much appreciated. I can provide any more information as necessary.

1 Solution

Accepted Solutions
RichardBush
Hot Shot
Hot Shot

Just checked and the NIC is on the HAL

VMware Compatibility Guide: I/O Device Search

Can you try this fix ? http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=203570...

Doesn't state v6 but worth a shot at the moment.

Someone else appeared to be seeing the same issue on 5.5 with that card:

ESXi 5.5 host not able to connect in network

Rich

View solution in original post

0 Kudos
17 Replies
jlanders
VMware Employee
VMware Employee

> The network configuration hasn't changed from the previous server, just the physical nic is different.

The Broadcom BCM5751 is supported by ESXi 6.0.

What does the ARP (neighbor) cache show on the ESXi server and your Mikrotik Routerboard?

On the ESXi server, you can use the 'esxcli network ip neighbor list' command in the shell.

I'm not sure what the command would look like on the Mikrotik, but you should see the MAC of ESXi's BCM5751.

If you still see the old MAC, you should flush the ARP cache on the Mikrotik.


A 10.0.0.4/30 network only has two addresses available: 10.0.0.5 and 10.0.0.6, so this should be simple to debug.

0 Kudos
Kenn213
Enthusiast
Enthusiast

I get nothing when I run that command on the server. Just for fun I tried switching it to my LAN and giving it a static IP on that subnet and still wont list anything in the arp cache. It also won't pick up a DHCP address when I try that.

Is it possible the nic is just bad?

0 Kudos
jlanders
VMware Employee
VMware Employee

What does the command 'esxcli network nic list' show?

0 Kudos
Kenn213
Enthusiast
Enthusiast

I see my only nic listed when I run that command:

Name: vmnic0

PCI Device: 0000:03:00.0

Driver: tg3

Admin Status: Up

Link Status: Up

Speed: 100

Duplex: Full

Mac: 00:10:18:06:12:c5

MTU: 1500

Description: Broadcom Corporation NetXtreme BCM5751 Gigabit Ethernet

I also did try running throuh this KB article: VMware KB: vmk0 management network MAC address is not updated when NIC card is replaced or vmkernel ...

However it didn't seem to be my problem.

I also tried disabling the firewall to rule that out with: esxcli network firewall set -e false

0 Kudos
RichardBush
Hot Shot
Hot Shot

Hi,

Could this be an issue at the network side rather than the host ? Are you connecting to a trunk or access port ?

Can you show interface of the network port on that side ?

Just a couple of ideas.

Rich

0 Kudos
Kenn213
Enthusiast
Enthusiast

Thanks Richard,

It is just a router port so no trunk/access. Could be a network problem however network is unchanged from previous server. I did try hooking it up to my LAN with both DHCP and static IPs but it wouldn't pull an address, and still couldn't be seen. LAN is just a dumb switch.

Here is the port configuration:

address=10.0.0.5/30

comment="P2P to ESXi Management Interface"

interface="FastEthernet 1"

network=10.0.0.4

And the route:  (Active, Dynamic, Connected)

          DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE

ADC  10.0.0.4/30              10.0.0.5         FastEthernet 1       0

When I ping the server from my router (or my PC on my LAN) I see the orange light on the nic blink each time the ping is sent, but the ping isn't returned. I can't access by http/ssh either.

0 Kudos
RichardBush
Hot Shot
Hot Shot

Ok,

I see you say this is a fresh install, but you had issues with the old network cards ? did you install the new card and then reinstall ?

have you confirmed that the new nic is selected in management networks, from the console, configure management networks, network adapters ?

can you see packets in/out on the nics ? esxcli network nic stats get -n vmnic0

esxcli network ip interface list - should show you if the nic is assigned to a kernel and the vswitch

esxcli network ip interface ipv4 get - shows you the IP addresses assigned to the vmk

esxcli network vswitch standard list - vswitch information

Can you paste the output from the above ?

Rich

0 Kudos
Kenn213
Enthusiast
Enthusiast

Yes, this is a fresh clean install. I tried installing with the original nics in there at first, using an ISO with the Realtek drivers injected, but the installer wouldn't even complete telling me the driver package had been obsoleted. Then I purchased the Broadcom and installed using the original ISO (non-modified) and it went smoothly.

My only nic is selected in the network adapters section. There are a lot of errors listed under the first command, might those be related to my problem?

esxcli network nic stats get -n vmnic0: (highlights... the rest of the values are 0)

Packets received: 2844

Packets sent: 63

Bytes received: 348627

Bytes sent: 110563648

Multicast packets received: 1361

Broadcast packets received: 0

Total receive errors: 65144

Receive over errors: 1

Recieve CRC errors: 22939

Receive frame errors: 12604

Receive missed errors: 2362

esxcli network ip interface list:

name: vmk0

mac address: <my mac>

enabled: true

portset: vSwitch0

portgroup: Management Network

netstack instance: defaultTcpipStack

VDS Name: n/a

VDS UUID: n/a

VDS Port: n/a

VDS Connection: -1

Opaque network id: n/a

opaque network type: n/a

external ID: n/a

mtu: 1500

tso mss: 65535

port id: 33554439



esxcli network ip interface ipv4 get all looks correct (IP/Mask/Broadcast)


esxcli network vswitch standard list:

num ports: 2212

used ports: 4

configured ports: 128

mtu: 1500

CDP status: listen

beacon enabled: false

beacon interval: 1

beacon threshold: 3

beacon required by:

uplinks: vmnic0

portgroups: vm network, management network

0 Kudos
RichardBush
Hot Shot
Hot Shot

Hey,

i wouldn't expect that many errors on a link, have you tried replacing the patch ? also, might be worths setting the nic to speed 100 full duplex since thats all the switch supports.

Are you seeing errors on the switch side port?

R

0 Kudos
Kenn213
Enthusiast
Enthusiast

I'm not seeing any errors on the router side. In the DHCP log it shows the errors trying to offer a lease to the mac of the ESXi server from when I tried that on my LAN.  I feel like some communication is happening, just things aren't going through completely.

I may just try re-installing and or replacing the nic.

0 Kudos
RichardBush
Hot Shot
Hot Shot

That might be the quickest option.

If you ve replaced the cable, you could try connecting a laptop to the switch port with same IP config and confirm that works, should at least isolate which side the issue is.

After that i would assume its down to the NIC in the server, wether driver or faulty hardware.

Kenn213
Enthusiast
Enthusiast

So I decided to try to eliminate the nic itself. I installed a copy of Windows 8 and the nic worked seamlessly. CentOS 6 was the same story. Reinstalled ESXi 6 and I can't talk on the network. Didn't touch the hardware and used the exact same network settings in all 3 cases.

0 Kudos
RichardBush
Hot Shot
Hot Shot

Just checked and the NIC is on the HAL

VMware Compatibility Guide: I/O Device Search

Can you try this fix ? http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=203570...

Doesn't state v6 but worth a shot at the moment.

Someone else appeared to be seeing the same issue on 5.5 with that card:

ESXi 5.5 host not able to connect in network

Rich

0 Kudos
Kenn213
Enthusiast
Enthusiast

Thanks for the links. I can give the KB article a try in a minute here...

The other thread sounds almost identical to what I'm experiencing. I get an option for network boot before my BIOS loads however I've never done that before. I suppose I could give it a try because it is what appears to have fixed it in the other thread.

0 Kudos
Kenn213
Enthusiast
Enthusiast

Okay so the solution in the other thread wound up working for me. The problem was I disabled the network boot option in the BIOS and simply had the HD and DVD drives selected. Adding the network boot option as the first one and sitting through the failing to find a network operating system screen allowed my server to finally talk on the network.

Thanks for the help. Much appreciated.

DavidIsSain
Enthusiast
Enthusiast

I know this is old but something similar happened to me on a fresh 6.7u2 install.  Couldn't ping the IP address.  Server has 4 NICS so I connected them all to a gig switch, but still could not access the management IP.

I thought I'd add this to your thread as someone else might come across it as I did.

Server is a Lenovo ST550.  There are 2 LOM ports, a management port and a (Lenovo added) dual port NIC.

Here are the steps I took to isolate and resolve.

I enabled the shell

I entered the console (ALT+F1)

Logged in.

[root@localhost:~] esxcli network nic list

I see 5 NICs, 4 Ethernet ports and one vusb0.  The vusb0 is the embedded IBM XClarity Controller/IMM/BMC device.

[root@localhost:~] esxcli network ip interface ipv4 get

Shows me that vmk0 has the IP I set.

[root@localhost:~] esxi network switch standard list

Shows the vSwitch only has the vusb0 NIC

[root@localhost:~] esxcli network vswitch standard uplink add -u vmnic0 -v vSwitch0

I add a LOM NIC to the vswitch

[root@localhost:~] esxi network switch standard list

lists vSwitch0 with uplinks as vusb0,  vmnic0

I still cannot ping the IP.

[root@localhost:~] esxcli network vswitch standard uplink add -u vmnic2 -v vSwitch0

I add a second port to use the paddle board NIC.

[root@localhost:~] esxi network switch standard list

lists vSwitch0 with uplinks as vusb0, vmnic2, vmnic0

I still cannot ping the IP.

[root@localhost:~] esxcli network vswitch standard policy failover get -v vSwitch0

I see vusb0 is listed as the active adapter and vmnic0 & vmnic2 are listed as standby.

[root@localhost:~] esxcli network vswitch standard uplink remove -u vusb0 -v vSwitch0

I remove the vusb0 NIC

[root@localhost:~] esxcli network vswitch standard policy failover get -v vSwitch0

and now the Active Adapter is listed as vmnic0.  After a few seconds, I can ping the interface and access the web UI.

0 Kudos
EvertHoegee
Contributor
Contributor

Thank you for the commend!! I did install ESXI 6.5 with the driver package becouse i have a nic that is on the "Block list".

Installation went fine but I was unable to ping the host.

NIC boot was enabled in the bios but not the first boot option.

Afthere moving it to the first place I was able to ping the address.

Thank you for the post!!

#System Dell optiplex 3020

more iformation if people have the same realtec nic

Vmware installation, I did install 6.5:

.\ESXi-Customizer-PS-v2.6.0.ps1 -v65

https://www.v-front.de/2015/03/vsphere-6-is-ga-ultimate-guide-to.html

https://www.v-front.de/p/esxi-customizer-ps.html#download

0 Kudos