ESXi access the NFS via the wrong VMkernel

[Itzikr]

Hi,

our new farm consist of ESX4i hosts.

vswitch 0 has two nics and two port groups

one vmkernel port group for the ESXi management network

one vmkernel port group for the ESXi NFS

since both of their port groups are now vmkernel based (as oppose to the past where the management network was on the service console port group, and they both reside on the same subnet, the ESXi now try to access the NFS server via the management interface IP which is blocked by the firewall and NOT via the vmkernel for the NFS.

how can i force the right vmkernel to be used..

thanks

Itzik

[marcelo_soares]

Both interfaces are on the same network segment, ESX uses the first created on the same net. You will need to have different net segments in order to get the traffic going with a different IP (like keeping 172.20.215.168 for the management and creating a 192.168.1.100 for the vmkernel, and placing the NFS server on this same new network). I also think that, maybe, there is some nfs config file where you can specify the IP which will be used from the client, but now I am not so sure about that. Will take a look at my lab here, and let you know if I find something.

Marcelo Soares

VMWare Certified Professional 310/410

Technical Support Engineer

Globant Argentina

Consider awarding points for "helpful" and/or "correct" answers.

[Itzikr]

thanks

please let me know if you find this file..

[vmetc_rich]

I'm experiencing the same wrong vmkernel problem.

I have the ESXi 4 Management vmkernel portgroup using 192.168.1.0/24 on vSwitch0, and I want a separate NFS vmkernel portgroup on vSwitch3 using 10.10.10.0/24. I can't ping either from the NFS filer (has an interface on both IP subnets) to the ESXi server or visa versa on the 10.10.10..0 IP network. I can ping each way on the 192.168.1.0/24 IP network.

Obviously, I can't mount my NFS share on the NIC assigned to vSwitch 3 (10.10.10.0). I have no problem connecting the NFS share using the mgt vmkernel interface (192.168.1.0/24)

Surely ESXi lets you separate Management from NFS traffic?

[marcelo_soares]

It let and is the normal behavior. Maybe you are not using vmkping to reach your NFS server.

I think the original problem on the post still are both interfaces on the same network segment. You will need to change it in order to get it working.

Marcelo Soares

VMWare Certified Professional 310/410

Technical Support Engineer

Globant Argentina

Consider awarding points for "helpful" and/or "correct" answers.

[Exwork]

From what I've read before, ESXi only supports one default gateway for management and vmkernel.

The full ESX edition allows the management interface and vmkernel to have different default gateways.

[marcelo_soares]

That's why management and vmkernel ports are the same on ESXi.

Marcelo Soares

VMWare Certified Professional 310/410

Technical Support Engineer

Globant Argentina

Consider awarding points for "helpful" and/or "correct" answers.

[vChr1s]

Sorry to bring up an old thread but I am having the same problem. I have always built ESX environments. This is my first ESXi I'm testing. Per best practices, with NFS, I've always seperated storage traffic from everything, one of which is the management traffic/Service Console.

I know there is no service console with ESXi but I am assuming there is still management traffic?

So then if I were to put storage traffic on a physically seperate network, I would need to create a new vmKernel port to use to connect to ESXi?

Any help is appreciated.

[jpratt_at_norwi]

Ok - SO... Seeing how 4.1 is the "final release" with a SC,, and then it's ESXi or nothing, umm.... is VMware going to fix this? I cannot re-architect my network to accomodate for this stupidity - no matter what I've tried i cannot get two default gateways on an ESXi box, one for mgmt and the other for vmotion - THIS IS BROKEN. THANK YOU VMWARE...

Also, SUPER-glad i got dual-certified last year, and now i have to learn ESXi, which completely bites it management-wise).. what a completely wasted year that was... woo-hooo--.. guess it's time to start looking at the competition (did i really just say that?) ... sigh.. /frustration , rant end

[jpratt_at_norwi]

Update -

For my issue, I was able to resolve it by simply not changing the default gw for the vmotion port, and it just worked anyhow.

this makes me wonder - if the default vmkernel/vmotion gw is being ignored , even on ESX (?) how does ISCSI or NFS/NAS work properly , or are completely flat networks (layer-2) the best/only answer here?

I know of another person/site who is having serious ISCSI failover issues with ESXi vs ESX with the whole gateway thing, which is putting a serious dent in his redundancy setup, so methinks theremay still be issues here, but i'm not sure - I don't have enough isci or nfs experience....

thanks - Jamie

[RParker]

I know there is no service console with ESXi but I am assuming there is still management traffic?

Yes that is the principle difference between ESX and ESXi, no service console. If you need the console, you should switch back to ESX.

That's the ONLY difference between the 2 servers. Otherwise they are identical, including management interface, you can setup only 1 gateway.

[RParker]

Also, SUPER-glad i got dual-certified last year, and now i have to learn ESXi, which completely bites it management-wise).

I know you are being SUPER sarcastic, but there really is NO difference between ESX and ESXi, so you are being SUPER sensitive, and SUPER obtuse, if you didn't read about these products.

No need to certify on both, they are the same product. Just one includes the console, the other does not. So it's not really different management wise, just SOME things are done different.

[RParker]

this makes me wonder - if the default vmkernel/vmotion gw is being ignored , even on ESX (?) how does ISCSI or NFS/NAS work properly , or are completely flat networks (layer-2) the best/only answer here?

You can actually leave the Gateway blank, it will figure it out. The gateway is for some other features, but MOST networks use the x.x.x.1 as the gateway, so unless you use some real archaic proxy server, you probably don't even need the gateway IP.

[vChr1s]

What I found , particularly in regards to my previous comment, was that storage traffic needs to use a port group with the "Management Traffic" check box checked. For example, if I create a new vSwitch with ip of 10.10.1.1 and ONLY allow that IP on my storage side, it won't work until I check the checkbox to tell the port group that it is for "Management Traffic". Until then, I cannot mount the NFS volume.

[aenagy]

RParker and jpratt_at_nor…:

I ran into these issues while transitioning from ESX 3.0.2 to ESXi 4.0 U1. I kept reconfiguring the default gateway on my Management and VMotion interfaces of my ESXi 4.0 hosts and noticed that the change would show up on the other (VMotion, Management) interface. The default gateway value would keep flipping back and forth with each configuration change. What I concluded and confirmed with VMware technical support is that ESX has two different IP stacks which means two different default gateways, while ESXi only has one IP stack which means only one default gateway.

Furthermore, there is no way (that I know of) to add static routes to ESXi 4.0. This as big implications for if you want to separate your Management vs. VMotion vs. IP Storage traffic. Assuming that you only configure the Management interface with the default gateway, then all other traffice (VMotion, IP Storage, etc) will use this same default gateway if it needs to be routed to a different subnet. This definately sucks. The only options I see are to open a support case to see if there is a way to add static routes to ESXi 4.x, or make a suggestion (http://www.vmware.com/support/policies/feature.html).

With respect to managability of ESXi vs. ESX, don't get me started...

[fletch00]

I just found this "ESX vs ESXi" difference as well - for me it showed up as vMotions to the ESXi host failing at 10%:

http://www.vmadmin.info/2010/09/esxi-vmotion-fails-10.html

Has anyone opened a support case to get vmware's take on this?

VCP4

[jshelly]

Any further updates on this?

I would think there would be an easier way to separate IP Storage from Management traffic.

We are in the process of switching from ESX 4.1 to ESXi 4.1 and our NFS server is currently on a routed network so this creates a bit of a sticky situation.

Are there any issues with simply taking the two physical nics that were used for the extra vm kernel port and service console and adding them to the management network to support our NFS datastores as illustrated above?

[Dave_Mishchenko]

Can you simply put 2 NIC ports in a new vSwitch on the same subnet as your NFS host? If you can only support a single management port, but need to route traffic away from the default gateway just add a static route.




Dave

VMware Communities User Moderator

Now available - vSphere Quick Start Guide

Do you have a system or PCI card working with VMDirectPath? Submit your specs to the Unofficial VMDirectPath HCL.

[jshelly]

The subnet where my NFS server resides is the same subnet as the Management subnet.

I created a new vSwitch with the 2 NIC ports and the traffic still defaults to vmk0

How do you add a static route in ESXi?

[fletch00]

Opened case 1582181211 on this - will summarize the wisdom from the mothership 😉

VCP4