Strange ESXi 4.1 IP Addressing Issue

[gogogo5]

Hello

We have experienced a strange ESXi 4.1 IP addressing anomaly where the ESXi host's management IP address is changed to the vMotion address.

In our lab we have 2 ESXi hosts with 6 pNICs. All pNICS are configured on a single standard vSwitch. All pNICS are Active (no Standby or Unused) and configured for Route Based on IP hash. The 6 pNICS are uplinked to a Cisco 3120 stack where Etherchannel is configured. All works ok so far.

We have found that if the ESXi host has assigned vmk0 to vMotion and vmk1 to Management, upon rebooting the ESXi host the vMotion IP address is shown as the IP address (i.e. the management address) on the DCUI!!!! When using the DCUI to change the IP address back to its correct IP address, when exiting from the IP configuration screen and restarting the management network the IP address changes back to the vMotion address.

As mentioned earlier, we have narrowed this down to the vmk numbering. If vmk0 is assigned to vMotion and vmk1 assigned to Management then this issue occurs. For ESXi hosts that have assigned vmk0 to Management and vmk1 to vMotion this issue does not occur. To fix the issue we have to remove the vMotion and Management portgroups, then re-create the Management port group which then claims vmk0 and all is ok.

So the question we are trying to answer is why/how ESXi changes the Management vmkernel port from vmk0 to vm1. We built the hosts ESXi manually so its safe to assume that vmk0 would have been allocated to the Management vmkernel port as this is the first vmkernel port to be created. Is this expected behaviour to have vmk0 take higher priority? Should it matter which vmk number is assigned to Management anyway?

I have seen other users post similar observations and it seems Host Profiles is the culprit. I am surprised VMware have not chimed in to these previous posts considering the impact. See:

http://communities.vmware.com/message/1521966#1521966

and

http://communities.vmware.com/message/1597396#1597396

I will be logging a call with VMware but would like to hear if anyone who has ESX 4.1 running and has vmk0 assigned to vMotion and vmk1 assigned to Management to reboot the host and report the IP address on the DCUI.

Cheers

gogogo5

[davidw_davis]

I see what you're saying. I think my vmk0 was associated with my vSwitch0 but I removed that vmkernel network to make it a vm only network. Perhaps this is why vmk1 is being used for managment? I currently don't have a vmk0 network. Just vmk1, vmk2 and vmk3.

[DyJohnnY]

We are seeing the same problem, the ESXi hosts have no fault, the host profiles are the root cause of the problem.

Obviously the ramifications of this are much greater than simply having the vmk ports out of order. We have had nexus 1000V switches report the IP address of the host they were managing as the vMotion address. i guess it has to do more with the etc/hosts file, than the switches actually asking for the management IP adddress.

Also when ESXi hosts registered on our Clariion SAN they registered with the vMotion IP address (it was more a cosmetic issues but strange to see it there).

___________________

IonutN, I blog here - www.rivnet.ro

Message was edited by: DyJohnnY I removed the workaround we proposed since we discovered it doesn't really work as we expected it to

[Datto]

Just wanted to mention this problem still exists in VSphere 5.0 so whatever coding errors existing in ESX 4.x that were causing this problem more than a year ago must still exist in ESXi 5.0.

It appears to me the problem is caused by applying the profile (I'm positive all the ESXi 5.0 hosts were setup correctly with Management on vmk0 with the correct IP prior to applying the host profile). However, after Host Profiles were applied, most (but not all) ESXi 5.0 hosts had their vmk0 and vmk1 assignments flipped (ie, Management became vmk1 and vmotion became vmk0 after applying host profile to the ESX 5.0 hosts). The ESXi 5.0 reference host that was the example used to create the host profile remained with Management on vmk0 and VMotion on vmk1 while all the other ESXi 5.0 hosts in the cluster got the vmk0 and vmk1 assignment flipped. One of the negative results of this, as mentioned by others above, is the on-screen address of the ESXi 5.0 console is shown wrong -- it shows the vmotion IP instead of the mangement IP.

Two other clusters in the same VC datacenter that had host profiles applied to the cluster and associated cluster members didn't have this problem during the process of applying the host profile (each cluster had it's own reference host used to create the reference host profile).

Also, these were all fresh ESXi 5.0 installs that were installed from media, not upgrades. I used kresimir.pirkl workaround to fix it with these ESXi 5.0 hosts. We should officially call that the "KP workaround".

Hope that helps someone who many be looking into this.

Datto

[gogogo5]

Hello Datto - thanks for providing an update and its disappointing this issue still occurs in vSphere 5.0.  Do you have a support agreement with VMware and if so, would you be willing to log a SR for this as I was told by VMware this issue was going to be fixed in v5.0.

Cheers

gogogo5

[hharold]

Very disappointed that this still has not been fixed.

We had the same issue in 4.1, and were hoping that his would be fixed in 5.0

In vSphere 5.0 VMware has introduced another dependency on vmk numbering...

The VMware Dump collector has to be configured on a fixed vmk port (eg. vmk0, vmk1 etc)

If you want to do this with Host Profiles, you must be able to rely on consistent vmk numbering, which we can't.

[haroldf]

@kresimir

Still seeing the issue on 5.0 and this corrected it. Thanks for the fix!

[SuperSpike]

The steps to recreate the issue posted by Datto is spot on. That's exactly how to recreate the issue in 4.1 as well.

I've had an SR open with VMware Support on this issue since at least July 2011! The PR associated with the bug is 580080. The latest update I've received from Support is that the fix will be implemented in version 5.1. Keep on waiting!

It is pretty simple to fix by manually modifying the offending host profile (as previously documented in this thread). However, if you've gone ahead and applied the bad profile to many hosts already, it could take a long time to resolve due to each host having to be in maintenance mode. All the more reason I feel host profiles are a huge waste of money. Everything they do can be done with PowerCLI for nothing. Plus, they don't do anything with your storage config. Worthless!

[MichaelSteg]

Thank you for your e-mail.

I'm currently out of office till Monday, 23th of April 2012. During this time I will have no access to my email.

Your e-mail will not be forwarded. For urgent concerns please feel free to contact the main office on +41 43 388 72 00

Best regards,

Michael Steg

Michael Steg

Systems Engineer

Bechtle Regensdorf AG

Bahnstrasse 58/60

CH-8105 Regensdorf

tel: +41 43 388 79 31

gsm: +41 79 964 25 14

fax: +41 43 388 72 01

michael.steg@bechtle.ch<mailto:michael.steg@bechtle.ch>

www.bechtle.ch<http://www.bechtle.ch/

[danzig69]

problem always there with ESXi 5.0.0 Build 702118 !! In short, leave Management traffic at vSwitch0 and vmk0 after the installation.

[erickmiller]

Anybody try this in 5.1?  I just ran into this issue with 5.0 Update 1b.  Fun fun.

Eric