We have experienced a strange ESXi 4.1 IP addressing anomaly where the ESXi host's management IP address is changed to the vMotion address.
In our lab we have 2 ESXi hosts with 6 pNICs. All pNICS are configured on a single standard vSwitch. All pNICS are Active (no Standby or Unused) and configured for Route Based on IP hash. The 6 pNICS are uplinked to a Cisco 3120 stack where Etherchannel is configured. All works ok so far.
We have found that if the ESXi host has assigned vmk0 to vMotion and vmk1 to Management, upon rebooting the ESXi host the vMotion IP address is shown as the IP address (i.e. the management address) on the DCUI!!!! When using the DCUI to change the IP address back to its correct IP address, when exiting from the IP configuration screen and restarting the management network the IP address changes back to the vMotion address.
As mentioned earlier, we have narrowed this down to the vmk numbering. If vmk0 is assigned to vMotion and vmk1 assigned to Management then this issue occurs. For ESXi hosts that have assigned vmk0 to Management and vmk1 to vMotion this issue does not occur. To fix the issue we have to remove the vMotion and Management portgroups, then re-create the Management port group which then claims vmk0 and all is ok.
So the question we are trying to answer is why/how ESXi changes the Management vmkernel port from vmk0 to vm1. We built the hosts ESXi manually so its safe to assume that vmk0 would have been allocated to the Management vmkernel port as this is the first vmkernel port to be created. Is this expected behaviour to have vmk0 take higher priority? Should it matter which vmk number is assigned to Management anyway?
I have seen other users post similar observations and it seems Host Profiles is the culprit. I am surprised VMware have not chimed in to these previous posts considering the impact. See:
I will be logging a call with VMware but would like to hear if anyone who has ESX 4.1 running and has vmk0 assigned to vMotion and vmk1 assigned to Management to reboot the host and report the IP address on the DCUI.