ESXi 3.5u4 (build 153875) not starting VMs automatically on start-up.

[asincero]

I'm having an issue with my ESXi 3.5u4 (build 153875) server not automatically starting VMs on system start-up. First, let me say that "yes" I do have it configured to automatically start VMs on system start-up. The problem seems to be related to the fact that I have the NICs configured to use DHCP and the VMs live on an NFS server.

The ESXi box has two NICs (actually it has 5, but for the sake of this discussion I'll just say 2). One NIC provides access to the LAN so I can connect to it via VI Client. The other NIC provides access to a dedicated gigabit LAN to the NFS server. I have both NICs configured to get it's IP settings via DHCP. With this configuration, this is what I see from VI Client after system start up:

As you can see, the virtual machines configured on ESXi host "charlie.milcom.arinc.com" are listed as "Unknown" and "Inaccessible". When I go to the storage configuration screen, this is what I see:

As you can see, the two configured NFS datastores are listed as "inactive". However, when I go on the NFS server (which is a Debian GNU/Linux 4 server, if that is relevant), I see in the logs that those shares have been successfully mounted by "charlie".

When I click on "Refresh", then everything comes back to life:

The NFS datastores are listed as being available, and my VMs appear in the left hand panel. At this point, I can then manually start the VMs. It'd be nice if ESXi would retry starting the VMs by itself so it'll come on just as soon as whatever datastore the VM lives on becomes available, but apparently it doesn't do this.

This problem goes away if I configure the NICs to not use DHCP and just configure them statically; everything comes up by itself as it should. However, I would prefer to have the NICs be configured via DHCP. This way IP assignments for everybody on the network is done at one spot (the DHCP server). Besides, the option to use DHCP is being made available so it should work without any problems.

Is this a bug or am I doing something wrong?

[DSTAVERT]

Does DNS resolve properly?

I personally would want to have fixed addresses on important services and unless you have a very large installation fixed addresses on ESXi host.

[asincero]

Yes, DNS does resolve properly. Like I mentioned, ESXi does in fact mount the NFS datastores eventually (as evidenced by the logs on the server and if I manually hit "Refresh" on the storage configuration screen from VI Client). But apparently, it doesn't mount them soon enough during the system start-up process when DHCP is being used to configure the NICs.

I agree, using DHCP to configure IP settings for important servers does sound a little fragile. However, the DHCP server, NFS server, and DNS server all live on the same box. Since the VMs for the ESXi servers live on the NFS server, the ESXi servers are useless anyway if the DHCP/NFS/DNS server is down. The DHCP/NFS/DNS server isn't a single box, though, but a two-node high availability cluster. So it being an HA cluster alleviates some of the concerns of the ESXi servers needing to depend on another machine in order to be useful.

Also, the DHCP server doles out the same IP for a given MAC address; so IPs aren't dynamically allocated from a pool of available IPs. IPs are "pseudo-static" if you will. Again, my reasoning for using DHCP here is to have IP assignments being done in one spot. We have a lot of machines on the network. Using VMs, I think, makes the situation worse since we can really go hog-wild with the creation of VMs with ESXi. I found it to be a real pain keeping track of whose got what IP so I'm thinking keeping all of this in one central location would go a long way to keeping things more organized.

[DSTAVERT]

Since you would add the host and IP to your dns server I'm not sure you would be out of control with IP addresses. With some things fixed addresses are important. You don't get to assign DNS servers by name.

But that's me. Have you tried extending the startup delays.

[RParker]

The most obvious thing I can point out that since you have to 'refresh' to see your datastore, that's the problem, and not a ESX / Autostart problem at all.

Check the logs on the ESX host to see if there is a failure to mount or some other problem, that might give a clue as to why you can't see the VM's. If the VM's aren't visible, then they won't start. So what needs to happen is what you do manually to 'refresh' that location that ESX isn't doing upon restart.

Another thing to point out is this may not ultimately be a problem. Why are you rebooting the ESX host in the first place? Normally they should not restart or shut down except for maintenance and patching, which is maybe twice a year, and you wouldn't start the maintenance and just walk away, so autostarts really isn't an issue because the server restarts should not happen that often, unless you have a need to shut them down regularly.

[asincero]

I have the start up delay set to 2 minutes. But I don't think it'll matter how long I set it. ESXi thinks the NFS datastores are not available, and thus doesn't try to start up the VMs.

[asincero]

Yes, you are right. This probably isn't going to be much of a problem in practice. The ESXi servers recently got rebooted due to a power outage, which is when I noticed that the VMs aren't starting up by themselves. I thought they were on UPS, but apparently they weren't. The electrician was supposed to have all electrical outlets in the lab connected to our central UPS, but some are and some aren't and wouldn't you know it they're not labeled. I've since put them on UPS and verified that they are.

I don't think everything could come back up by themselves anyway since the ESXi servers depend on the NFS server to be up and running first, and the NFS cluster takes quite a bit longer to come up than the ESXi servers. I'd have to somehow set it up so the ESXi boxes don't get powered on until the NFS server is fully available. Again, it would be nice if ESXi would continuously retry to mount the NFS datastores until success, and then start the VMs.

ESXi starting the VMs automatically in my particular setup can certainly be classified as "nice-to-have" but not essential.

[DSTAVERT]

Try an ESXi boot delay so the NFS server comes online before the ESXi host.

http://vm-help.com/esx/esx3i/boot_delay.php