OK I have a client that wants to test HA but does not want to pull the power plug. They would like me to move some VM's off a host and leave a couple for the test. They then want me to issue a shutdown command from the ESX host and they think HA will kick in once the ESX servers is down.
I do not think this will work. My gut tells me that once the host goes down clean all the VM's will have been stopped and HA will not think it needs to do anything .
Anyone done this test? Have an docs to point to?
Steve Beaver
VMware Communities User Moderator
Orlando Area VMware User Group Leader
====
Co-Author of "VMware ESX Essentials in the Virtual Data Center"
(ISBN:1420070274) from Auerbach
*Virtualization is a journey, not a project.*
Make sure your Host isolation reponse is set to "power off" to test this way.
Steve
I've gone back and forth in my head, I guess you can do a force shutdown but would that truely be a clean shutdown? I doubt the ESX server would allow you to do a clean shutdown if it is still running VM's on it. I think if you do a force shutdown, HA will kick in and you can run your test. The true test is the pull the plugs, even though it is hard on the server hardware... isn't that what you have service contracts for? I can try and get my R&D configured here today and run a trial on your theory though if you want.
Kyle
If you have the lab to do it that would be great. I am just more curious now to see what happens but I do not think they will get the desired results
Steve Beaver
VMware Communities User Moderator
Orlando Area VMware User Group Leader
====
Co-Author of "VMware ESX Essentials in the Virtual Data Center"
(ISBN:1420070274) from Auerbach
*Virtualization is a journey, not a project.*
I think that your scenario would invoke a DRS migration rather than a HA. I would try downing the Service Console network (make sure you have physical or ILO access) this should then kick a HA event.
If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points
Tom Howarth
VMware Communities User Moderator
Pull the NIC cable
I do not think this will work. My gut tells me that once the host goes down clean all the VM's will have been stopped and HA will not think it needs to do anything .
That's correct, HA will not kick in. But they don't have to pull the plug, won't simply pulling out the management Network do the same thing? What happens if an ESX server loses connectivity how would VC know if it's TRULY down or not?
Also this is a controlled method, the ONLY way to truly test this is to ACTUALLY pull the plug, otherwise when disaster happens, you won't get a TRUE method of what happens.. so even if this conjecture works like we THINK, it won't matter, because they don't to test like it SHOULD be tested. Pull the plug! That's the only way. It should be random, not on one particular server.
Steve you are the man! You know it's working.. prove it!
I had a machine in a HA /DRS cluster in maintenance mode that I was going to down anyway so I was able to test it.
Set DRS to manual, exited maintenance mode and moved a vm to the host.
At the console issued shutdown now -h
HA kicked in and the vm was restarted on another host.
OK I am not sure about you all, but for me, I do not set HA to act upon isolation. I configure things to let the VM keep running until I manually decide to act. Quick change should fix that so no problem
So during the shutdown does ESX use the vmware-cmd /path/ stop trysoft / hard on the VM's or just kill -9 anything left running?
Make sure your Host isolation reponse is set to "power off" to test this way.
So tell me in the logs did the VM go down clean first or hard. Just curious what methos the shutdown scripts use to stop a VM
Steve Beaver
VMware Communities User Moderator
Orlando Area VMware User Group Leader
====
Co-Author of "VMware ESX Essentials in the Virtual Data Center"
(ISBN:1420070274) from Auerbach
*Virtualization is a journey, not a project.*
> At the console issued shutdown now -h
yes but shutdown is a linux command, done from the console, not the same thing. the VMkernel doesn't know what'a happening, it just thinks there is a problem. So issuing a shutdown from VIC is different, since it will notify the VM Ware services first, and it won't react the same way.. It will shutdown cleanly, and HA will not kick in.
There are other things going on behind the scenes with VIC than a simple shutdown command from the console. . . .
hmmm now I wonder what command the VIC will use
Steve Beaver
VMware Communities User Moderator
Orlando Area VMware User Group Leader
====
Co-Author of "VMware ESX Essentials in the Virtual Data Center"
(ISBN:1420070274) from Auerbach
*Virtualization is a journey, not a project.*
I understand that they're different, that's why I shut it down via the command line.
I'm not finding anything in the logs, host or vm about it bieng shutdown.
I would assume that when you run shutdown from the command line it just kills all running processes.
You can also do it all remotely via ILO, just use the command esxcfg-vswif -s <vswif name> to disable the service console thus the HA heart beats... after testing use esxcfg-vswif -e <vswif name> to enable... esxcfg-vswif -l will display the vswif names...
I use this command all the time to test HA, also very good when you have more than one SC network so you can demonstrate that HA doesnt kick in until you loose ALL SC networking connectivity.
Have fun
I have tested this by using the esxcfg-vswif -D command. This disables all vsif interfaces, making like this was a trus host failure. Take in hand I have an iLO connection to the host. After testing, I do the esxcfg-vswif -E, which enables all vswif connections and do a services mgmt-vmware restart. all starts working after this.
Hope that helped.