vMA, PCNS & HA

[jpmyers]

I am using PCNS 3.0 on vMA 5.0 (esxi 5.0) and looking to have a complete shutdown in the event of having x number of minutes of battery time left.  My testing has been successful and is down to one last hurdle (at least I think it's the last one).  Our environment has HA enabled, so prior to shutdown I need PCNS to run a file or script to disable HA.  Each host has a vMA with PCNS on it.  Does anyone have a script that PCNS can call and run on the vMA to disable HA on the host?  I would REALLY appreciate any help with this as I am in unfamiliar territory. Thanks.

[lamw]

What's the reason for disabling HA as part of the shutdown process?

[jpmyers]

The reason for disabling HA is that during testing with 2 hosts (each with 1 vm) when we restarted everything after the test, we noticed one of the vm’s had been moved to the other host. We ended up with one host with no vms and the other with two. Each had one vm prior to the test. We don’t want mass moving to occur in the event of a shutdown originated by PCNS.

[jpmyers]

Is there a better way to initiate a complete shutdown of both physical and virtual machines on a low battery condition?  PCNS is working very well (and it's free now) if I can only get past this last snag.  I just need to know how to disable HA from the vMA.  Thanks in advance - any help is much appreciated.

[lamw]

If you have DRS enabled, it will automatically load balance your workload between the hosts. If the workload can in fact run on one host based on what DRS determines, you could even enable DPM to have the other host go into standby mode to conserve energy. In any case, disabling HA is not a good idea in terms of what you're trying to accomplish.

[jpmyers]

The vmware environment for this particular location is 3 DCs, 4 Clusters, 12 Hosts and about 120 vms.   The condition\situation I am trying to take care of is when we have lost power and the UPS battery time is down to 9 minutes (or so) and EVERYTHING needs to be shut down gracefully.  Using PCNS, I need a script to run on the vMA (called from PCNS) to disable HA on the host prior to shutting it down.  I don't want anything to move anywhere else.  This is a situation where everything has to be powered off, nothing needs to be load balanced or  moved somewhere else, just shut down gracefully.  My goal is to power everything down.

All I am looking for is how to disable HA from the vMA.

Thanks.

[lamw · Accepted solution]

Again, I'm still not sure why HA needs to be disabled? I'm trying to understand how that helps you in this situation, is it to ensure that DRS doesn't move workloads around as your VMs are being shutdown? It would be great if you could elaborate what you're seeing with HA if it was left enabled during this?

I've created a very simple vSphere SDK for Perl script called HAmanagement.pl that allows you to enable/disable vSphere HA which can run on vMA or a system with vCLI installed.

To use the script, you just need to specify --operation and --cluster

[jpmyers]

First, let me thank you very much for the script and the time you took to help me!

During testing while HA was enabled we were intermittently seeing where Host1 would either:

1. Shutdown without shutting down VMs properly and then HA starting them up on Host2 before it got the shutdown command.
   

Or

1. Shutdown the VMs correctly and then power off but HA for some reason would not recognize this and again Host2 would begin startup of the VM but Host2 would shut down in the midst of the VM startup.
   

Both scenarios caused the VMs to log event “the previous shutdown was unexpected.”

[lamw]

I'm not familiar with PCNS implementation but it sounds like there might be some issues on how they're actually performing the shutdown. If you're shutting down the VMs, HA will not be restarting them ... it's only if a host fails or if VM HA is enabled. Though you have a work around, I would still question on why you're seeing this and ask if they can give you better information.

Good Luck

[jpmyers]

Your expertise is requested one more time.

The only mod I made to the script is line 58 – where I added an “s” to find_entity_view – I did so because it wasn’t recognizing the cluster name.

my $cluster_view = Vim::find_entity_views(view_type => 'ClusterComputeResource',filter => {"name" => $cluster});

So it makes it to “Disabling HA …” then

script is generating ‘Can’t call method “ReconfigureCluster_Task” on unblessed reference at HAmanagment.pl line 79’.

Line 79 is

my $task = $cluster_view->ReconfigureCluster_Task(spec => $spec, modify => 'true');

Any additional help would be greatly appreciated.

[lawson23]

For everyone regarding pcns:

A simple explanation of how it works with esxi: pcns receives a message from APC management card saying battery to low, pcns then tells vma to issue a shutsdown command to the host and the host should be configured to then do a guest shutdown of the vm's.

I understand what is happening as HA is doing what it is designed to do but may i suggest something else as an option.

Install the pcns into your vm's set them to shutdown at 8 minutes of runtime after loss of power notification and set your pcns on your vma for your host to shutdown at 10 minutes of runtime.  Obviously adjust times as required by your environment.

This is not as pretty as how the pcns vma solution is set to work but would eliminate your need for disabling HA.

[lamw]

First off, you should understand what you're doing if you're going to make modifications to the script. The "s" you're adding means something completely different than without ....

Vim::find_entity_view(.....) = Will return a single cluster type and filter based on the name you've spcified on the command-line, this assumes you type in the correct cluster name (spaces, case-senstivity, etc).

Vim::find_entity_views(....) = Will return all clusters, hence the "s" as in more than one

Processing the latter requires some additonal logic, so again, just adding the "s" is not correct. I would double check the spelling of your cluster name in vCenter

[ewilsoniii]

I also have the same problem as described by jpmyers.We are running ESXi 5 with Update 1. If I have HA enabled, my powered off VMs will be migrated to the hosts that are still running which is not what we want to happen in this scenario. I tried using the HAmanagement.pl script to disable HA (which worked great !) first and the VMs and host shutdown properly. The problem I have is that in order to run the HAmanagement.pl script successfully, I first have to use vifptarget --set <hostname> where hostname is my Virtual Center server. How do I add this functionality to the HAmanagement.pl? If I can get this working, I am golden before hurrican season starts.

[lamw]

You can take a look at these two references:

http://www.virtuallyghetto.com/2011/01/how-to-automate-cron-vi-fastpass.html

http://communities.vmware.com/docs/DOC-10883

vi-fastpass is exposed in both Java/Perl, so you can definitely write a script that leverages without needing to use the vifptarget (that's just for ease of use from a CLI perspective).

[GTooke]

Hi I am responding to this post as I have come across the same problem of VM's migrating with HA as a host powers down with PCNS. I have tried to run the HAmanagement.pl but with no success.

Has anyone managed to get this to work?

If so do you have detailed instructions?

We have two Clusters with two hosts in each cluster. HA is enabled so if a host goes down it will migrate the VMs.

However what happens if both Hosts are instructed to shutdown?

Obviously the hosts will not shutdown at the same time so what happens if the migration takes place while the 2nd host shuts down. Does the migration just fail and not cause any problems or does it screw things up when the host comes back?

The other problem is that each host is configured to shutdown or power up the VMs attached to it. If any VMs migrate then the shutdown / start up list is then incorrect on each host and then incorrect when there is a power failure or when the Host tries to power the VMs back up.

Ok so not a real problem as we could sort this out after a power failure.

I have installed vMA and PCNS onto one of the Hosts and have tested it with a simulation of a power cut. It instructs the host to shutdown, the host then shuts down each VM. This works fine, just that the VMs migrate to the other host as it is still up. We are going to add the second host to the config file of the vMA script so that it shuts down as well.

We are just a bit concerned that the migration of the VMs might screw things up if there ever was a power cut. Hence we thought that disabling the HA via the vMA would be the best option.

Does anyone have any experience of this? What are the best practises and has anyone got any instructions for running the HAmanagement.pl?

Any help would be appreciated!

Kind Regards

Gary Tooke

[ewilsoniii]

Looking at the above references, I got it to work by creating a shell script that calls HAmanagement.pl. Attached is the shell script. Replace the example VCenter and cluster names with yours to make it work.

[MrPolarBear]

We ran into a similar problem with ESXi 4.1 cluster and the need to use PCNS 3.0, we tried vMA and ended up finding a workaround using our physical vCenter server as the PCNS host (not elegant but nevertheless works with a bit of tweaking and continue monitoring), see http://communities.vmware.com/thread/329180

Hope this provides an alternate workaround.

[dgrehan]

See Kbase on APC web site:

http://nam-en.apc.com/app/answers/detail/a_id/11622/kw/HA/session/L3RpbWUvMTMzOTYwMDg4My9zaWQvR1NPWk...

Script will connect to ESXi hosts and gracefully shut down any powered on VMs before PCNS issues the command to shut down the ESXi hosts, without disabling HA.

[0zzY]

hi lamw

i'm trying to run it on a vsphere 5.5 with this command: ./HAmanagement.pl --cluster Cluster --operation disable

but i receive this error: Error connecting to server at 'https://localhost/sdk/webService': Perhaps host is not a vCenter or ESX server

Someone can help me, i am new with vCLI

[0zzY]

sorry, i have resolved running:

./HAmanagement.pl --server 10.1.200.82 --cluster Cluster --operation enable