So after a long time of leaving this issue on the back burner, I was finally able to spend some time and come up with an acceptable solution.
I used a combination of the esxi-control.pl script, coupled with a powershell script I made, along with a third party vendor named Temperature Alert's product. We used their $129 USB version: http://www.temperaturealert.com/Wireless-Temperature-Store/Temperature-Alert-USB-Sensor.aspx
Here's a quick run down of our environment and how this works.
In our server cabinet we have four machines, currently two ESXi 4 hosts using Dell poweredge T610 hardware, one older dell server to run our accounting system and finally our voicemail server:
-Host #1: has one server 2008 vm which is our PDC/DNS/DHCP/File/Print server.
-Host #2: has two vm's. One server 2008 that's our SQL 2008 DB server and one server 2008 r2 that's our exchange 2010 server.
Our fear was that one day the small A/C unit, which is mounted to the ceiling in the basement below the server cabinet might fail and damage hardware/cause a failue while the business(s) are closed.
I first line of defence was notification. The temp@lert usb product satisfied this issue with the ability to send out an email when the temperature exceeds a difined level. In this case, if the temp was getting out of control, my boss or I could log in remotely and shut down the machines. But, what happens if neither one of us are available to log in remotely? This motivated us to find a way to automatically, gracefully shutdown both the vm's and the hosts based off the cabinet temperature.
The temperature alert device has the ability to execute a powershell script each time it reads the temperature; I initially wanted the temp alert software to execute a batch file upon reaching a certain temperature, that would call the perl script, thus gracefully shutting down our esxi hosts and vm's. Nope, not happening. The temp alert software is limited only to PowerShell scripts and runs it everytime it samples the temperature, in our case, every 120 seconds. I contacted tech support for temp alert and I was provided a PS script example that would record the temp data to a XML file which could later be parsed as needed. After tweaking the PS script I received from them, I then went on the write a new powershell script to parse the data in the XML file. The PS script is run as a scheduled task on our accounting server and is capable of the following: Sends out a warning email when the temp reaches defined level, shuts down the ESXi hosts and shuts down the accounting server.
I used an old workstation with and an evalution copy of ESXi 4 to test with and last weekend we tested it on our production servers. We shut off the A/C and basically sat there and watched the temp rise. It worked perfectly! It only took about 50 minutes for the cabinet to exceed 95*F. Warning email went out, email with shutdown immenent message went out and all machines shutdown cleanly. I also added our Verizon mobile phone numbers @vtext.com to a new exchange distribution group so we receive sms messages with the temperature/shutdown information; this is in case we are somewhere outside of data coverage.
Basically the order of operation is this:
Temp Alert software reads the temp >> writes/overwrites said xml file with temperature information every 120 seconds
>> Scheduled task runs PS script every 4 minutes on accounting server which is where the temp alert usb device is installed.
PowerShell script parses data from XML, if the values in the PS script are satisfied, a warning email, shutdown immenent email/host shutdown will occur.
To shutdown the hosts, the PS script executes batch files, which call the esxi-control.pl script and provide the credentials neccessary to perform the shutdown of the host. And since the vm's have the VMware Tools installed, the host is able to cleanly shutdown the guest machines. Not too bad for only having to spend $129 on a simple usb temperature device.
