My ESXi 3.5 machine runs 8-10 VMs (Win2k3 and WinXP) normally. At the moment, 5 of them are complaining that they cannot Power On. They seem to start and then complain "Could not power on VM: no swap file". I had a look with the data browser. It's a small installation, so the vswp files ought to be in the same directory as the vmx file (I did not inttionally put them anywhere else). Of course I don't see a vswp file there because the machine is not running. I don't know enough about the vmx file structure to identify if anything is wrong in the specifications. I have downloaded one of the vmx files and attached it here. Please either tell me what to change in that vmx file, or suggest another approach to get the machines to start.
Regards - Glen
Thanks Dan. Clearly it's not a process lock as I have rebooted the host. If the only other way, really, is that curiously named "heartbeat region" (sounds a lot like a cache to me) then I guess that must be it. Doesn't feel right though. As you say, curious that it only affected the vswp files across 4 or more VMs.
At the moment, I have 4 VM directories that contain all the original files plus the untouchable vswp files. I will delete almost everything in those once I'm convinced the rebuilt VMs are fine. I am even able to rename these directories, so it doesn't sound like software that used the fully qualified filespec to identify the file - it's probably at more of a hardware level that has identified the file by block number address on the platter (so maybe your heartbeat theory is right). If those deletes go OK I won't even bother to find ways to delete the corrupted vswp files.
The interesting thing to note is that these files have length = -1 bytes and a datetime stamp of 1 second before time zero which was Jan.1 1970. It's as though the file header in the table of contents has been overlaid with FFFFFFFF. In that state I cannot touch them - mv, cp, rm and ls all report that the file or directory does not exist. They're there, but they're not there. In Windows I bet a chkdsk would clean up the indexes - is there something like that in *nix?
In my shutdowns I have shutdown all machines but not usually gone to maintenance mode before power cycling the host. I'll try to remember that.
Regards - Glen
there's no chkdsk or fdisk type utility for VMFS that I'm aware of. There is a way to back up all of the file location data so that you can recover deleted/lost files, but you have to run the script prior to the occurance so the files can be recovered.
If the preceding steps fail, perform the following steps for an ESX 3.x host:
List all running virtual machines to find the VMID of the affected virtual machine with the command:
Determine the master world ID with the command:
less -S /proc/vmware/vm/####/cpu/status
Scroll to the right with the arrow keys until you see the group field. It appears similar to:
Run the following command to shut the virtual machine down with the group ID:
/usr/lib/vmware/bin/vmkload_app -k 9 ####
ESX 3.5 Update 4 / VC 2.5 Update 5
If your servers are in a cluster, you need to find the exact server in the cluster that "owns" the process.
Hope this helps!