What would cause a VM to get deleted?

[mlrhazi]

Hello,

I wrote a script that performs some database updates on a VM. It does so by attaching disks to the VM, uploading some files and running a script inside the VM.

The only CmdLets am calling are Invoke-VMScript, Copy-VMGuestFile, Get-HardDisk, Remove-HardDisk, New-HardDisk

Twice so far, the script would fail and I would discover that the VM does not exist in vcenter anymore! Looking in the logs I find that the VM got powered off, during my script run, and got unregistered:

2019-02-04T03:58:07.571560-05:00 esxsrv-c5.cua.edu Hostd:  info hostd[12C43B70] [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 96380 : Guest operation List Processes performed on Virtual machine CSDBORAUAT01-C.

2019-02-04T03:58:07.581340-05:00 vcenter vpxd[48551]  Event [2838593] [1-1] [2019-02-04T08:58:07.578664Z] [vim.event.EventEx] [info] [CUA\VEEAM-SERVICE] [Leahy-DR] [2838593] [Guest operation List Processes performed on Virtual machine CSDBORAUAT01-C.]

2019-02-04T03:58:10.968975-05:00 vcenter vpxd[48551]  Event [2838598] [1-1] [2019-02-04T08:58:10.966512Z] [vim.event.VmStoppingEvent] [info] [CUA\VEEAM-SERVICE] [Leahy-DR] [2838597] [CSDBORAUAT01-C on  esxsrv-c5.cua.edu in Leahy-DR is stopping]

2019-02-04T03:58:10.977536-05:00 esxsrv-c5.cua.edu Hostd:  info hostd[13985B70] [Originator@6876 sub=Vimsvc.ha-eventmgr opID=490e4cf4-83-0207 user=vpxuser:CUA\VEEAM-SERVICE] Event 96382 : CSDBORAUAT01-C on  esxsrv-c5.cua.edu in ha-datacenter is stopping

2019-02-04T03:58:35.159218-05:00 esxsrv-c5.cua.edu Hostd:  info hostd[13985B70] [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 96383 : CSDBORAUAT01-C on  esxsrv-c5.cua.edu in ha-datacenter is powered off

2019-02-04T03:58:42.365016-05:00 vcenter vpxd[48551]  Event [2838617] [1-1] [2019-02-04T08:58:42.363276Z] [vim.event.AlarmStatusChangedEvent] [info] [] [Leahy-DR] [2838617] [Alarm 'Virtual machine CPU usage' on CSDBORAUAT01-C changed from Green to Gray]

2019-02-04T03:58:42.368607-05:00 vcenter vpxd[48551]  Event [2838618] [1-1] [2019-02-04T08:58:42.366874Z] [vim.event.VmPoweredOffEvent] [info] [CUA\VEEAM-SERVICE] [Leahy-DR] [2838597] [CSDBORAUAT01-C on  esxsrv-c5.cua.edu in Leahy-DR is powered off]

2019-02-04T03:58:42.391214-05:00 vcenter vpxd[48551]  Event [2838619] [1-1] [2019-02-04T08:58:42.389822Z] [vim.event.AlarmStatusChangedEvent] [info] [] [Leahy-DR] [2838619] [Alarm 'Virtual machine memory usage' on CSDBORAUAT01-C changed from Green to Gray]

2019-02-04T03:58:44.215799-05:00 vcenter vpxd[48551]  Event [2838623] [1-1] [2019-02-04T08:58:44.213694Z] [vim.event.VmRemovedEvent] [info] [CUA\VEEAM-SERVICE] [Leahy-DR] [2838620] [Removed CSDBORAUAT01-C on esxsrv-c5.cua.edu from Leahy-DR]

2019-02-04T03:58:44.223485-05:00 esxsrv-c5.cua.edu Hostd:  info hostd[13985B70] [Originator@6876 sub=Vimsvc.ha-eventmgr opID=178aafac-1e-035a user=vpxuser:CUA\VEEAM-SERVICE] Event 96384 : Removed CSDBORAUAT01-C on esxsrv-c5.cua.edu from ha-datacenter

Is there any way my script would actually have requested that the VM be powered off and un-registered, even though I don't seem to have any call to such commands ?

[LucD]

I would need to see your script your script to actually see what it is doing.

But a VM can become unregistered when for example a vMotion goes bad.

Are you by any chance connected to an ESXi server and not a vCenter?

[mlrhazi]

Thanks LucD. My script is a bit tool long... I will try to write a smaller one that hopefully reproduces the issue.. if I can.

Yes, it does connect to vCenter, and not specific esx hosts. and no, no vMotion events appear to have occurred, judging by the  esx and vCenter logs.

The log line does mention the username which my script uses to connect to vCenter and issue its commands:

2019-02-04T03:58:44.215799-05:00 vcenter vpxd[48551]  Event [2838623] [1-1] [2019-02-04T08:58:44.213694Z] [vim.event.VmRemovedEvent] [info] [CUA\VEEAM-SERVICE] [Leahy-DR] [2838620] [Removed CSDBORAUAT01-C on esxsrv-c5.cua.edu from Leahy-DR]

Thanks a lot.

[LucD]

Perhaps you should a look at the Tasks & Events happening around that time.

They might give an indication what caused it.