Deleted Snapshot vmsn file Vmware ESXi 5.1

[RaimondsMazulis]

Hello,

Deleted ffom vSphere client in datastore one and only Snapshot vmsn file, i have xxxx-000001.vmdk file.

Can i recreate snapshot?

In snapshot manager i can see it! But error when reverting:

Failed to revert the execution state of the virtual

machine XXXX to snapshot XXXXX

1244, with ID 63

error

root

Best regards,

R

[continuum]

You have 2 options:

- the pragmatic approach: you do not really need the vmsn unless it is essential that you can go back to a hot snapshot (one that was created while the VM is running)

- the hard way .... - with esxi 5.1 you have a decent chance that the deleted vmsn is still available and can be recovered

I can check that for you - read Create a VMFS-Header-dump using an ESXi-Host in production | VM-Sickbay

Create a dump for version VMFS 5 and send the dump my way.

To start the VM without vmsn do this:

- unregister VM

- delete vmss-file if present

- rename existing vmsd to vmsd.org

- edit vmx-file manually so that it points to the xxxx-000001.vmdk instead of the one that is configured right now

- remove all references to the vmsn in the vmx-file - in case it is referenced

- reregister the VM

Do not expect that you can do this from the GUI - use WinSCP - embedded texteditor instead

---------------------------------------------------------------------------------------------------

this will force the VM to start from powered off state.

[RaimondsMazulis]

/vmfs/volumes/59f5094e-f86e82f9-607e-001e4f24fc12/Windows server # ls -l *

-rw-------    1 root     root     11190468608 Feb  4 05:59 Windows server-000001-delta.vmdk

-rw-------    1 root     root           354 Feb  4 06:52 Windows server-000001.vmdk

-rw-------    1 root     root         65536 Feb  4 22:24 Windows server-000002-delta.vmdk

-rw-------    1 root     root           338 Feb  4 22:24 Windows server-000002.vmdk

-rw-------    1 root     root         29268 Feb  4 22:24 Windows server-Snapshot64.vmsn

-rw-r--r--    1 root     root            13 Jan  7 09:44 Windows server-aux.xml

-rw-------    1 root     root     32212254720 Jan  7 10:28 Windows server-flat.vmdk

-rw-------    1 root     root          8684 Feb  4 06:58 Windows server.nvram

-rw-------    1 root     root           501 Jan  7 09:44 Windows server.vmdk

-rw-r--r--    1 root     root          1675 Feb  4 22:24 Windows server.vmsd

-rwxr-xr-x    1 root     root          3637 Feb  4 22:24 Windows server.vmx

-rw-r--r--    1 root     root           372 Feb  4 06:52 Windows server.vmxf

-rw-r--r--    1 root     root        252274 Dec 18 16:17 vmware-10.log

-rw-r--r--    1 root     root        368085 Jan  7 09:19 vmware-11.log

-rw-r--r--    1 root     root        527608 Feb  4 05:59 vmware-12.log

-rw-r--r--    1 root     root        181255 Feb  4 06:27 vmware-13.log

-rw-r--r--    1 root     root       2860090 Nov 20 11:28 vmware-8.log

-rw-r--r--    1 root     root        256583 Dec 13 00:23 vmware-9.log

-rw-r--r--    1 root     root        181976 Feb  4 06:58 vmware.log

[RaimondsMazulis]

THIS WILL BE CORECT DUMP?

[continuum]

You may need to review your problem description ? - the file-list says that there is a vmsn ???

[RaimondsMazulis]

I created snapshot after I deleted first vmsn

Windows server-Snapshot64.vmsn

[continuum]

Can we consolidate the snapshot-chain ?

Can you please connect to your ESXi with WinSCP and download:

the vmx-file

the vmsd and vmsn-file

and all vmdk-files without "flat" or "delta" in the name ?

[RaimondsMazulis]

There is

[continuum]

"Windows server-Snapshot63.vmsn" is missing. Without that file you will not be able to go back to the state "07012020 1244"

But do you really need to go back to the hot state "07012020 1244" ?

Are the files:

"/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_1.vmdk"

"/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_2.vmdk"

"/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_3.vmdk"

"/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_1-000001.vmdk"

"/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_2-000001.vmdk"

"/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_3-000001.vmdk"

available ?

Please show descriptor-vmdks for the those vmdks.

I think that if you rename the still existing name.vmsn to name.vmsn,org

and also hide or rename the existing vmsd the VM should work.

Well that assumes the other vmdk-descriptorfiles dont have additional issues.

I would recommend to completely get rid of the snapshots by doing a manual consolidation via vmkfstools if that is an option.

[RaimondsMazulis]

[RaimondsMazulis]

This is the only snapshot i have and server was decrypted (all files) by pharma virus.

[continuum · Accepted solution]

I see - then I would suggest to create a new empty directory on a datastore with sufficient free space.

Then use vmkfstools from commandline to consolidate all vmdks one by one.

This script should do that.

I removed the latest snapshot as the deltas apparently have no new data.

vmkfstools -i "/vmfs/volumes/59f5094e-f86e82f9-607e-001e4f24fc12/Windows server/Windows server-000001.vmdk"           "Windows server.vmdk"

vmkfstools -i "/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_1-000001.vmdk"         "Windows server_1.vmdk"

vmkfstools -i "/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_2-000001.vmdk"         "Windows server_2.vmdk"

vmkfstools -i "/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_3-000001.vmdk"         "Windows server_3.vmdk"

Once the vmkfstool commands are done

you can use this vmx-file to run the VM with the consolidated vmdks.

This is riskfree - it will not make it worse if anything fails along the way.

.encoding = "UTF-8"

config.version = "8"

cpuid.coresPerSocket = "2"

disk.EnableUUID = "TRUE"  

displayName = "Windows server"

ethernet0.addressType = "generated"

ethernet0.generatedAddress = "00:0c:29:7b:4f:04"

ethernet0.generatedAddressOffset = "0"

ethernet0.networkName = "VM Network"

ethernet0.pciSlotNumber = "32"

ethernet0.present = "TRUE"

ethernet0.virtualDev = "e1000"

evcCompatibilityMode = "FALSE"

floppy0.present = "FALSE"

guestOS = "windows7srv-64"

hpet0.present = "TRUE"                       

ide0:0.deviceType = "cdrom-image"

ide0:0.fileName = "/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/KNOPPIX_V7.2.0CD-2013-06-16-EN.iso"

ide0:0.present = "TRUE"

ide1:0.present = "FALSE"

memsize = "10240"

numvcpus = "8"

nvram = "Windows server.nvram"

pciBridge0.pciSlotNumber = "17"

pciBridge0.present = "TRUE"

pciBridge4.functions = "8"

pciBridge4.pciSlotNumber = "21"

pciBridge4.present = "TRUE"

pciBridge4.virtualDev = "pcieRootPort"

pciBridge5.functions = "8"

pciBridge5.pciSlotNumber = "22"

pciBridge5.present = "TRUE"

pciBridge5.virtualDev = "pcieRootPort"

pciBridge6.functions = "8"

pciBridge6.pciSlotNumber = "23"

pciBridge6.present = "TRUE"

pciBridge6.virtualDev = "pcieRootPort"

pciBridge7.functions = "8"                       

pciBridge7.pciSlotNumber = "24"

pciBridge7.present = "TRUE"

pciBridge7.virtualDev = "pcieRootPort"

replay.filename = ""      

scsi0.pciSlotNumber = "160"

scsi0.present = "TRUE"

scsi0.sasWWID = "50 05 05 6d 42 12 00 40"

scsi0.sharedBus = "none"

scsi0.virtualDev = "lsisas1068"

scsi0:0.deviceType = "scsi-hardDisk"

scsi0:0.fileName = "Windows server.vmdk"

scsi0:0.present = "TRUE"  

scsi0:1.deviceType = "scsi-hardDisk"

scsi0:1.fileName = "Windows server_1.vmdk"

scsi0:1.present = "TRUE" 

scsi0:2.deviceType = "scsi-hardDisk"

scsi0:2.fileName = "Windows server_2.vmdk"

scsi0:2.present = "TRUE" 

scsi0:3.deviceType = "scsi-hardDisk"

scsi0:3.fileName = "Windows server_3.vmdk"

scsi0:3.present = "TRUE"

svga.vramSize = "8388608" 

virtualHW.productCompatibility = "hosted"

virtualHW.version = "8"

vmci0.id = "-629453052"  

vmci0.pciSlotNumber = "33"

vmci0.present = "TRUE"

[a_p_]

I could be wrong, but from what I understand, the virus hit/encrypted the VM while it was running on Snapshot 1.

If that's the case, it  would be necessary to reset the VM to its base disks from January, 7th, i.e. discarding all snapshots.

André

[RaimondsMazulis]

Thanks, I will try it.

[continuum]

Raimond - Andres question is very important - if all 000001-deltas are infected you must discard all snapshots.

So the question is: when did you hit the virus ?

We have to answer this question BEFORE you run any of the vmkfstools command

[RaimondsMazulis]

I hit the virus on 03.02.2020!

[continuum]

Just saw log-11 !

According to that we have to assume that on 7.1.2020 the config was modified manually - otherwise I dont have an explanation for basedisks with a timestamp from 71.2020.

According to vmware-011.log the VM was running on a snapshot since 18.12.2019

So it is essential to know when the virus hit -it may be possible that the basedisks are affected if the config was changed manually ....

What ever you do - never start this VM in any state with connect network.

If we cant figure out the complete history precisely I would also avoid to ever boot the into its own Windows.

Rather experiment with one datadisk from a Linux-liveCD !!!

If possible I would even suggest to rebuild the Windows from scratch and just use the existing vmdks to extract the essential data - preferably using an isolated LinuxliveCD to do that.

[RaimondsMazulis]

My steps:

1. I will add new datastore (new hard drive), because i have small space available.

2. Copy all datastores files to new datastore (for backup)

3. I will try vmkfstools from commandline to consolidate all vmdks one by one and using vmx-file to run the VM with the consolidated vmdks.

4. If it doesnt work or if these files will be infected, i will create new VM and install it.

5. Using livecd linux on new created VM mount vdmk files and copy necessary files to VM filesystem.

Thats correct steps?

[continuum]

The order of steps makes sense.

But make sure you play it safe ! - remove the nics and dont boot the operating system on disk.

If possible I would like to inspect the results after vmkfstools - so far I have not seen malware like that in action.

I would like to look at such a case just to find out if I can diagnose an infected virtual disk before actually starting a VM at all.

Just curious - so decide yourself if a teamviewer-session is possible / wanted.

Ulli

[RaimondsMazulis]

Hello,

to recreate snapshop from 07.01.2020 I used these commands:

vmkfstools -i "/vmfs/volumes/59f5094e-f86e82f9-607e-001e4f24fc12/Windows server/Windows server.vmdk"           "Windows server.vmdk"

vmkfstools -i "/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_1.vmdk"         "Windows server_1.vmdk"

vmkfstools -i "/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_2.vmdk"         "Windows server_2.vmdk"

vmkfstools -i "/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_3.vmdk"         "Windows server_3.vmdk"

These commands restored VM, but files was crypted:

vmkfstools -i "/vmfs/volumes/59f5094e-f86e82f9-607e-001e4f24fc12/Windows server/Windows server-000001.vmdk"           "Windows server.vmdk"

vmkfstools -i "/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_1-000001.vmdk"         "Windows server_1.vmdk"

vmkfstools -i "/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_2-000001.vmdk"         "Windows server_2.vmdk"

vmkfstools -i "/vmfs/volumes/59f511db-09abd1a1-6abd-001e4f24fc12/Windows server/Windows server_3-000001.vmdk"         "Windows server_3.vmdk"

Thanks continuum for help and for .vmx configuration. continuum Thanks Thanks Thanks....

If you want remote session to see logs or somethink, just ask.

Now I need to figure out wy it happens...