lockstar007
Contributor
Contributor

Recover failover data

Hi Folks,

I have a couiple of VMs that was used as part of the Disaster Recovery Environment. I was using Veeam to do failover from my production to environment.

Everything was working well until we decided to switch back to production. One of our junior admins did a "undo failover" instead of "permanent failover"

This meant that all our data have been lost for up to 1 week. We had the VMs running for 1 week. Is there any way to recover the lost data?

0 Kudos
13 Replies
daphnissov
Immortal
Immortal

No, because this action deletes the delta disk and reverts to the previous snapshot, discarding it immediately. Hopefully you have some backups or replica points since it was in a failover state for a full week.

0 Kudos
lockstar007
Contributor
Contributor

Great. You understood me.

Unfortunately I did not make backup so replica restore points. I know that is the most important thing. but i messed up.

Is there a way to recover the delta disks?

Please help!!

0 Kudos
scott28tt
VMware Employee
VMware Employee

A good place to head to: Veeam Community Forums


-------------------------------------------------------------------------------------------------------------------------------------------------------------

Although I am a VMware employee I contribute to VMware Communities voluntarily (ie. not in any official capacity)
VMware Training & Certification blog
0 Kudos
daphnissov
Immortal
Immortal

Unfortunately, even if you recover the delta VMDKs because this VM has been running for some time, the deltas can no longer be applied to it because those references points are invalid. It would essentially be corrupting all that data anyhow. This seems like a teachable moment all the way around.

0 Kudos
lockstar007
Contributor
Contributor

Thanks for replying guys.

One other thing i noticed while i was poking around in the datastore on the DR site, there are delta disks within the virtual machine folders. The name of the delta disks begin with mailserver-0001-delta.vmdk...,etc.

Is it possible to mount these disks and retrieve the files and folders from 1 week ago?

0 Kudos
daphnissov
Immortal
Immortal

No, those are the replica points that are used for failover purposes. When the undo was done, the necessary delta was deleted from disk. I don't think you have any recourse at this point.

0 Kudos
a_p_
Leadership
Leadership

Although I agree with daphnissov​ that chances to recover data are close to zero, we can take a look at the VM's files that you have in the datastore.

For that, please run ls -lisa > filelist.txt in the VM's folder, and attach the filelist.txt to your next reply.


André

0 Kudos
lockstar007
Contributor
Contributor

Hi Andre

I have attached the file list. I am interested in the XXXXXXXXXX-000001-delta.vmdk file.

Would that be any use for me to recover files from?

0 Kudos
a_p_
Leadership
Leadership

Just to make sure that I get this correctly.

The file list that you've posted, is this the one from the currently powered off replica? If yes, it should be possible to somehow mount the snapshot, if the snapshot chain is still healthy.

Please provide the details about the listed files , and the VM, so that we can figure out the proper steps to mount the virtual disk, i.e. access the contents.


André

0 Kudos
lockstar007
Contributor
Contributor

Thanks for replying A.P. It sounds like we may be able mount the snapshot and recover the files i need from 1 week ago right?

Not sure what details you need. I am posting the logs and descriptor files for the snapshots as well as screenshots.

Let me know should you need more details

0 Kudos
a_p_
Leadership
Leadership

According to the presented files, the VM has been powered on yesterday (either manually, or by a e.g. backup verification job), so that answers my question on whether the files are in use.

If this is the current replica, then it contains the same data as the original VM, and the previous snapshot holds data from the March, 10th, which may not help.

Does the original VM have active snapshots?

André

0 Kudos
lockstar007
Contributor
Contributor

Thanks for checking out the files.

Yes, the original VM does have active snapshots. They are dated for 9th and 10th March.

I have attached the active snapshots.

Two other snapshots were created recently.

0 Kudos
a_p_
Leadership
Leadership

I'm afaid that in this case there's no chance to recover missing data other than from a backup. If I interpret this correcttly the undo has most likely reset the VM to the state on March, 10th.

André

0 Kudos