Moving a VM to a new datastore on same host - PRESERVING SNAPSHOTS

[radman]

Hi folks,

I see a lot of threads about how to migrate VMs to a new datastore on the local host, but they always say to remove the snapshots first. I want to migrate to larger storage and that means I need to move all my VMs to a new datastore, but I don't want to lose my valuable snapshots. Is there any way to achieve this?

Should I perhaps backup the old datastore and then restore it afterwards on the new storage? That might be easiest given my large number of VMs.

Thanks,

Bob

[RParker]

have you tried migrating 1 VM w/ a Snapshot to see if it will work?

[radman]

Yes, I get "Migration of virtual machines with snapshots is not supported between the source and destination".

Is there something I can do so that migration with snapshots will work in my case (same host, different datastores)?

[kitcolbert]

Nope. There's no way to do it through the UI.

As the error message states, migrating a VM with snapshots is not currently supported. There are ways to do it by hand, manually copying files and possible editing files with a text editor, but of course this is also completely unsupported.

[radman]

I thought I'd read that copying VMs around with Linux tools resulted in disk fragmentation and resultant performance degradation?

I could understand that there are challenges with VMotion between different platforms, but on a single system it seems like it should be a no-brainer...

[kitcolbert · Accepted solution]

In general, I don't believe using 'cp' and friends should create disk fragmentation or performance degradation. Where did you hear that?

Anyway, I completely agree with you that, yes, it should be fairly simple to migrate VMs between datastores (especially powered off VMs!). In general, VC can, of course, migrate VMs that don't have snapshots without any problems. It's just snapshots that pose a problem for VC.

But we are looking at fixing this. As a first step, we'd like to support relocation of VMs with snapshots where all VM files are contained within the same directory (i.e. all disks and snapshots, etc, are all in the same location). Eventually we'd like to support relocation of VMs with snapshots regardless of where the VM files resides (e.g. you could have disks in different datastores).

[radman]

In general, I don't believe using 'cp' and friends should create disk fragmentation or performance degradation. Where did you hear that?

On an archived thread - sorry I no longer have a pointer. I should try this then if there are no supported alternatives. What about backup/restore, could I do it that way?

Anyway, I completely agree with you that, yes, it should be fairly simple to migrate VMs between datastores (especially powered off VMs!). In general, VC can, of course, migrate VMs that don't have snapshots without any problems. It's just snapshots that pose a problem for VC.

But we are looking at fixing this. As a first step, we'd like to support relocation of VMs with snapshots where all VM files are contained within the same directory (i.e. all disks and snapshots, etc, are all in the same location). Eventually we'd like to support relocation of VMs with snapshots regardless of where the VM files resides (e.g. you could have disks in different datastores).<div>

This is exactly my situation - powered-off VMs, all VM files within single directories. I want to migrate the VMs to a new datastore so that I can decommision the old one, but don't want to lose my snapshots...

If you need a guinea pig to test something like this out, just let me know

</div>

[radman]

Just a quick follow-up to say that good old "cp -rp" seems to have worked just fine to copy over my VMs and everything seems to be working. Thanks!

[kitcolbert]

That's great to hear!

So I checked with a VMFS engineer and you are correct that cp can cause fragmentation. But in general, due to the very large block sizes VMFS uses, this fragmentation shouldn't lead to much, if any, of a performance hit. However, the bigger problem is that cp causes 2x+ writes and twice the SCSI reservations compared to using vmkfstools -i. Thus in general, we do recommend vmkfstools -i.

For this one-off type thing though, cp -r is fine.

[radman]

Well, now I have terrible news.

Everything seemed to be working great for several days. Then I decided it was time to remove the old original ESX/datastore disks, which were RAID striped, and replace them with disks I could use for data storage. Unfortunately, to put new disks in I had to deconfigure the old RAID IS array. Upon reboot I found none of my machines would start up. I get errors that appear to be saying that some of the snapshots refer to the old datastore. And, I don't think there's a way to recreate the original RAID IS array, although I have the original disks. I haven't found one, anyway (anybody have helpful advice here?). So it's looking like I may have to scratch my ESX server and start recreating my VMs all over again :_|

[kitcolbert]

Ouch, that sucks.

I don't know anything about recreating the RAID array, but we can probably salvage the VMs. However, unless you recreate the array, you'll have lost any changes that occurred since migration. If you're interested in doing that, let me know and I can try to put together a list of steps for you.

[radman]

Ouch, that sucks.

I don't know anything about recreating the RAID array, but we can probably salvage the VMs. However, unless you recreate the array, you'll have lost any changes that occurred since migration. If you're interested in doing that, let me know and I can try to put together a list of steps for you.<div>

Are you kidding me? That would be great! I'm perplexed as to why on earth it thinks I have snapshots referencing the old datastore anyway. All my disks/snapshots etc for each VM are in a single directory, and I copied all of that. It's possible that some snapshots had CDROM drives which were connected to ISOs on the old datastore, although that no longer mattered, because that's how I installed a lot of software in the VMs. I may have inadvertently left the CDROM connected after the install. But I wouldn't have thought that could be the case for all my VMs. I had about 9-12 VMs which had multiple snapshots of different versions of software that I'd go back to to experiment with (I'm a developer). Solaris, Linux, XP, and W2K3. I've already started recreating them from scratch, which will take a long time as you can imagine. Any tips gratefully accepted.

</div>

[kitcolbert]

Sorry it's taken me awhile to get back to you.

So I believe the basic problem is that you have absolute paths in your snapshot tree. These absolute paths point back to the original datastore for the VM (i.e. its pre-relocation datastore). Thus even though you copied everything, the VM was actually trying to use some of the old files. So unbeknownst to you, some files on the original datastore were being used and possibly modified. Thus the basic idea of the workaround is to remove all those absolute paths so that the VM will look in its current directory for the snapshots and disks.

Here's the workaround. Please let me know if anything's unclear or if you're confused about something. It's definitely better to ask me first than try anything and make a mistake. Be sure to back up your VM. As I mentioned before, most likely you've lost any changes that happened after the migration (and that's irrespective of this workaround). And again, this type of migration is not officially supported by VMware, so if you bork your VM, you're on your own (I'll try to help where I can, but I'm very limited in the free time I have to spend on the forums).

Thanks,

Kit

-

---

*** THIS IS AN UNSUPPORTED OPERATION. USE AT YOUR OWN RISK. ***

This operation is only applicable to VMs that have all their files (config file, disks, snapshots, etc) in a single directory on a single datastore.

For each VM you would like to migrate, do the following:

(When editing any file listed below, please perform the edit from the Service Console using either vi or nano.)

*** THIS IS AN UNSUPPORTED OPERATION. USE AT YOUR OWN RISK. ***

0. **Back up the VM!**

1. Power off the VM and unregister it.

2. Check all occurrences of scsiX:Y.fileName in the VM's config file and ensure that either it is a relative path or an absolute path that points to the VM's old directory.

3. For each .vmdk file whose filename has "-00000X" in it but does not have "-delta", edit the file (these .vmdk's are actually just text files) and ensure that the "parentFileNameHint" field is a relative path or an absolute path that points to the VM's old directory.

4. If either step 2 or 3 results in an absolute path that points outside the current directory (i.e. the new datastore) or the VM's old directory, then the method below is not applicable.

5. Edit the VM's config file and remove the path component of all scsiX:Y.fileName's that have an absolute path. E.g. 'scsi0:0.fileName = "/vmfs/volumes/45e73365-118af320-01ad-01143e7bc790/foo/foo.vmdk"' would be changed to 'scsi0:0.fileName = "foo.vmdk"'.

6. For each .vmdk file whose filename has "-00000X" in it but does not have "-delta", edit the file (again, this .vmdk is just a text file) and remove the path component of all "parentFileNameHint" fields that have an absolute path. E.g. 'parentFileNameHint="/vmfs/volumes/4637121e-de27635f-25b2-00145f7be790/foo/foo-000006.vmdk"' would be changed to 'parentFileNameHint="foo-000006.vmdk"'.

7. Double check that all absolute paths have been converted to relative paths in the .vmdk and .vmx files. Note: sched.swap.derivedName can be left as-is. It will not affect the VM.

8. Register the VM with the host through either vmware-cmd or the datastore browser in VC.

9. Power-on the VM. Answer the question about whether to keep or create a UUID (presumably you'd like to keep the UUID).

10. The VM should power-on successfully and you should be able to use it freely, revert to previous snapshots, etc.

*** THIS IS AN UNSUPPORTED OPERATION. USE AT YOUR OWN RISK. ***