13 Replies Latest reply on Nov 5, 2014 4:24 AM by swinster

    Migrating VMs from one datastore to another on ESXi 5.1 (free license)

    telecastle Novice

      I am running ESXi in my lab (for Cisco voice servers and some Windows VMs). The VMs are on an iSCSI datastore hosted by a Netgear ReadyNAS Pro Business. Due to major issues with iSCSI on the ReadyNAS platform and the inability by Netgear to troubleshoot the issues (NAS lockup requiring hard reset), I decided to purchase a QNAP TS-569L and use it to host my ESXi datastore. I'm now trying to migrate my VMs from the iSCSI datastore hosted on the ReadyNAS to the iSCSI datastore hosted by the QNAP.

       

      My VMs are thin-provisioned, and I would like to preserve thin provisioning after the migration. The ESXi datastore browser can move (or copy) VMs from one datastore to another, but thin-provisioned VMs become thick-provisioned after they have been moved (or copied). Someone advised Veeam for this purpose, which I installed and configured. Unfortunately, Veeam errors out with the message that the current license does not allow the migration of the VMs from one datastore to another. Just today I found out that one must have at least the ESXi Essentials license ($500) to allow this type of migration by Veeam.

       

      I've also tried OVF tool (the MMware command line utility), using the following syntax:

      ./ovftool -ds QNAP-iSCSI -dm=thin vi://root@192.168.200.10/CUCM-Pub vi://root@192.168.200.10

       

      where:

      QNAP-iSCSI is the name of the datastore on the QNAP

      192.168.200.10 is the IP of my ESXi box that is connected to both iSCSI datastores (one hosted by the ReadyNAS and the other by the QNAP).

      CUCM-Pub is the name of the VM on the ReadyNAS iSCSI datastore that I"m trying to migrate to the QNAP iSCSI datastore.

       

      However, I keep getting the following error message:

      Error: Unexpected option: vi://root@192.168.200.10/CUCM-Pub

       

      This is probably due to the same issue - the free ESXi license does not allow this type VM migration. So, what are my options with the free ESXi license? I am not prepared to pay $500 for the Essentials license at this point.

       

      Thank you!

        • 1. Re: Migrating VMs from one datastore to another on ESXi 5.1 (free license)
          a.p. Guru
          Community WarriorsUser ModeratorsvExpert

          A couple of options I can think of are:

          • export to OVF and re-import
          • use Trilead's VM Explorer and backup/restore the VMs
          • use the vmkfstools command to copy the virtual disks

           

          André

          • 2. Re: Migrating VMs from one datastore to another on ESXi 5.1 (free license)
            telecastle Novice

            Thank you for your reply with the options.

             

            I like the vmkfstools option, and I've just tried it like this:

             

            /vmfs/volumes # vmkfstools -i "ReadyNAS-iSCSI/CUCM-Pub 9.x/CUCM-Pub 9.x.vmdk" "QNAP/CUCM-Pub9.x/CUCM-Pub 9.x.vmdk" -d thin

             

            Unfortunately, I received the following error:

            Destination disk format: VMFS thin-provisioned

            Failed to clone object parameters


            Is there a way to get the vmkfstools to clone a virtual disk to a thin-provisioned VMFS datastore?


            Thank you!

            • 3. Re: Migrating VMs from one datastore to another on ESXi 5.1 (free license)
              a.p. Guru
              vExpertUser ModeratorsCommunity Warriors

              Just to rule this out. Can you confirm the destination folder already exists?

               

              André

              • 4. Re: Migrating VMs from one datastore to another on ESXi 5.1 (free license)
                telecastle Novice

                This was a good catch. The destination path was misspelled.

                 

                The virtual drive is now being cloned. What do I do with the rest of the files? Just copy the with the datastore browser in vSphere Client?

                 

                Thanks!

                • 5. Re: Migrating VMs from one datastore to another on ESXi 5.1 (free license)
                  a.p. Guru
                  Community WarriorsUser ModeratorsvExpert

                  Yes, remove the VM from the inventory, copy the other files over to the new folder (using the cp command or e.g. WinSCP), add the copied VM to the inventory and select "I moved it" when you power on the VM for the first time (to preserve the VM's UUID and MAC address).

                  I assume there are no entries with absolute paths in the VM's .vmx file and you don't have active snapshots on the VMs.

                   

                  André

                  • 6. Re: Migrating VMs from one datastore to another on ESXi 5.1 (free license)
                    telecastle Novice

                    Actually, I do have snapshots. I have the following files in the source folder:

                    CUCM-Pub 9.x-000001.vmdk

                    CUCM-Pub 9.x-000002.vmdk

                    CUCM-Pub 9.x-000003.vmdk

                    CUCM-Pub 9.x-000004.vmdk

                     

                    CUCM-Pub 9.x-Snapshot1.vmsn

                    CUCM-Pub 9.x-Snapshot2.vmsn

                    CUCM-Pub 9.x-Snapshot3.vmsn

                    CUCM-Pub 9.x-Snapshot4.vmsn

                     

                    First off, I'm not sure why there are "-00000X.vmdk" and "SnapshotX.vmsn" files in the source folder. I think they are delta files and snapshots?I thought I would have one .vmdk file and a bunch of snapshots, which are deltas between the initial .vmdk file and all the changes that happened till the point the snapshot was taken. So, I think I understand the presence of "-SnapshotX.vmsn" files, but I do not understand why I have a discrete "-00000X.vmdk" file for every snapshot.

                     

                    Second, in order for me to clone the entire VM to the QNAP iSCSI datastore, do I need to clone every one of these files? I guess I have a hard time understanding this concept. Can I run any of these .vmdk disks as the actual VM's disk without having other vmdk files in the same VM folder?

                    • 7. Re: Migrating VMs from one datastore to another on ESXi 5.1 (free license)
                      a.p. Guru
                      User ModeratorsvExpertCommunity Warriors

                      The .vmsn files contain metadata and - in case the snapshot was created with the VM powered on - the VM's memory, which is only important if you want to revert to a snapshot. The important data is what's in each of the .vmdk files. Since manually migrating VM's with snapshots require some additional steps, let me know whether you need the snapshots at all? If you don't need to revert to a previous state, I'd recommend you use the Snapshot Manager and run "Delete All" to get rid of the snapshots prior to copying them to the new destination.

                       

                      André

                      • 8. Re: Migrating VMs from one datastore to another on ESXi 5.1 (free license)
                        telecastle Novice

                        For most of my VMs, I just need the latest state they are in. So, should I delete all the snapshots for them in the Snapshot Manager?

                         

                        For three of the VMs, I need all the snapshots. Do I clone every "-00000x.vmdk" drive the new datastore? Do I also clone all the "-SnapshotX.vmsn" files to the new datastore?

                         

                        Thank you!

                        • 9. Re: Migrating VMs from one datastore to another on ESXi 5.1 (free license)
                          a.p. Guru
                          Community WarriorsUser ModeratorsvExpert

                          Yes, for the VM's which don't require any snapshots, run "Delete All" from the snapshot manager to consolidate all snapshots.

                          For the VM's which require snapshots, you can only use vmkfstools for the base virtual disk. You cannot use vmkfstools to copy/clone snapshot .vmdk file without consolidating them. I can't tell you for sure whether the following procedure will work, but I think it should:

                          • remove the VM from the host's inventory
                          • clone the base virtual disk using vmkfstools
                          • use WinSCP and copy all files except for the flat.vmdk to the destination (overwriting the base disk's descriptor .vmdk file to maintain its original CID)
                          • rename the source folder (as a precaution)
                          • add the copied VM to the host's inventory

                          Since the .vmsn files contain metadata which may include the source datastore path and name, you may see this datastore name in the VM's Summary and I'm also not sure whether reverting to a snapshot will work, so you need to test this with one of the VMs.

                           

                          André

                          • 10. Re: Migrating VMs from one datastore to another on ESXi 5.1 (free license)
                            telecastle Novice

                            Below is the process I used to migrate all of my VMs from one iSCSI datastore to another iSCSI datastore. Both datastores were connected to the same ESXi host (ESXi 5.1.0). I also used this same method to migrate VMs from the datastore located on the local drive to an iSCSI datastore on ESXi 5.1.0

                             

                            1. In vSphere Client, properly power down the VM you are going to migrate to a new datastore.

                             

                            2. Create a directory in the target datastore for the migrated VM (either in vSphere Client’s GUI or via SSH using the mkdir command).

                             

                            3. In order to migrate a VM from a source datastore to the target datastore (with both datastores attached to the same ESXi host), use the following command:

                             

                            vmkfstools -i [<path_to_VM_directory_in_source_datastore>]/<source_VM.vmdk> [<path_to_VM_directory_in_target_datastore>]/<destination_VM.vmdk> -d thin

                             

                            Note 1: if the source VM has had snapshots made and was running off a snapshot (which can be seen by looking inside the <VM_name>.vmx file), use the following name for the source VM’s .vmdk file:

                            <VM_name>-00000X.vmdk, where X is the number designated to the snapshot used for running this VM. Even though the file size shown for this file in the source datastore is small, the vmkfstools command will consolidate all snapshots up to the snapshot specified for the source .vmdk file in this command and will properly migrate the VM to the new datastore.

                            Note 2: The -d thin parameter allows the VM to be migrated as thin-provisioned if it was in fact thin-provisioned when created. Therefore, if you don't want the migration process to expand the disk size of this VM to the "thick" size specified when the VM was created, use this parameter. If your VM was already thick-provisioned, I don't know what this parameter will do - probably it will do nothing.

                             

                            4. Once the migration process has completed, copy the following files from the old datastore to the new datastore:

                            <VM_name>.vmx

                            <VM_name>.vmxf

                            <VM_name>.vmsd

                            <VM_name>-XY.log (if you want to keep the logs)

                             

                            5. Edit the <VM_name>.vmx file in target VM directory using the vi command (or by downloading the <VM_name>.vmx file to your local Windows machine via vSphere, editing it with Notepad and uploading it to the same location in the target VM directory) and change the name of scsi0:0.filename = from the name of the snapshot it was running off (which is the name you used for the source .vmdk file in the vmkfstools -i command) to the name you assigned to this .vmdk file in the target datastore location (using the same vmkfstools -i command). Don’t forget to save the <VM_name>.vmx file in the target VM datastore folder.

                             

                            6. In vSphere Client, remove the old instance of this VM from the host inventory (without deleting the files).

                             

                            7. In vSphere Client, add the new instance of this VM (located in the target datastore) to the inventory on this host by navigating to the target datastore in vSphere Client, finding the VM’s directory, right clicking on the <VM_name>.vmx file you have just edited, and selecting Add to Inventory.

                             

                            8. In vSphere Client, start this VM. The start process will halt at 95%. You will have to right click on the name of this VM in the inventory, navigate to Guest > Answer Question, and choose Moved to answer the question about  whether you have copied this VM or moved it.

                             

                            9. Start the VM again in vSphere Client. The VM should launch without any issues.

                            1 person found this helpful
                            • 11. Re: Migrating VMs from one datastore to another on ESXi 5.1 (free license)
                              twardlow Lurker

                              Hi Telecastle. Awesome write up. I followed your instructions step by step, but it's still converting my thin disk to thick. Any ideas? Thanks in advance.

                              • 12. Re: Migrating VMs from one datastore to another on ESXi 5.1 (free license)
                                jfh777 Novice

                                Thanks to all for the leads in this post.  I was able to accomplish everything through the vSphere Client GUI.

                                • Mount your new storage alongside your old storage.
                                • Browse to where the VM exists and note the folder name of where it lives.
                                • Create that same folder in your new storage.
                                • Go back to the folder where the VM exists.
                                • Select all files that pertain to the VM, right click, and choose "Move To..."
                                • You will be asked to confirm your selection.  Choose Yes.
                                • Now select the destination data store and the appropriate folder that you have just created.
                                • All the files are moved over to the new store.
                                • Remove the VM from inventory.
                                • Browse to the new datastore and to the new folder.
                                • Right click the VMX file of the VM and add it to inventory.
                                • Power on the VM.
                                • Choose the "Moved" option when asked.

                                 

                                Worked for me on several VMs when moving some dev work from local storage on an old host to SAN storage connected to a newer host.  Carried over snapshots, etc.  I have not tried to delete any snapshots, but the move worked and the systems boot normally.

                                 

                                Regards,

                                Justin

                                • 13. Re: Migrating VMs from one datastore to another on ESXi 5.1 (free license)
                                  swinster Enthusiast

                                  The Datastore browser is great for this functionality, but why is it so sloooooowwwwww?