1 2 3 Previous Next 35 Replies Latest reply on Aug 30, 2018 3:09 PM by continuum

    The best way how to swap two hard drives

    Mufimufin Novice

      Dear all,

       

      first of all I do not have any RAID on my home server and I plan to add it later when I will switch to some other solution (matter of years).

       

      One of the non-system hard drives (datastore2) is showing symptomps of a faulty drive. I have also made a few tests and confirmed that (Hirenboot, Linux commands).

       

      I plan to do the following:

      a) connect another 10 Tb WD Red NAS hard drive to my ESXi 6.7 server (its already connected to my MSI H61MA-E35 (B3) MB, but ESXi refuses to detect it - that is however another problem I need to solve first)

      b) copy the datastore2 content to datastore 3 (the new drive) using the Datastore browser option

      c) rename datastore2 to some other name (e.g. goodbye-my-old-hard-drive)

      d) rename datastore3 (new hard drive) to datastore2

      e) disconnect old failing hard drive

       

      I do not see any vMotion options in my ESXi 6.7 web interface (vSphere) because I do not have any Virtual Center installed.

       

      I therefore believe that I will have to remove the VM from my invetory and re-add it once moved to make ESXi know that it was moved on another hard drive.

       

      Potential important information is that there is another datastorage which is mounted to the same VM as the one on datastore2 and I would like to keep it that way as it contains some data. Does this represent a problem? I think that I will have to re-add it as well once the VM will be removed from the inventory, right?

       

      Thank you.

        • 1. Re: The best way how to swap two hard drives
          a.p. Guru
          User ModeratorsCommunity WarriorsvExpert

          b) The datastore browser may not be the best option for this. I'd go ahead and either export/import as OFV, backup/restore, or copy the VM's files (at least the .vmdk files) manually from the command line using the vmkfstools command line utility. Please be careful if the VM has active snapshot to avoid possible data loss!

          In any case, use the "copy" option rather than "move". In case something goes wrong, you will still have the original files. After copying the files, rename the source folder!

           

          c + e) Please take a look at https://kb.vmware.com/s/article/2004605 for how to properly disconnect a LUN/disk from ESXi.

           

          I therefore believe that I will have to remove the VM from my invetory and re-add it once moved to make ESXi know that it was moved on another hard drive.

          Yes, remove the VMs from the inventory prior to copying the files over to the new datstore, and then register the VMs again from the new location.

          Btw. datastore names are not that important, they are basically just "friendly" names. Internally, ESXi uses the UUIDs instead, so that you can rename datastores without impacts to the VMs. In case you are moving/copying files which are not located in the VM's home folder, you will need to adjust the VM's settings (absolute paths).

           

          André

          • 2. Re: The best way how to swap two hard drives
            Mufimufin Novice
            b) The datastore browser may not be the best option for this

            I have started the migration yesterday by a simple copy of the VM through the data store explorer. Its just 2 Tb HDD, the procedure is running from 2 AM until now = 10,75 h and its only made 43% progress. This seems extremely slow. I am going to stop that. Hopefuly export/import or vdmk option will be faster than this.

             

            I'd go ahead and either export/import as OFV, backup/restore, or copy the VM's files (at least the .vmdk files) manually from the command line using the vmkfstools command line utility. Please be careful if the VM has active snapshot to avoid possible data loss!

            Export OVF option in the web interface seems to download files on my computer and I do not have enough space for such large files at the moment. It also ended on the network error a few seconds after the start. This is therefore not an option.

             

            I have not found any other backup option in the web vSphere interface. Did you meant Veeam backup solution (free option)? This does not seems to work with free ESXi tier that I use. Perhaps I should use Trilead's VM Explorer?


            Anyway using vmkfstools command line method seems to be also extremely slow, but at least its working. I am waiting until its finished and will copy the remaining portion of files as described in this thread Move VMware ESXi VM to new datastore - preserve thin-provisioning - Server Fault  .  Not sure why it has to be so cumbersome and why ESXi web interface does not offer anything more user friendly.

             

            In any case, use the "copy" option rather than "move". In case something goes wrong, you will still have the original files. After copying the files, rename the source folder!

            That is what I did given to the fact that the original hard drive might be faulty.

             

            c + e) Please take a look at https://kb.vmware.com/s/article/2004605 for how to properly disconnect a LUN/disk from ESXi.

            I will use the unmount option in the ESXi vSphere web interface and the physically detach the hard drive once finished.

             

            Yes, remove the VMs from the inventory prior to copying the files over to the new datstore, and then register the VMs again from the new location.

            Its unregistered and I will re-register once moved.

             

            Thank you.

            • 3. Re: The best way how to swap two hard drives
              Mufimufin Novice

              I have moved the VM using vmkfstools and registered through the data store browser. It appeared in the web Vsphere interface.

               

              The problem I face now is that it does not start (Linux) after the power-on. ESXi shows green dot which means that there are most likely no obvious issues on the virtualization part.

               

              By does not start I mean that its stucked on the OS logo being shown after I choose that it should boot normally.

               

              Here is the vmware.log output:

              https://hastebin.com/anoqalenid.makefile

               

              Here is the vmkernel.log output:

              https://hastebin.com/wocipafiga.sql

               

              When I choose the recovery boot option in GRUB the output looks like this:

               

              2018-08-26_18-55-17.png

               

               

              I do not really know what I should do now.

              • 4. Re: The best way how to swap two hard drives
                a.p. Guru
                Community WarriorsUser ModeratorsvExpert

                I have moved the VM using vmkfstools and registered through the data store browser.

                The vmkfstools utility only copies the virtual disk file(s). Did you also copy the other files (e.g. the .vmx,...) over to a new folder on the target datastore, and rename the old folder prior to registering the VM from the target location, and powering it on?

                 

                André

                • 5. Re: The best way how to swap two hard drives
                  Mufimufin Novice

                  The vmkfstools utility only copies the virtual disk file(s). Did you also copy the other files (e.g. the .vmx,...) over to a new folder on the target datastore

                  Thanks for the answer. Yes, I did using the following command:

                   

                  • find "/vmfs/volumes/source_datastore/Some VM" -maxdepth 1 -type f | grep -v ".vmdk" | while read file; do cp "$file" "/vmfs/volumes/destination_datastore/Some VM"; done

                   

                  and then this one:

                   

                  • find "/vmfs/volumes/source_datastore/Some VM" -maxdepth 1 -type f | grep [0123456789][0123456789][0123456789][0123456789][0123456789][0123456789] | grep ".vmdk" | while read file; do cp "$file" "/vmfs/volumes/destination_datastore/Some VM"; done

                   

                  After that the VM got registered. I had another problem with the file lock but that was caused by vmkfstools clone process being stucked on 100% in another putty instance and I had to kill it. The data size however matches the cloned file (bytes), so it was moved as whole 1,62 Tb file.

                   

                  and rename the old folder prior to registering the VM from the target location, and powering it on?

                  No, I have not renamed the old VM folder as it is on another data storage (datastore2 - OLD instead of datastore3 - NEW).

                   

                  I have did that now:

                   

                  mv /home/user/oldname /home/user/newname

                   

                   

                  and tried to re-register the VM on my new data store without copying the files again. Unfortunately the problem still persists.

                   

                  Was it screwed up just because I have not renamed the folder and should I copy the files again?

                  • 6. Re: The best way how to swap two hard drives
                    Mufimufin Novice

                    Here is a comparison of the old folder and the new folder.

                     

                    2018-08-26_19-24-53.png

                     

                    2018-08-26_19-24-14.png

                    Some files have different data size, so perhaps I can try to owerwrite them manually?

                     

                    Also I have clicked on "I moved the VM" and not on "I have made a copy of the VM".

                    • 7. Re: The best way how to swap two hard drives
                      Mufimufin Novice

                      I have deleted all log files and tried to move all files except 2 vmdk files again (using Winscp) by owerwriting the old files. ESXi asked me whether I have moved or made a copy so this time I choosed the copy option as my answer. When I power up the VM it still does not work.

                       

                      I am out of ideas and thinking that I might try another option like Trilead's VM Explorer and start over again.

                      • 8. Re: The best way how to swap two hard drives
                        a.p. Guru
                        User ModeratorsCommunity WarriorsvExpert

                        From what you write, I think it's likely an issue with the copied virtual disk files.

                        The "cp" command may work, but is not what you want to use for large files on VMFS datastores.

                        "vmkfstools" is build to copy virtual disk files (including metadata, and user data) in an optimized way. The target file will be created up front with it's provisioned size, and then the contents will be copied over block by block to avoid the need to resize the file (as cp does it), i.e. modify VMFS metadata. So even if the file sizes matche, the content may not be consistent if vmkfstools was killed.

                         

                        André

                        • 9. Re: The best way how to swap two hard drives
                          Mufimufin Novice

                          I have compared the content of the smaller vdmk file (Flawless Server.vmdk) and noticed that there is an additional row in the new one.

                          ddb.thinProvisioned = "1"

                           

                          The previous vmdk file did not had such row. Its most likely caused by the command that I have used for the initial file copy:

                          • vmkfstools -i "/vmfs/volumes/source_datastore/Some VM/Some VM.vmdk" -d thin "/vmfs/volumes/destination_datastore/Some VM/Some VM.vmdk"

                           

                          Should I try to delete it or move files again without the thin parameter?

                          • 10. Re: The best way how to swap two hard drives
                            Mufimufin Novice

                            The "cp" command may work, but is not what you want to use for large files on VMFS datastores.

                            Ok, so should I use another command instead?

                             

                            "vmkfstools" is build to copy virtual disk files (including metadata, and user data) in an optimized way. The target file will be created up front with it's provisioned size, and then the contents will be copied over block by block to avoid the need to resize the file (as cp does it), i.e. modify VMFS metadata. So even if the file sizes matche, the content may not be consistent if vmkfstools was killed.

                            Understood, but what should I do if its stucked on cloning 100% done for an hour? I can do the vmkfstools copy round no. 2.

                             

                            Do you recommend this step by step guide Revisions to Move VMware ESXi VM to new datastore - preserve thin-provisioning - Server Fault  or I should rather follow something else?

                             

                            Thank you a lot for your ongoing support. Its greately appreciated.

                            • 11. Re: The best way how to swap two hard drives
                              a.p. Guru
                              vExpertUser ModeratorsCommunity Warriors

                              Which target format (thin, or thick) do you want/need?

                              In case you want to clone the virtual disk as is, omit "-d thin" from the command line.

                              There are a couple of ways to find out whether a virtual disk has been thin, or thick provisioned. An easy way is to run ls -lisa, and compare the provisioned size (the column left to the time stamp) with the real size in kB (2nd column). If the sizes match, the virtual disk has most likely been thick provisioned.

                              Understood, but what should I do if its stucked on cloning 100% done for an hour? I can do the vmkfstools copy round no. 2.

                              I'm not 100% sure about this, but I'd give it a second chance. Maybe it's a result of converting a thick disk to a thin disk!?

                               

                              André

                              • 12. Re: The best way how to swap two hard drives
                                Mufimufin Novice
                                Which target format (thin, or thick) do you want/need?
                                In case you want to clone the virtual disk as is, omit "-d thin" from the command line.

                                I would rather stick with the original format. It looks that I have added thin just because it was in the step by step guide. I will therefore omit "-d thin" as mentioned above.

                                 

                                There are a couple of ways to find out whether a virtual disk has been thin, or thick provisioned. An easy way is to run ls -lisa, and compare the provisioned size (the column left to the time stamp) with the real size in kB (2nd column). If the sizes match, the virtual disk has most likely been thick provisioned.

                                Here is the output of ls -lisa (original VM folder):

                                2018-08-26_20-09-50.png

                                It looks that the size does not match and it therefore was most likely using the thin format of provisioning.

                                I'm not 100% sure about this, but I'd give it a second chance. Maybe it's a result of converting a thick disk to a thin disk!?

                                I will give it one more chance and if that will not work I will most likely try that VM Explorer with a hope that it will allow me to move from one datastore to another without getting my local Windows computer involved in terms of its hard drive.

                                • 13. Re: The best way how to swap two hard drives
                                  a.p. Guru
                                  Community WarriorsvExpertUser Moderators

                                  It looks that the size does not match ...

                                  Column 2 is in kB, whereas the provisioned size is in Bytes. Both values show a size of 1,853 GB. So to me it looks like a thick provisioned virtual disk.

                                   

                                  André

                                  • 14. Re: The best way how to swap two hard drives
                                    Mufimufin Novice

                                    It seems to be stucked at 100% again.

                                     

                                    I am not sure if it actually still does something or not.

                                     

                                     

                                    It might still be running as there is nothing else on that new hard drive at the moment.

                                     

                                    Unless I cancel this the ESXi web vSphjere VM power-up will complain about the locked file. There are no *.lck files, just this hanging copy process in its 100% stage.

                                     

                                    Can this be caused by differences in the data store VMFS versions (5.58 for the old data store and 6.82 for the new one)? I will rather wait for some more time.

                                     

                                    Edit: 11:52 (4 more hours) = still running. At least properly working progress would be nice :-/. Its running for about 10,5 hours now (1,82 Tb).

                                    1 2 3 Previous Next