1 2 3 4 Previous Next 81 Replies Latest reply on Jan 29, 2010 3:27 AM by anonimous Go to original post
      • 15. Re: Moving VMs from one ESXi server to another - what a pain!!!
        ShahidSheikh Enthusiast
        glim wrote:

         

        Would anyone be interested in an rsync binary that can be used on esx3i?

         

         

        It may not be faster, but it is restartable.

         

         

        Yeah I would like to try it out. Which binary are you using? I.e. from which distribution?

         

        I was thinking of trying out binaries from BusyBox to see if I can get any of them to work in the console.

        • 16. Re: Moving VMs from one ESXi server to another - what a pain!!!
          ShahidSheikh Enthusiast
          kpc wrote:

          I export to a local datastore purely to speed up the whole process and it's only temp.  orginally I mounted an nfs share using the VI client and exported to that - jesus was it slow!!!  I tried loads of different settings in the nfs config but none increased the speed.  And yes I meant 20mbps bits not bytes   This is the way I do my backups. Export locally, SCP that pulls the files off the server to a NFS mount on the serverthat is running the backup script.  However 120GB is a large VM, only have one like it running on ESXi thankfully

           

          Very interesting. So it may be writing to the mounted NFS volume that was slow. That was exactly how I imported my VMs from my Ubuntu boxes. Mounted stores from the Ubuntu/VMware Server 1.0.7 boxes using NFS on my ESXi boxes and ran vmkfstools -i to import vmdks into the localstore. And that import was fairly fast. Didn't think about looking at the speed at that time. My source vmdks were thin.

          • 17. Re: Moving VMs from one ESXi server to another - what a pain!!!
            glim Novice

             

            Ok. I'll get it and post it for you.

             

             

            It's not from a specific distro, though it was compiled on Slackware-11.0, though I doubt that specific version matters.

             

             

            It is a completely standard rsync-3.0.3 built from source.

             

             

            The only things you need to do to build it yourself if you don't want to trust a binary from some random person on the net:

             

             

            1. Build statically. Your libraries are not going to be around on the target ESX box.

             

             

            2. Build without TLS support. The mgmt kernel doesn't do TLS, so your binary cannot either.

             

             

            3. Optional: strip the binary when done.

             

             

            To use this:

             

             

            1. All of this is unsupported. Take responsibility for your own actions.

             

             

            2. Enable unsupported sshd to your ESXi system.

             

             

            3. Place this 'rsync-static-stripped' binary into /bin or somewhere else in your path on your ESXi system.

             

             

            4. You will probably want to rename this binary to just "rsync", so that you don't need to specify it on the commandline.

             

             

            Note that you should probably not try to rsync any files that are currently in use. 

             

             

            There is no warranty and no guarantees of any kind.

             

             

             

             

             

            • 18. Re: Moving VMs from one ESXi server to another - what a pain!!!
              ShahidSheikh Enthusiast

              Got it. Many thanks. Will try compiling it myself too.

              • 19. Re: Moving VMs from one ESXi server to another - what a pain!!!
                kpc Enthusiast

                Interesting Glin, have you tried a new build of NFS in the same way as rsync?  May be able to get past the slow export to NFS share.

                • 20. Re: Moving VMs from one ESXi server to another - what a pain!!!
                  ShahidSheikh Enthusiast

                  kpc, what NFS device are you using? If there was one thing I was impressed with in ESXi it was NFS support and speed. What kind of speeds are you seeing?

                  • 21. Re: Moving VMs from one ESXi server to another - what a pain!!!
                    kpc Enthusiast

                     

                    Hi Shahid

                     

                     

                    I was seeing real slow write speeds to my NFS share, however I've been doing some tests and seems that if I export the share with the switch 'async' I get decent speeds - it was 'sync' before.  Funny how this doesn't affect ESX. 

                     

                     

                    • 22. Re: Moving VMs from one ESXi server to another - what a pain!!!
                      ssapp80 Novice

                       

                      when I drop your compiled rsync in esxi and try to run it i get "permission denied"..........any particular method in which i need to place it there?

                       

                       

                      • 23. Re: Moving VMs from one ESXi server to another - what a pain!!!
                        ShahidSheikh Enthusiast
                        kpc wrote:

                         

                        Funny how this doesn't affect ESX. 

                         

                         

                        Hi kpc

                         

                        I'm assuming that you are referring to ESX/ESXi reading/writing to NFS Vs tools like vmkfstools through an SSH to the ESXi reading/writing to NFS and not a comparison between ESX and ESXi.

                         

                        I have not sniffed traffic between ESXi and NFS server to see what the difference is between the vmkernel accessing data from NFS and the linux OS in the management console accessing data. I suspect the internal caching built-in to ESX mask a lot of the issues introduced by high latency as a result of using sync. And I suspect the NFS volume is not mounted in two separate places (vmkernel and management console.) I suspect the vmkernel just exposes its own NFS mounted volume to the management console through the /vmfs/volumes mount. Don't have an ESXi box in front of me right now to verify.

                         

                        You know, I have always exported NFS in async mode. Never had a problem. But then I've never really experienced a bad power failure either. With all the cache built-in to RAID controllers these days, it would be interesting to find out how badly an async exported NFS volumes running on a server with good RAID controller experiences data loss/corruption when the power dies. Of course that would depend on the write caching policy of the RAID controller as well.

                         

                        Sourceforge documentation on NFS and its performance tuning is very good place to start. Really well written. http://nfs.sourceforge.net/nfs-howto/ar01s05.html

                        • 24. Re: Moving VMs from one ESXi server to another - what a pain!!!
                          BThunderW Novice

                          Did you chmod +x?

                          ssapp80 wrote:

                          when I drop your compiled rsync in esxi and try to run it i get "permission denied"..........any particular method in which i need to place it there?

                           

                          • 25. Re: Moving VMs from one ESXi server to another - what a pain!!!
                            kpc Enthusiast

                            Thanks for the helpful insight Shahid.  Thinking back you setup an NFS share on ESXi and ESX totally differently, or well I did.  I'm just glad I'm getting decent speeds now

                            • 26. Re: Moving VMs from one ESXi server to another - what a pain!!!
                              glim Novice

                              NFS is kind of a kernel space thing. In a Linux-like OS, NFS is now commonly done in kernelspace for client and server, and the utilities only serve to assist/configure the kernel.

                               

                              Altering whatever NFS that ESX does would require rebuilding their kernel.

                               

                               

                               

                               

                               

                               

                               

                               

                               

                              There does exist a very old implementation of a userspace NFS server, but I don't think that would buy you much here.

                               

                               

                               

                              I think that the main limitation on speed is that the maintenance-VM is bandwidth-limited by ESX itself.

                               

                               

                               

                               

                               

                               

                               

                               

                               

                              If someone has anything else they'd like built, I can try to take a look at it.

                               

                               

                              EDIT:

                               

                               

                              And apologies for forgetting the chmod +x if it caused any confusion...

                              • 27. Re: Moving VMs from one ESXi server to another - what a pain!!!
                                ssapp80 Novice

                                thanks BThunderW.....that did the trick.

                                • 28. Re: Moving VMs from one ESXi server to another - what a pain!!!
                                  ssapp80 Novice

                                  BThunderW.......I'm using the rsync you compiled and its working great, I'm sending to a remote rsync daemon and getting 10mb/s on a 100mbit connection.........many thanks!!!!!

                                  • 29. Re: Moving VMs from one ESXi server to another - what a pain!!!
                                    josby Lurker

                                    Thanks glim, I did this and it works great.  However, the rsync binary in /bin disappears on reboot.  I am guessing the root filesystem in ESXi is a ramdisk that gets created from a compressed image file on each boot.  Did you experience this as well?  Any thoughts on a workaround?  I doubt updating the contents of the image file ESXi is using to include the binary would be easy.

                                     

                                    Oh, wait, there they are, in /vmfs/volumes/Hypervisor1.  Just tar files...I guess that wouldn't be that difficult after all.

                                     

                                    But, more than I want to mess with.  I ended up making a bin directory in /vmfs/volumes/datastore1 and putting rsync in there so it will persist across reboots, then added "--rsync-path=/vmfs/volumes/datastore1/bin/rsync" to the rsync command on my remote system that initiates the copying.

                                    1 2 3 4 Previous Next