8 Replies Latest reply on Oct 6, 2011 11:43 AM by ChrisColotti(VMware)

    Cloning initiated from vCD

    1Duncan Novice

      In vSphere, a VM or template is "owned" by a host. When that machine or template is cloned, the cloning takes place on that host.


      In a cloud environment it is possible, even likely, that multiple clones will be initiated close together as part of a bulk vApp deployment. In our case via the vCD API as part of testing to build 10, 100 vApps.


      All those clone operations take place on one ESXi host. This places an undue load on that host. Does anyone have any suggestions about how that load can be spread over the cluster?


      The same behaviour would occur if multiple clones were initiated from a vSphere client so it's not necessarily a vCD issue, but vCD certainly highlights the problem.




      Duncan Baillie

        • 1. Re: Cloning initiated from vCD
          Hot Shot

          I would have to do some testing, but I am not 100% convinced that all vApp deployments will always be performed on the same single host in the cluster. If the cluster has multiple hosts available there should be some logic within vCD/vSphere to use more than one host to run the deployment operations.  Have you seen this behaviour all the time?  This may be something I can investigate and take back to engineering if for some reason there is always a "Deployment" host chosen in the cluster.  To your point that would be not only a lot of load on a single host but it would cause that host to take longer to perform those operations.


          Let me look into it on my side, but I completely see your point.  My guy tells me this is handled by vSphere same as it is today when multiple clone operations are requested by say 10 different vSpere admins.  You are right that it is not a vCD issue, but I always thought it was taken care of by vSPhere Logic.


          We know that DRS will choose a host at POWER on, but I feel like there is some initial placement logic that chooses a host before power on to pick a host for the clone operations.  I have asked some others to weigh in.

          1 person found this helpful
          • 2. Re: Cloning initiated from vCD
            1Duncan Novice

            The deployment must be made of several discrete steps, the first (or early) step being the clone. The other steps may take place on different hosts (I haven't checked) but the clone always takes place on the same one, so yes this behaviour happens all the time.


            For reference our vCenter Server is 4.1.0 258902.

            • 3. Re: Cloning initiated from vCD
              Hot Shot

              I am still looking into it.  My lab has 4.1 and 5.0 running so I just need to figure out the test.  How are you determining the clone always lands on the same host?  CPU spikes, network usage, processeS?

              • 4. Re: Cloning initiated from vCD
                Hot Shot

                One working theory Duncan Epping came up with is it COULD be tied to the host that the template is actually registered on.  That host owns the template and could thus own the clone processes for it.  This should be easy to test, and we are asking internally to validate if this theory is right.  That being the case vCD could certainly compound that in a sense, but all vApp templates are usually powered on at some point.  That means DRS would load the hosts as needed, and when you power off the invididual VM's would remain on their respective hosts.  That also means you should see tasks on those hosts where the vApps Virtual Machines are still registered since they are no actual templates.


                This should be easy to test, but Duncan and I are now very interested in this


                I stil think this is a vSphere function question not specific to vCD.  The behaviour should be the same either way, vCD just is deploying multiple VM's as part of a vApp.  My focus is on the actual vSphere cloning process first.

                • 5. Re: Cloning initiated from vCD
                  depping Champion
                  VMware EmployeesUser Moderators

                  I just tested it.


                  The clone process is indeed spawned on the host on which the VM is registered.... If you are "unlucky" and have all VMs registed to the same host this could indeed be the effect it has.

                  • 6. Re: Cloning initiated from vCD
                    depping Champion
                    User ModeratorsVMware Employees

                    Your option would of course be moving them to the different hosts in your cluster. Yes I realize this is far from optimal and Chris and I will address this with engineering to see if there's a way to improve this.

                    1 person found this helpful
                    • 7. Re: Cloning initiated from vCD
                      1Duncan Novice

                      To address some of the points above...my theory was also that the behaviour was caused by the host on which the vm was registered :-). I determined that simply by looking at the tasks view of the Tasks and Events tab of the host owning the vm/template. Every single vApp deployment I kicked off appeared there.


                      We are trying to minimise the number of templates we maintain and dynamically adjust VMs via the API. As you say, if you had several templates you should spread them over the hosts but it's suboptimal. You could always get a user deploying en masse from the same vCD template.


                      Look forward to your engineering feedback.

                      • 8. Re: Cloning initiated from vCD
                        Hot Shot

                        Here is an update on the topic.


                        We have confirmed what we all suspected that the host the powered off VM is registered to will currently always perform the clone operation.  That being said, we will have gained some agreement that creating some additional logic to handle the clone process on the "most Available" host would be preferred. With that, there is not going to be a short term fix as this was to go in as a feature request and thus go through the development process.  Until then, the only work around as I see it to manually examine the registered powered off VM's and move them around.  Of course setting hosts in MM will move them around.  The real concern is the catalog vApp virtual machines here not the powered off deployed vApps.


                        This is a great topic, unfortunately with no short term answer.  Thanks for brining it up it has allowed for a couple days of good internal conversations on the topic.  In my mind this is how we better the product sets. 


                        Let's remember this is NOT a vCD specific issue as indicated in the title since vSphere is what handles the actual clones themselves. vCD just tells it to start the process but vCenter and ESX ultimately handle the clone.