12 Replies Latest reply on Sep 21, 2011 6:54 AM by AmarPatel

    VCB could not renew disk lease

    powers Enthusiast

       

      I'm running a HP MSA1000 SAN, Virtual center 2.5 with VCB on the same box, and esx 3.02

       

       

       

       

       

      When trying to manually run vcbmounter, I get the error

       

       

      "Could not renew disk lease. Aborting operation"

       

       

      during the backup of the vmdk file. Happens with all the VMs.

       

       

      I've tried changing the leasetimeoutSecs to 180 in the BLKLISTSVC section of /etc/vmware/hosts/config.xml  as per other people's posts, and have restarted the esx host to no avail.

       

       

       

       

       

      Anyone have any ideas or anything I can try?  

       

       

      Vmware tech support seems to have abandoned my SR ticket without the issue being resolved.  :|

       

       

        • 1. Re: VCB could not renew disk lease
          bigdee Enthusiast

           

          We had a similar problem. vcbmounter was able to create the snapshots on the VM but was not able to access the disk and aborted with the error "Could not renew disk lease".

           

           

          In our case it was a problem with permissions in VirtualCenter. The VCB-User needs permissions at the "Hosts & Clusters" Level.

           

           

          That solved our problem.

           

           

           

           

           

          CU

           

           

           

           

           

          • 2. Re: VCB could not renew disk lease
            powers Enthusiast

             

            Hi,

             

             

             

             

             

            I've tried using the Administrator account and another user that has Administrator level permissions, but it still doesn't work.

             

             

            • 3. Re: VCB could not renew disk lease
              dconvery Virtuoso
              vExpert

               

              powers -

               

               

              run vcbmounter with the "-L 6" switch and post the log.

               

               

              Dave

               

               

              • 4. Re: VCB could not renew disk lease
                powers Enthusiast

                 

                Hi Dave,

                 

                 

                I've attached the log file.

                 

                 

                 

                 

                 

                Thanks for your help

                 

                 

                 

                 

                 

                Eloise.

                 

                 

                • 5. Re: VCB could not renew disk lease
                  dconvery Virtuoso
                  vExpert

                   

                  Make sure you have enough free space on C:\mnt2.  Make sure that you delete any snapshots after a failure. Other than that, I don't see anything obvious. The only thing that I can think of is that maybe the MSA is overworked and you have a lot of SCSI disk locks cusing the problem. I assume by the disk path that you have created a 610G VMFS LUN. If you have more that 15-20 VMs running on the LUN, it could cause contention.

                   

                   

                  If it is actually copying the logs and vmx files during the operation, which the log indicates, then the connection from the VCB proxy to the VMFS is good.

                   

                   

                  Anyone else have ideas??

                   

                   

                  Dave

                   

                   

                  1 person found this helpful
                  • 6. Re: VCB could not renew disk lease
                    powers Enthusiast

                     

                    I'll try adding some more storage space on the vcb box.

                     

                     

                    Its got 60G free and I'm only trying to back up a 10G VM, but maybe it needs more for some reason......

                     

                     

                     

                     

                     

                    Thanks for checking the logs Dave

                     

                     

                    • 7. Re: VCB could not renew disk lease
                      dconvery Virtuoso
                      vExpert

                      Well - 60GB free is plenty for a 10GB VM. Do you have some free SAN space on the MSA? can you create a smaller LUN and svmotion a VM to the new LUN and try a backup?

                      • 8. Re: VCB could not renew disk lease
                        powers Enthusiast

                         

                        Finally got a response from vmware support (after no responses for  2 1/2 weeks)

                         

                        During backup procedure, VCB proxy periodically asks ESX host of disk lease extension.

                         

                        If this request delayed/lost due to poor network condition or increased load on VC (which acts as a broker between the VCB proxy and ESX host), the host declares disk lease timeout and fails the mounting operation.

                         

                         

                         

                        So when running vcbmounter and connecting to the esx server directly, the backup worked fine. Looks like VC was the problem...  (for whatever reason.. its not under much load at all)

                         

                         

                         

                         

                         

                         

                         

                         

                         

                         

                         

                         

                         

                         

                         

                        • 9. Re: VCB could not renew disk lease
                          dconvery Virtuoso
                          vExpert

                           

                          OK...now you know where to look. I would check speed/duplex on the network connections of the VC server. I see many many times in a 10/100 that the nic negotiates to full duplex and the switch negotiates to half. You should force the port speed and duplex on both the switch and NIC.  Is it doing anything OTHER than VC? Is there a backup client running other jobs at the same time? This causes CPU to spike while processing files. Is the database on the VC server or a separate server? Is it SQL express or MSDE?

                           

                           

                          You're right - the VC should not be so over taxed that it cannot process disk leases. I have worked with VC servers with 75-80 ESX servers associated and have had no issue with disk leases. Don't let them close the case until the problem is REALLY resolved. They just narrowed down the cause....

                           

                           

                          Dave

                           

                           

                          • 10. Re: VCB could not renew disk lease
                            powers Enthusiast

                             

                            You were spot on with the network diagnosis.  Gigabit card was running at 100mb fd.  Got it at 1000mb fd (driver upgrade & cable swap)

                             

                             

                             

                             

                             

                            However I still have the disk lease issue.  CPU load was around 5 - 10% while vcbmounter was running.

                             

                             

                             

                             

                             

                            The box runs SQL server 2000 and symantec (shudder) corporate antivirus server (it distributes virus def updates to around 30 pcs every few days)

                             

                             

                             

                             

                             

                            No backups or anything else running on it when I'm running vcbmounter......  Its not doing much of anything that would tax it.

                             

                             

                             

                             

                             

                            • 11. Re: VCB could not renew disk lease
                              dconvery Virtuoso
                              vExpert

                              It doesn't sound like SAV is bogging it down, but you can try to stop the SAV services to see if that makes a difference. Also, make sure DNS is rock-solid. Everything must resolve long, short, forward and reverse.

                              • 12. Re: VCB could not renew disk lease
                                AmarPatel Enthusiast

                                Hi,

                                 

                                Did you manage to find a solution this issue? I'm having a similar issue:

                                 

                                http://communities.vmware.com/message/1831791#1831791

                                 

                                Thanks

                                Amar