5 Replies Latest reply on Mar 8, 2011 7:58 AM by vmbru

    getFinalList: ERROR - Unable to locate VM:

    vmbru Enthusiast
    vmbru Feb 4, 2011 6:01 AM

      Wondering why ghettoVCBg2 skips over one of our VM's (PC682 Email Server).  Anyone have a clue?  We have script run at night daily.  Seems to happen every other day.  If I run manually it runs fine.  There is no Storage vMothion happening, stays on same host.  VM is not rebooted, strange.

       

       

      02-03-2011 18:31:13 -- info: ============================== ghettoVCBg2 LOG START ==============================

      02-03-2011 18:31:13 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log

      02-03-2011 18:31:13 -- info: CONFIG - VM_BACKUP_DATASTORE = VDRBACKUPS

      02-03-2011 18:31:13 -- info: CONFIG - VM_BACKUP_DIRECTORY =

      02-03-2011 18:31:13 -- info: CONFIG - DISK_BACKUP_FORMAT = thin

      02-03-2011 18:31:13 -- info: CONFIG - ADAPTER_FORMAT = lsilogic

      02-03-2011 18:31:13 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO

      02-03-2011 18:31:13 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO

      02-03-2011 18:31:13 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = YES

      02-03-2011 18:31:13 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2011-02-03

      02-03-2011 18:31:13 -- info: CONFIG - VM_VMDK_FILES = all

       

      02-03-2011 18:31:15 -- info: Initiate backup for Sign Server found on server1.kembacu.org

      02-03-2011 18:31:38 -- info: Sign Server has 1 VMDK(s)

      02-03-2011 18:37:48 -- info: Backup completed for Sign Server!

      02-03-2011 18:37:49 -- info: Initiate backup for DC found on server1.kembacu.org

      02-03-2011 18:38:14 -- info: DC has 1 VMDK(s)

      02-03-2011 18:40:21 -- info: Backup completed for DC!

      02-03-2011 18:40:21 -- info: Initiate backup for CITRIXMASTER found on server1.kembacu.org

      02-03-2011 18:40:49 -- info: CITRIXMASTER has 1 VMDK(s)

      02-03-2011 18:43:58 -- info: Backup completed for CITRIXMASTER!

      02-03-2011 18:43:59 -- info: Initiate backup for PC635 Credit Retrieval Server found on server1.kembacu.org

      02-03-2011 18:44:16 -- info: PC635 Credit Retrieval Server has 1 VMDK(s)

      02-03-2011 19:07:47 -- info: Backup completed for PC635 Credit Retrieval Server!

       

      02-03-2011 19:07:48 -- info: Initiate backup for PC682 Email Server found on server1.kembacu.org

       

      02-03-2011 19:08:50 -- info: Initiate backup for HawkSoft found on server1.kembacu.org

      02-03-2011 19:09:17 -- info: HawkSoft has 1 VMDK(s)

      02-03-2011 19:13:30 -- info: Backup completed for HawkSoft!

       

      02-04-2011 01:39:33 -- error: getFinalList: ERROR - Unable to locate VM: PC682 Email Server

      02-04-2011 01:39:33 -- info: ============================== ghettoVCBg2 LOG END ==============================

        • 1. Re: getFinalList: ERROR - Unable to locate VM:
          lamw Guru
          Community WarriorsVMware Employees
          lamw Feb 4, 2011 8:13 AM (in response to vmbru)

          Could you try enabling debugging in the logs? If you're able to manually back up the VM, it should not fail via your scheduled task ... I'm wondering if this VM is doing anything during this backup period? Is it always this VM, have you tried automating the backup during another backup window?

          • 2. Re: getFinalList: ERROR - Unable to locate VM:
            vmbru Enthusiast
            vmbru Feb 17, 2011 9:33 PM (in response to lamw)

            It is always this VM, happened tonight, last time it worked was the 16th, sporadic failures and different times.  Will try debug mode.

             

            02-18-2011 00:15:26 --  error: getFinalList: ERROR - Unable to locate VM: PC682 Email Server

            • 3. Re: getFinalList: ERROR - Unable to locate VM:
              lamw Guru
              VMware EmployeesCommunity Warriors
              lamw Feb 22, 2011 8:00 AM (in response to vmbru)

              You mentioned this is being schedule, is it via cron? If so, can you please provide your cron entry, you can use "crontab -l" to do so.

              • 4. Re: getFinalList: ERROR - Unable to locate VM:
                vmbru Enthusiast
                vmbru Feb 25, 2011 5:58 AM (in response to lamw)

                It happens non-scheduled too.

                 

                Is there another option besides debug to get more info?  Below doesn't really tell the story.  Ran it manually last night and it failed on a different VM, (PC601 FILE SERVER), I looked at the Event Viewer on the failed VM and looks like the Microsoft VSS timed out after about 4 minutes.  My guess is the vmware snapshot process OR VSS is real culprit, not ghettoVCBg2.  It seems the snapshot process starts, not sure if it completes, snapshot is removed successfully, ghettoVCBg2 does not get notified and hence the error: 02-25-2011 02:33:17 -- error: getFinalList: ERROR - Unable to locate VM: PC601 FILE SERVER

                 

                That sound about right?  Other then that not sure what the deal is, plenty of space, VM did not reboot during the backup.  It's a Windows 2003 R2 Enterprise 32bit server.

                 

                ---

                 

                02-25-2011 00:20:27 -- info: Backup completed for Synergy Web Server!

                02-25-2011 00:20:27 -- info: Initiate backup for PC601 FILE SERVER found on server3.kembacu.org

                 

                02-25-2011 00:20:35 -- debug: PC601 FILE SERVER original powerState: poweredOn

                02-25-2011 00:20:35 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2011-02-25" for PC601 FILE SERVER

                02-25-2011 00:21:15 -- debug: Main: Disconnect from: server3.kembacu.org

                02-25-2011 00:21:15 -- debug: Main: Login by vi-fastpass to: server4.kembacu.org

                 

                02-25-2011 00:21:17 -- info: Initiate backup for PRINTSERVER found on server4.kembacu.org

                02-25-2011 02:33:17 -- debug: Main: Disconnect from: server5.kembacu.org

                 

                02-25-2011 02:33:17 -- error: getFinalList: ERROR - Unable to locate VM: PC601 FILE SERVER

                 

                02-25-2011 02:33:17 -- debug: Main: Calling final clean up

                02-25-2011 02:33:17 -- debug: cleanUP: Thread clean up starting ...

                02-25-2011 02:33:17 -- debug: cleanUp: Send exit to copyThread

                02-25-2011 02:33:17 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit

                02-25-2011 02:33:17 -- debug: copyTask: die ...

                02-25-2011 02:33:17 -- debug: cleanUp: Join passed

                02-25-2011 02:33:17 -- info: ============================== ghettoVCBg2 LOG END ==============================

                 

                • 5. Re: getFinalList: ERROR - Unable to locate VM:
                  vmbru Enthusiast
                  vmbru Mar 8, 2011 7:58 AM (in response to vmbru)

                  Seems more of a VMware snapshot process issue, the snapshot process has issues from time to time which leads to GhettoVCBg2 getting out of sync.

                   

                  I think future GhettoVCBg2 needs a retry procedure.

                   

                  Can GhettoVCBg2 track what VM's fail to backup in a file so that I can run a script to re-try the backup later just for failed backups?  The file would just need to have the VM name(s).

                   

                  Ex.

                   

                  failedbackups.dat

                   

                  ----cut here-----

                  fileserver

                  printserver

                  ---end cut here-----

                   

                   

                  I could then cron the "backup only failed backups" job to run sometime after normal backup window.