vSphere Management Assistant (vMA) : ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!) : Comments

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

smyle — Fri, 22 May 2009 05:07:54 GMT

I have a VM with a virtual RDM. When I run the script, it immediately says 'WARN - RDM was found for (vmname), backup will not take place.' The comments in the code seem to indicate that this is normal, though the 'features list' above says they're supported. Any advice?

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Fri, 22 May 2009 07:27:12 GMT

Hi smyle,

I'm sorry, I think when I originally started writing the script I was implementing the features in ghettoVCB which did not support RDM period...I'm looking at the code right now and it is in fact blocking RDM's all together, let me jot that down as a note and I'll get back to you.

Thanks

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
http://twitter.com/lamw

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Fri, 22 May 2009 18:38:01 GMT

Everyone,

There is a known bug at the moment with vMA 4.0 and vSphere ESX(i) 4.0 that will not allow you to backup more than 1 VM, the logs will state that VM X can not be found after the first VM on the list is completed. I initially saw this with vMA 4.0 GA with vSphere ESX 4.0 RC but was hoping this might be due to the miss-match of the build. Something was dramatically alter/changed during RC release of vMA 4.0 and the GA release of vMA 4.0 that's causing this problem. I'll be investigating the root cause, but with VIMA 1.0 & ESX(i) 3.5u2+ will work and has been tested/verified.

Please let me know if you run into any other issues.

Thanks

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Fri, 22 May 2009 21:22:44 GMT

The bug has been identified and the issue actually lies with the vSphere CLI's version of vifs, I'm still awaiting feedback from VMware to confirm the issue.

Some details can be found here: http://engr.ucsb.edu/~duonglt/vmware/vifs_bug.html

Once the issue has been identified, I'll push out a temporarily fix to remediate the issue.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Wed, 27 May 2009 03:21:33 GMT

Just updated the script to fix both the virtual RDM issue & a bug with VMware's vifs, let me know if anyone runs into any other issues

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

yp1 — Fri, 29 May 2009 00:34:33 GMT

Hi,

It does not work for me with ESXi 4. It exits in 10-15 seconds. Looks like it fails to take a snapshot. I tried different types of the snapshots - same result. This is the latest version (same problem with the previous one).

Here is a typical log.

05-27-2009 18:03:53 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
05-27-2009 18:03:53 -- CONFIG - VM_BACKUP_DATASTORE = backup
05-27-2009 18:03:53 -- CONFIG - VM_BACKUP_DIRECTORY = backups
05-27-2009 18:03:53 -- CONFIG - DISK_BACKUP_FORMAT = eagerzeroedthick
05-27-2009 18:03:53 -- CONFIG - ADAPTER_FORMAT = lsilogic
05-27-2009 18:03:53 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
05-27-2009 18:03:53 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-27-2009 18:03:53 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-27-2009 18:03:53 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-05-27

05-27-2009 18:03:58 -- Initiate backup for "winxp" found on aaa.bbb.ccc.ddd!
05-27-2009 18:04:11 -- winxp original powerState: poweredOff
05-27-2009 18:04:11 -- winxp has 1 VMDK(s)
05-27-2009 18:04:11 -- Backing up "nexenta_nfs winxp/winxp.vmdk" to "backup backups/winxp/winxp-2009-05-27/winxp.vmdk"
05-27-2009 18:04:15 -- Backup completed for nexenta_nfs winxp/winxp.vmdk
05-27-2009 18:04:20 -- Backup completed for "winxp"!

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Fri, 29 May 2009 01:12:44 GMT

Are you using the free version of ESXi unlicensed? Licensed ESXi 4.0 or eval ESXi 4.0? I assume you're using vMA 4.0?

Can you please re-run the script and set the loglevel to "verbose" and attach the entire log as an attachment. Also run the script passing in the commandline param: --debug 1 and provide the entire log output as an attachment.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

yp1 — Fri, 29 May 2009 02:31:38 GMT

Sorry, I could not figure out how to attach a file, so have to add the log inline.
It is Free ESXi 4 with a registered license (not eval).

Verbose mode run, followed by debug:

============================== 05-28-2009 18:16:52 -- ghettoVCBg2 LOG START ==============================
05-28-2009 18:16:52 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
05-28-2009 18:16:52 -- CONFIG - VM_BACKUP_DATASTORE = backup
05-28-2009 18:16:52 -- CONFIG - VM_BACKUP_DIRECTORY = backups
05-28-2009 18:16:52 -- CONFIG - DISK_BACKUP_FORMAT = thin
05-28-2009 18:16:52 -- CONFIG - ADAPTER_FORMAT = lsilogic
05-28-2009 18:16:52 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
05-28-2009 18:16:52 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-28-2009 18:16:52 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-28-2009 18:16:52 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-05-28

05-28-2009 18:17:49 -- Initiate backup for "winxp" found on aaa.bbb.ccc.ddd!
05-28-2009 18:18:28 -- winxp original powerState: poweredOff
05-28-2009 18:18:28 -- winxp has 1 VMDK(s)
05-28-2009 18:18:28 -- Backing up "nexenta_nfs winxp/winxp.vmdk" to "backup backups/winxp/winxp-2009-05-28/winxp.vmdk"
05-28-2009 18:18:33 -- Backup completed for nexenta_nfs winxp/winxp.vmdk
05-28-2009 18:18:39 -- Backup completed for "winxp"!

============================== 05-28-2009 18:18:39 -- ghettoVCBg2 LOG END ==============================

============================== 05-28-2009 18:27:08 -- ghettoVCBg2 LOG START ==============================
05-28-2009 18:27:08 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
05-28-2009 18:27:08 -- CONFIG - VM_BACKUP_DATASTORE = backup
05-28-2009 18:27:08 -- CONFIG - VM_BACKUP_DIRECTORY = backups
05-28-2009 18:27:08 -- CONFIG - DISK_BACKUP_FORMAT = thin
05-28-2009 18:27:08 -- CONFIG - ADAPTER_FORMAT = lsilogic
05-28-2009 18:27:08 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
05-28-2009 18:27:08 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-28-2009 18:27:08 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-28-2009 18:27:08 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-05-28

05-28-2009 18:27:12 --

DEBUG INFO winxp

05-28-2009 18:27:12 -- DEBUG - Host Build: VMware ESXi 4.0.0 build-164009
05-28-2009 18:27:12 -- DEBUG - Host: aaa.bbb.ccc.ddd
05-28-2009 18:27:12 -- DEBUG - Virtual Machine: winxp
05-28-2009 18:27:12 -- DEBUG - VM ConfigPath: nexenta_nfs winxp/winxp.vmx
05-28-2009 18:27:12 -- DEBUG - VMX File: winxp.vmx
05-28-2009 18:27:12 -- DEBUG - BackupConfigPath: backup backups/winxp/winxp-2009-05-28/winxp.vmx
05-28-2009 18:27:12 -- DEBUG - BackupPath: backup backups/winxp/winxp-2009-05-28
05-28-2009 18:27:12 -- DEBUG - VM Datastore: nexenta_nfs
05-28-2009 18:27:12 -- DEBUG - VMDK(s):
05-28-2009 18:27:12 -- DEBUG - nexenta_nfs winxp/winxp.vmdk
05-28-2009 18:27:12 --

DEBUG INFO winxp

============================== 05-28-2009 18:27:12 -- ghettoVCBg2 LOG END ==============================

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Fri, 29 May 2009 02:35:28 GMT

Is your license either Foundation, Standard or Enterprise?

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

yp1 — Fri, 29 May 2009 02:45:03 GMT

It is a Free license (I just downloaded and requested the license)

Product: ESXi 4 Single Server Licensed for 1 physical CPUs (1-6 cores per CPU)

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Fri, 29 May 2009 02:48:46 GMT

Please take a look at the FAQ section, specifically 9A for your answer.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

kpc — Wed, 17 Jun 2009 10:25:50 GMT

Hi William

Good work on the scripts, not had a chance to try out this new script yet.

Love the centralized feature, will save on having to setup the NFS share on each ESXi host.

just wondering if you plan on adding in a feature where we don't have to keep lists of VM's at all, basically assuming that the user will always want to backup every VM they have. I know you now only have to keep the single list but I find I have so many VM's which come and go I often forget to add them to the backup list.

Cheers

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

angelone — Wed, 17 Jun 2009 19:28:59 GMT

Hi William,
thanks for this great script, this is exactly what I looked for.
But I have a small problem: The cloning doesn't work

I'm on ESXi 3.5u3 with vMA 4.0

When I'm on the ESXi via ssh and do the cloning with vmkfstools -i this works.
Your script displays "Unable to clone virtual disk : destName"

This is what it says:
~~vi-admin@service01 ~~~$ ./ghettoVCBg2.pl --config_dir backupConfigs --vmlist vmlist --debug 0
Unable to clone virtual disk : destName

This is the logfile:
============================== 06-17-2009 12:15:10 -- ghettoVCBg2 LOG START ==============================

06-17-2009 12:15:13 -- CONFIG - USING CONFIGURATION FILE = pcbae04
06-17-2009 12:15:13 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
06-17-2009 12:15:13 -- CONFIG - VM_BACKUP_DATASTORE = nas_7_backup
06-17-2009 12:15:13 -- CONFIG - VM_BACKUP_DIRECTORY = sicherung
06-17-2009 12:15:13 -- CONFIG - DISK_BACKUP_FORMAT = thin
06-17-2009 12:15:13 -- CONFIG - ADAPTER_FORMAT = buslogic;
06-17-2009 12:15:13 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = YES
06-17-2009 12:15:13 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-17-2009 12:15:13 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-17-2009 12:15:13 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-06-17
06-17-2009 12:15:13 -- Initiate backup for "pcbae04" found on 5.20.33.2!
06-17-2009 12:15:28 -- pcbae04 original powerState: poweredOff
06-17-2009 12:15:28 -- pcbae04 has 1 VMDK(s)
06-17-2009 12:15:28 -- Backing up "datastore1 pcbae04/pcbae04.vmdk" to "nas_7_backup sicherung/pcbae04/pcbae04-2009-06-17/pcbae04.vmdk"
06-17-2009 12:15:32 -- Backup completed for datastore1 pcbae04/pcbae04.vmdk
06-17-2009 12:15:38 -- Purging "nas_7_backup sicherung/pcbae04/pcbae04-2009-06-17--2" due to rotation max
06-17-2009 12:15:46 -- Backup completed for "pcbae04"!

============================== 06-17-2009 12:15:47 -- ghettoVCBg2 LOG END ==============================

And this is the backupconfig for pcbae04:
=====================================
VM_BACKUP_DATASTORE = "nas_7_backup"
VM_BACKUP_DIRECTORY = "sicherung"
VM_BACKUP_ROTATION_COUNT = "2"
DISK_BACKUP_FORMAT = "thin"
ADAPTER_FORMAT = "buslogic";
POWER_VM_DOWN_BEFORE_BACKUP = "1"
LOG_LEVEL = "verbose"
VM_SNAPSHOT_MEMORY = "0"
VM_SNAPSHOT_QUIESCE = "0"
=====================================

The only file copied to the NAS is the .vmx file.
It seems the whole backup process works, even snapshots are generated.
Just the clone command itself fails.

Do you have an idea what happens here?

thanks in advance,
Martin

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Thu, 18 Jun 2009 02:30:28 GMT

Thanks for the comments.

There will be no plans on changing the script to automatically assume all VM(s) will be backed up. If you need this feature you can modify the script accordlying, it's not that hard

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Thu, 18 Jun 2009 02:40:48 GMT

I know I ran into some oddities with VIMA 1.0's version of vmkfstools.pl but since you're using vMA 4.0, there should not be any issues. I've tested vMA with ESX 3.5u3 but not with ESXi and it should not matter in either case.

Looking at the logs, everything looks okay to me and I can't see anything that could be off. Can you provide the debugging output by running the following:

./ghettoVCBg2.pl --config_dir backupConfigs --vmlist vmlist --debug 1

Also can you modify the script and see what the actual command for trying to execute the backup is, please find line 623 which should look like the following:

$vmkfstools_cpy = `$vmkfstools_cmd --server "$host" --username "$host_username" --password "$host_password" -        i "$_" -a $ADAPTER_FORMAT -d $DISK_BACKUP_FORMAT "$vmdk_backup_destination" 2>&1`;

and replace it with:

$vmkfstools_cpy = `$vmkfstools_cmd --server "$host" --username "$host_username" --password "$host_password" -        i "$_" -a $ADAPTER_FORMAT -d $DISK_BACKUP_FORMAT "$vmdk_backup_destination"`;
print "Copy command: ", $vmkfstools_cpy,"\n";

Re-run the script as the following:

./ghettoVCBg2.pl --config_dir backupConfigs --vmlist vmlist

FYI - You don't need to specify "--debug 0" as it defaults to 0.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

angelone — Thu, 18 Jun 2009 07:58:14 GMT

Mea culpa!
Got it...

After an hour of debugging and digging in my great Perl knowledge I found the problem in the backupconfig:

ADAPTER_FORMAT = "buslogic";

That small semicolon at the end of the line leads to the ADAPTER_FORMAT variable being empty
So the vmkfstools command fails because -d needs a parameter.

Everything works now, thanks for the good work.

regards,
Martin

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

kpc — Thu, 18 Jun 2009 08:45:36 GMT

Tried the script, great work. If anyone else is backing up to NFS datastores you can mount the same store in the VMA appliance using the following:

sudo service portmap start
sudo service nfs start
sudo chkconfig portmap on
sudo chkconfig nfs on

sudo vi /etc/fstab (Add your nfs entry here e.g.)

10.192.4.99:/vmware /backup_vmware nfs defaults 0 0

sudo mount -a

Hope this helps

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

roadfox — Fri, 19 Jun 2009 20:39:40 GMT

Thanks for this great script and documentation.

Some small errors:

In "vm-backup-config-template" there is a semicolon at the of of ADAPTER_FORMAT which breaks the script when using config_dir

And some text in the log output is wrong QUIESCE where it should be MEMORY:

diff -f ghettoVCBg2.pl.bak ghettoVCBg2.pl
c195
print LOG timeStamp('MDYHMS'), "CONFIG - VM_SNAPSHOT_MEMORY = $quitext\n";
.
c390
print LOG timeStamp('MDYHMS'), "CONFIG - VM_SNAPSHOT_MEMORY = $quitext\n";
.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Sat, 20 Jun 2009 15:30:38 GMT

Thanks, a new vm-backup-config-template has been uploaded.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Sat, 20 Jun 2009 15:32:06 GMT

Is there any reason you would mount the backup datastore to vMA? In general, I would recommend locking down the NFS datastore to only ESX(i) hosts that way no accidental mounting from another host is allowed.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Sat, 20 Jun 2009 15:32:21 GMT

Thanks, a new vm-backup-config-template has been uploaded.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

IT_Architect — Thu, 25 Jun 2009 15:06:32 GMT

THANK YOU for this script. It's a real life saver. However, GhettoVCBg2 will be the deciding factor of whether or not people upgrade from ESXi 3.5u3. You could remedy that by adding thin provisioning support and a logging fix to GhettoVCB4i.

There are many people just getting in to virtualization who are not going to buy a foundation license. They will grow with the first virtualization vendor that has a solution that makes businesses sense for their current need, which includes a backup. Those who buy a foundation license, automatically receive VMware's VCB. For one or two servers, I would rather use GhettoVCB4i and a Windows machine to schedule it. People will need custom oem.tgz files with ESXi to address their needs for the foreseeable future anyway.

Please consider adding thin provisioning support and a logging fix to GhettoVCB4i, with the thin provisioning being the most important.

Thanks!

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Al3xandr3 — Fri, 26 Jun 2009 13:48:06 GMT

Hello lamw,

thanks a lot for your work but i got problems to run it.

Just after the snapshot, i got an error to copy virtual disk. It say: "general fault cause by file. One of the parameters supplied is invalid."

I use ESXi 3.5U3 with vMA4.0.

I tried with 2 differents VMs, with the local datastore and one NFS store and it still...

in /var/log/messages:
Jun 26 13:46:52 Hostd: ~~2009-06-26 13:46:52.014 'VdisksvcPlugin' 114696 warning~~ Invalid desination disk format 'preallocated'
Jun 26 13:46:52 Hostd: ~~2009-06-26 13:46:52.050 'BaseLibs' 114696 info~~ DISKLIB-LIB : Opened "/vmfs/volumes/4a4470d9-522186bf-68b6-0016355d8d4b/w2k3oem/w2k3oem.vmdk" (flags 0x6). A821FA4
Jun 26 13:46:52 Hostd: ~~2009-06-26 13:46:52.050 'BaseLibs' 114696 info~~ DISKLIB-LIB : Invalid type (0)
Jun 26 13:46:52 Hostd: ~~2009-06-26 13:46:52.050 'BaseLibs' 114696 info~~ DISKLIB-LIB : Failed to clone : One of the parameters supplied is invalid (1).
Jun 26 13:46:52 Hostd: ~~2009-06-26 13:46:52.050 'VdisksvcPlugin' 114696 warning~~ Failed to clone disk : One of the parameters supplied is invalid (1).

Ok, i found the solution. It was an error from DISK_BACKUP_FORMAT definition

sorry

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Shawn-H — Sun, 28 Jun 2009 22:42:16 GMT

Hi lamw,

Thanks for the script. I think it will be exactly what we are looking for but I am having some issues getting it to work. I am running it from VMA 4.0 and connecting to 2 ESXi 3.5 hosts. I have attached the log with a trial run and a debug run. Looks like it is having troubles doing a snapshot. Would really appreciate if you could take a look and give me some ideas. Thanks.

============================== 06-27-2009 22:09:00 -- ghettoVCBg2 LOG START ==============================
06-27-2009 22:09:00 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
06-27-2009 22:09:00 -- CONFIG - VM_BACKUP_DATASTORE = /vmfs/volumes/SANDisk2
06-27-2009 22:09:00 -- CONFIG - VM_BACKUP_DIRECTORY = Backups
06-27-2009 22:09:00 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
06-27-2009 22:09:00 -- CONFIG - ADAPTER_FORMAT = buslogic
06-27-2009 22:09:00 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
06-27-2009 22:09:00 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-27-2009 22:09:00 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-27-2009 22:09:00 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-06-27

06-27-2009 22:09:02 -- Initiate backup for "AntivirusServer" found on esx2.brwmsc!
06-27-2009 22:09:06 -- AntivirusServer original powerState: poweredOn
06-27-2009 22:09:06 -- AntivirusServer has 1 VMDK(s)
06-27-2009 22:09:06 -- Backing up "SANDisk2 Antivirus Server/Antivirus Server.vmdk" to "[/vmfs/volumes/SANDisk2] Backups/AntivirusServer/AntivirusServer-2009-06-27/Antivirus Server.vmdk"
06-27-2009 22:09:07 -- Backup completed for SANDisk2 Antivirus Server/Antivirus Server.vmdk
06-27-2009 22:09:07 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-06-27" not found
06-27-2009 22:09:08 -- Unable to purge "[/vmfs/volumes/SANDisk2] Backups/AntivirusServer/GET https:esx2.brwmscfolderBackupsAntivirusServer?dsName=vmfsvolumesSANDisk2 unsuccessful : 404 Not Found" due to rotation max
06-27-2009 22:09:08 -- Backup completed for "AntivirusServer"!

============================== 06-27-2009 22:09:08 -- ghettoVCBg2 LOG END ==============================

============================== 06-28-2009 14:30:24 -- ghettoVCBg2 LOG START ==============================
06-28-2009 14:30:24 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
06-28-2009 14:30:24 -- CONFIG - VM_BACKUP_DATASTORE = /vmfs/volumes/SANDisk2
06-28-2009 14:30:24 -- CONFIG - VM_BACKUP_DIRECTORY = Backups
06-28-2009 14:30:24 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
06-28-2009 14:30:24 -- CONFIG - ADAPTER_FORMAT = buslogic
06-28-2009 14:30:24 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
06-28-2009 14:30:24 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-28-2009 14:30:24 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-28-2009 14:30:24 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-06-28

06-28-2009 14:30:25 --

DEBUG INFO AntivirusServer

06-28-2009 14:30:25 -- DEBUG - Host Build: VMware ESX Server 3i 3.5.0 build-163429
06-28-2009 14:30:25 -- DEBUG - Host: esx2.brwmsc
06-28-2009 14:30:25 -- DEBUG - Virtual Machine: AntivirusServer
06-28-2009 14:30:25 -- DEBUG - VM ConfigPath: SANDisk2 Antivirus Server/Antivirus Server.vmx
06-28-2009 14:30:25 -- DEBUG - VMX File: Antivirus Server.vmx
06-28-2009 14:30:25 -- DEBUG - BackupConfigPath: [/vmfs/volumes/SANDisk2] Backups/AntivirusServer/AntivirusServer-2009-06-28/Antivirus Server.vmx
06-28-2009 14:30:25 -- DEBUG - BackupPath: [/vmfs/volumes/SANDisk2] Backups/AntivirusServer/AntivirusServer-2009-06-28
06-28-2009 14:30:25 -- DEBUG - VM Datastore: SANDisk2
06-28-2009 14:30:25 -- DEBUG - VMDK(s):
06-28-2009 14:30:25 -- DEBUG - SANDisk2 Antivirus Server/Antivirus Server.vmdk
06-28-2009 14:30:25 --

DEBUG INFO AntivirusServer

============================== 06-28-2009 14:30:26 -- ghettoVCBg2 LOG END ==============================

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Koldijk — Mon, 29 Jun 2009 13:07:36 GMT

Hello thanks for this script, I am still using your older script! Trying your new script gives me some errors

~~vi-admin@bbikol-vima ~~~$ ./ghettoVCBg2.pl --config_dir backupConfigs --vmlist vm_backup_list
Use of uninitialized value in pattern match (m//) at /usr/lib/perl5/5.8.8/VMware/VIExt.pm line 124.
Use of uninitialized value in pattern match (m//) at /usr/lib/perl5/5.8.8/VMware/VIExt.pm line 124.
Use of uninitialized value in pattern match (m//) at /usr/lib/perl5/5.8.8/VMware/VIExt.pm line 124.
Use of uninitialized value in pattern match (m//) at /usr/lib/perl5/5.8.8/VMware/VIExt.pm line 124.
Use of uninitialized value in concatenation (.) or string at /usr/bin/vifs.pl line 294.
Unable to copy NFS BBIKOL-ZW-DC01/BBIKOL-ZW-DC01.vmx to :
destinationName

backupConfigs: 1 file named BBIKOL-ZW-DC01

##################################

VM BACKUP CONFIGURATIONS ###

##################################
VM_BACKUP_DATASTORE = "vmbackup"
VM_BACKUP_DIRECTORY = "esx"
VM_BACKUP_ROTATION_COUNT ="7"
DISK_BACKUP_FORMAT = "zeroedthick"
ADAPTER_FORMAT = "lsilogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
LOG_LEVEL = "normal"
VM_SNAPSHOT_MEMORY = "0"
VM_SNAPSHOT_QUIESCE = "0"

Ideas?

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

freez267 — Tue, 30 Jun 2009 14:56:12 GMT

I have some problem to backup a VM in my esxi 4.0 server. I always receive this error:

/vmfs/volumes/48d3c7c5-9c6dddaa-59ff-001ec9b5cb21 # sh ghettoVCB4i.sh test
Taking backup snapshot for ANTHEAPRO ...
DiskLib_Check() failed for source disk The file specified is not a virtual disk (15).
Destination disk format: VMFS zeroedthick
Cloning disk '/vmfs/volumes/datastore1/antheapro.GruppoAnthea.local/antheapro.GruppoAnthea.local.vmdk'...
Clone: 100% done.
Removing snapshot from ANTHEAPRO ...
sh: /vmfs/volumes/VMBACKUP/ESX15/ANTHEAPRO/ANTHEAPRO-2009-06-27: bad number
ghettoVCB4i.sh: line 349: syntax error: /vmfs/volumes/VMBACKUP/ESX15/ANTHEAPRO/ANTHEAPRO-2009-06-27+1
/vmfs/volumes/48d3c7c5-9c6dddaa-59ff-001ec9b5cb21 #

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Wed, 01 Jul 2009 04:48:11 GMT

I can't see anything out of the ordinary, just make sure you don't have any "semi-colon" in your configuration file and that you're abiding by the "ESX/ESXi Version Support Table" on this page.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Wed, 01 Jul 2009 04:50:11 GMT

Looks like it's complaining about a param that's not correct or unable to parse, what version of ESX(i) and VIMA/vMA are you using? I would suggest keeping the two consistent between the old VI 3.5 and vSphere 4.0

Also make sure you don't have any "semi-colon" in your configuration file

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Wed, 01 Jul 2009 04:51:16 GMT

Not sure why you're getting this issue, are you using the latest ghettoVCB4i.sh script? Is it just failing for this single VM?

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

freez267 — Wed, 01 Jul 2009 08:47:45 GMT

Yes I'm using the last ghettoVCB4i.sh script.
The script is used to backup 20 vms and this is the only one that has some problems.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

angelone — Fri, 03 Jul 2009 07:02:32 GMT

Hi William,
I still have problems here.

I have 2 ESXi 3.5u3 with one VMA4 each.

And on both systems the script can't find the snapshot it created prior to copying the disks.

This is what I get every night on both systems:

============================== 07-03-2009 02:00:01 -- ghettoVCBg2 LOG START ==============================

07-03-2009 02:00:04 -- CONFIG - USING CONFIGURATION FILE = stobae03
07-03-2009 02:00:04 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
07-03-2009 02:00:04 -- CONFIG - VM_BACKUP_DATASTORE = nas_9_backup
07-03-2009 02:00:04 -- CONFIG - VM_BACKUP_DIRECTORY = sicherung
07-03-2009 02:00:04 -- CONFIG - DISK_BACKUP_FORMAT = thin
07-03-2009 02:00:04 -- CONFIG - ADAPTER_FORMAT = lsilogic
07-03-2009 02:00:04 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-03-2009 02:00:04 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-03-2009 02:00:04 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-03-2009 02:00:04 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-03
07-03-2009 02:00:04 -- Initiate backup for "stobae03" found on 5.20.33.17!
07-03-2009 02:00:18 -- stobae03 original powerState: poweredOn
07-03-2009 02:00:20 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-03" for stobae03
07-03-2009 02:00:20 -- stobae03 has 2 VMDK(s)
07-03-2009 02:00:20 -- Backing up "datastore1 stobae03/stobae03.vmdk" to "nas_9_backup sicherung/stobae03/stobae03-2009-07-03/stobae03.vmdk"
07-03-2009 02:05:01 -- Backup completed for datastore1 stobae03/stobae03.vmdk
07-03-2009 02:05:01 -- Backing up "datastore1 stobae03/stobae03_1.vmdk" to "nas_9_backup sicherung/stobae03/stobae03-2009-07-03/stobae03_1.vmdk"
07-03-2009 04:52:35 -- Backup completed for datastore1 stobae03/stobae03_1.vmdk
07-03-2009 04:52:35 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-03" not found
07-03-2009 04:52:42 -- Backup completed for "stobae03"!

============================== 07-03-2009 04:52:42 -- ghettoVCBg2 LOG END ==============================

The snapshot is still there and works.
I then have to shut down the machine and then delete all snapshots in the virtualcenter snapshotmanager.
"removeallsnapshots" somehow doesn't work from console.

Another interesting point is, that after errors like this one, all other VMs in my vmlist won't be backed up.
I have 5 vms in my vmlist.
The first is backed up without problems, the second might have an error like the one above and the other three will not be found later on.

If the first backup has an error the following four VMs won't be found.

The problem lies imho somewhere in the VMA or the script, I can put the last one of my VMs on top of the vmlist and it will be found and backed up.

The hostd.log on the ESXi says the following:
http://www.myangelone.de/files/hostd.log-esxi.txt (the ESXi is 2h behind in time, sry)

And has anyone seen this error before?
This pops up on my ESXi every 20min all day long.
Regardless if it's used or not.
Even if no vm is running and no Infrastructure client connected.
I get this on both ESXi, too.

~~2009-07-03 05:40:32.721 'App' 147466 error~~ Failed to send response to the client: Broken pipe
~~2009-07-03 06:00:33.552 'Vmomi' 65541 info~~ Activation N5Vmomi10ActivationE:0x18e03e60 : Invoke done waitForUpdates on ~~vmodl.query.PropertyCollector:ha-property-collector~~
~~2009-07-03 06:00:33.552 'Vmomi' 65541 info~~ Throw vmodl.fault.RequestCanceled
~~2009-07-03 06:00:33.552 'Vmomi' 65541 info~~ Result:
(vmodl.fault.RequestCanceled) {
dynamicType = <unset>,
msg = ""
}
~~2009-07-03 06:00:33.557 'App' 65541 error~~ Failed to send response to the client: Broken pipe
~~2009-07-03 06:20:34.383 'Vmomi' 131081 info~~ Activation N5Vmomi10ActivationE:0x18702a40 : Invoke done waitForUpdates on ~~vmodl.query.PropertyCollector:ha-property-collector~~
~~2009-07-03 06:20:34.384 'Vmomi' 131081 info~~ Throw vmodl.fault.RequestCanceled
~~2009-07-03 06:20:34.384 'Vmomi' 131081 info~~ Result:
(vmodl.fault.RequestCanceled) {
dynamicType = <unset>,
msg = ""
}
~~2009-07-03 06:20:34.385 'App' 131081 error~~ Failed to send response to the client: Broken pipe

Any help is appreciated, maybe I should switch to the old script.

Is there a way to modify the script, so that it deletes all snapshots after the copy process?
Maybe the problem is the named snapshot.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

grantdavies — Sat, 04 Jul 2009 01:32:56 GMT

Great job on the script, I've got it backing up 3 Virtual Machine guests to a Windows XP 64 bit target.

One of the VMs works perfectly as expected, but two of them fail to close the Snapshots and move on to the next virtual machine.

I've checked the names and the scripts (and the logs) and the script tries to close snapshots with exactly the same name as it opens, but for some reason fails. Any ideas?

I think this has to do with the NFS mount (OpenFiler 2.3) that both ZEROEDTHICK and EAGERZEROEDTHICKas the disk type result in 'General fault caused by file. One of the parameters supplied is invalid' in the VI client.

When using disk type 'THIN' the backup completes, but the snapshot if left open and the log says it failed to find the snap - even though it's exactly the same name of the snap it created!

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

coobic — Fri, 10 Jul 2009 00:05:40 GMT

Hi, angelone and William.
We have the same problem with 'angelone' with backup more than 2 vm`s (ESXi 4, vMA 4).
We try to separate backups for Unix and Windows machines. Backup for Unix (Linux and FreeBSD) machines working perfectlty (3 vm`s), but for Windows machines backup work only for one, that located on top of backup list. Log:
============================== 07-10-2009 08:57:50 -- ghettoVCBg2 LOG START ==============================
07-10-2009 08:57:50 -- CONFIG - BACKUP_LOG_OUTPUT = /var/log/ghettoVCBg2.log
07-10-2009 08:57:50 -- CONFIG - VM_BACKUP_DATASTORE = Backup
07-10-2009 08:57:50 -- CONFIG - VM_BACKUP_DIRECTORY = BACKUPS
07-10-2009 08:57:50 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
07-10-2009 08:57:50 -- CONFIG - ADAPTER_FORMAT = lsilogic
07-10-2009 08:57:50 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-10-2009 08:57:50 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-10-2009 08:57:50 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-10-2009 08:57:50 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-10

07-10-2009 08:57:54 -- Initiate backup for "maildat.xxx" found on c3000-box1-vmware2.xxx!
07-10-2009 08:58:03 -- maildat.xxx original powerState: poweredOn
07-10-2009 08:58:05 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-10" for maildat.xxx
07-10-2009 08:58:05 -- maildat.xxx has 1 VMDK(s)
07-10-2009 08:58:05 -- Backing up "Array10-1 maildat.xxx/maildat.xxx.vmdk" to "Backup BACKUPS/maildat.xxx/maildat.xxx-2009-07-10/maildat.xxx.vmdk"
07-10-2009 09:13:56 -- Backup completed for Array10-1 maildat.xxx/maildat.xxx.vmdk
07-10-2009 09:13:59 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-07-10" for maildat.xxx
07-10-2009 09:14:30 -- Backup completed for "maildat.xxx"!
07-10-2009 09:14:33 -- ERROR - Unable to locate VM: FC.xxx
07-10-2009 09:14:33 -- ERROR - Unable to locate VM: R2D2.xxx
07-10-2009 09:14:33 -- ERROR - Unable to locate VM: ts.xxx
07-10-2009 09:14:33 -- ERROR - Unable to locate VM: n2000-bms.xxx

Sometimes backup completed successfully, but error for 'top' machine persist:
07-09-2009 12:31:29 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-09" not found
And we can delete snapshot with Snapshot Manager 'by hand'.

I can e-mail verbose logging for this situations.

Thanks for future help.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

angelone — Fri, 10 Jul 2009 10:04:34 GMT

now, does anyone know why removing all snapshots with the GUI manager works and "removeallsnapshots" from CLI tells me that there are no snapshots?

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Fri, 10 Jul 2009 13:55:42 GMT

Sorry for the delay of response but I've been extremely busy over the last month and I currently do not have the cycles to look into the current issues. I'm aware there are some interesting problems being faced that may all be related. I will provide some more details when I get a chance to take a look at the issue hopefully in a few weeks after my VCP4 exam. Thanks for your patience

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

coobic — Wed, 15 Jul 2009 22:34:15 GMT

http://kb.vmware.com/kb/1012383

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

angelone — Thu, 16 Jul 2009 06:27:19 GMT

Hm, I don't think this is the problem here since ghettoVCB doesn't set these flags.

Our problem is that the snapshots aren't found and therefore can't be deleted.
Above KB article says, that you can't delete your snapshots because the left over flag phobits it.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

cookieme — Sat, 18 Jul 2009 23:30:02 GMT

I've read through this thread and I am really interested in using this script to backup my VMs. I am still learning how to use ESXi 4, so this might be a repeat question.

Can I use this script on my ESXi 4 host if I am using the "free" version? I saw something about it not working if you have not paid for a license due to some CLI thingy missing. If this is the case, then what to do for backup?

Thanks

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Sat, 18 Jul 2009 23:32:13 GMT

Please re-read the document/requirements, specifically the section labeled "ESX/ESXi Version Support Table".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Sat, 18 Jul 2009 23:33:07 GMT

Correct, that has nothing to do with the issues being seen now. The KB relates to using vDR (VMware Data Recovery) where snapshots are not being removed and there has been a fix for that which is NOT related to this script at all, unless you're using vDR.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Mon, 20 Jul 2009 14:02:59 GMT

For those that are running into issues where backups are occurring but snapshots are not being removed or other issues that are similarly related, please take a look here http://tinyurl.com/layekn

The drawing starts today and you have ~1week to enter. Thanks everyone

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

chris647 — Wed, 22 Jul 2009 13:14:33 GMT

Is there any way to prevent the script from backing up all vmdks attached to the VM?
I just want to backup the primary vmdk, not vmdks across multiple datastores.
Right now when I backup a VM, the primary vmdk processes fine, but when it hits the secondardy vmdk on another datastore, it gives an error that the file is locked or in use.

Thanks in advance!

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Wed, 22 Jul 2009 13:53:03 GMT

No, that is not a supported feature of the script. You're more than welcome to add logic to exclude certain VMDK(s), no support will be provided for this.

The script should work if you have multiple VMDK(s) across multiple datastores. Is the script taking snapshots prior to backup? Please provide more details and take a look at FAQ #10

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

chris647 — Wed, 22 Jul 2009 13:59:37 GMT

Yes I am doing snapshots prior to the backup.
The second disk is configured as an Independant Disk which is why snapshots will not run, I assume.
I was hoping by using it as an Independant disk, the backup would skip it.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Wed, 22 Jul 2009 14:03:26 GMT

Bingo, Independant Disk can not be snapshotted hence the error that its locked since w/o the snapshot and I assume the VM was powered on, it would be locked for write access.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Al3xandr3 — Mon, 03 Aug 2009 08:59:52 GMT

Hello,

I'm trying to use the script on two similar configuration: ESXi 3.5u3 free licensied and vMA 4.0

I got always same problems.

Snapshot is not removed.
VMs turned off are not turned on
The backup rotation isn't working either.

============================== 07-29-2009 22:01:01 -- ghettoVCBg2 LOG START ==============================

07-29-2009 22:01:04 -- CONFIG - USING CONFIGURATION FILE = SRVLINUX
07-29-2009 22:01:04 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
07-29-2009 22:01:04 -- CONFIG - VM_BACKUP_DATASTORE = NFS
07-29-2009 22:01:04 -- CONFIG - VM_BACKUP_DIRECTORY = DAILY
07-29-2009 22:01:04 -- CONFIG - DISK_BACKUP_FORMAT = thin
07-29-2009 22:01:04 -- CONFIG - ADAPTER_FORMAT = buslogic
07-29-2009 22:01:04 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-29-2009 22:01:04 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-29-2009 22:01:04 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-29-2009 22:01:04 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-29
07-29-2009 22:01:04 -- Initiate backup for "SRVLINUX" found on 192.168.0.110!
07-29-2009 22:01:15 -- SRVLINUX original powerState: poweredOn
07-29-2009 22:01:17 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-29" for SRVLINUX
07-29-2009 22:01:17 -- SRVLINUX has 1 VMDK(s)
07-29-2009 22:01:17 -- Backing up "datastore1 SRVLINUX/SRVLINUX_2.vmdk" to "NFS DAILY/SRVLINUX/SRVLINUX-2009-07-29/SRVLINUX_2.vmdk"
07-29-2009 22:33:21 -- Backup completed for datastore1 SRVLINUX/SRVLINUX_2.vmdk
07-29-2009 22:33:21 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-29" not found
07-29-2009 22:33:45 -- Unable to purge "NFS DAILY/SRVLINUX/SRVLINUX-2009-07-21--5" due to rotation max
07-29-2009 22:33:49 -- Backup completed for "SRVLINUX"!

============================== 07-29-2009 22:33:49 -- ghettoVCBg2 LOG END ==============================

============================== 07-29-2009 22:34:00 -- ghettoVCBg2 LOG START ==============================

07-29-2009 22:34:03 -- CONFIG - USING CONFIGURATION FILE = SRVHP
07-29-2009 22:34:03 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
07-29-2009 22:34:03 -- CONFIG - VM_BACKUP_DATASTORE = NFS
07-29-2009 22:34:03 -- CONFIG - VM_BACKUP_DIRECTORY = DAILY
07-29-2009 22:34:03 -- CONFIG - DISK_BACKUP_FORMAT = thin
07-29-2009 22:34:03 -- CONFIG - ADAPTER_FORMAT = lsilogic
07-29-2009 22:34:03 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-29-2009 22:34:03 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
07-29-2009 22:34:03 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
07-29-2009 22:34:03 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-29
07-29-2009 22:34:03 -- Initiate backup for "SRVHP" found on 192.168.0.110!
07-29-2009 22:34:14 -- SRVHP original powerState: poweredOn
07-29-2009 22:36:05 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-29" for SRVHP
07-29-2009 22:36:05 -- SRVHP has 1 VMDK(s)
07-29-2009 22:36:05 -- Backing up "datastore1 SRVHP/SRVHP.vmdk" to "NFS DAILY/SRVHP/SRVHP-2009-07-29/SRVHP.vmdk"
07-30-2009 01:39:17 -- Backup completed for datastore1 SRVHP/SRVHP.vmdk
07-30-2009 01:39:17 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-29" not found
07-30-2009 01:39:40 -- Unable to purge "NFS DAILY/SRVHP/SRVHP-2009-07-21--5" due to rotation max
07-30-2009 01:39:40 -- Backup completed for "SRVHP"!

============================== 07-30-2009 01:39:40 -- ghettoVCBg2 LOG END ==============================

============================== 07-30-2009 01:39:54 -- ghettoVCBg2 LOG START ==============================

07-30-2009 01:39:57 -- CONFIG - USING CONFIGURATION FILE = SRVAPPLI
07-30-2009 01:39:57 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
07-30-2009 01:39:57 -- CONFIG - VM_BACKUP_DATASTORE = NFS
07-30-2009 01:39:57 -- CONFIG - VM_BACKUP_DIRECTORY = DAILY
07-30-2009 01:39:57 -- CONFIG - DISK_BACKUP_FORMAT = thin
07-30-2009 01:39:57 -- CONFIG - ADAPTER_FORMAT = lsilogic
07-30-2009 01:39:57 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-30-2009 01:39:57 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
07-30-2009 01:39:57 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
07-30-2009 01:39:57 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-30
07-30-2009 01:39:57 -- Initiate backup for "SRVAPPLI" found on 192.168.0.110!
07-30-2009 01:40:08 -- SRVAPPLI original powerState: poweredOn
07-30-2009 01:41:44 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-30" for SRVAPPLI
07-30-2009 01:41:44 -- SRVAPPLI has 1 VMDK(s)
07-30-2009 01:41:44 -- Backing up "datastore1 SRVAPPLI/SRVAPPLI.vmdk" to "NFS DAILY/SRVAPPLI/SRVAPPLI-2009-07-30/SRVAPPLI.vmdk"
07-30-2009 05:10:32 -- Backup completed for datastore1 SRVAPPLI/SRVAPPLI.vmdk
07-30-2009 05:10:32 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-30" not found
07-30-2009 05:10:55 -- Unable to purge "NFS DAILY/SRVAPPLI/SRVAPPLI-2009-07-21--5" due to rotation max
07-30-2009 05:10:59 -- Backup completed for "SRVAPPLI"!

============================== 07-30-2009 05:10:59 -- ghettoVCBg2 LOG END ==============================

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Tue, 04 Aug 2009 03:48:00 GMT

Curious, have you tried to backup the 3 VMs but instead of using the configuration file just a flat file containing the VMs that need to be backed up. I know few others have ran into similar situations and so far everyone has been using the configurations to execute their backups. Just wondering if there is a bug in the configuration backup method or if this is a general issue.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Al3xandr3 — Tue, 04 Aug 2009 07:25:58 GMT

I made a test on saturday on a similar architecture.

I launched manually the script for each VM (so I modified each time vmlist file) without configuration file.I didn't try the backup rotation and power off VM.

All is ok for the first VM but for the two others, snapshot still not found.

It's really strange because, like i told you, i launched each backup process independently.

============================== 08-01-2009 16:19:48 -- ghettoVCBg2 LOG START ==============================
08-01-2009 16:19:48 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-01-2009 16:19:48 -- CONFIG - VM_BACKUP_DATASTORE = NFS
08-01-2009 16:19:48 -- CONFIG - VM_BACKUP_DIRECTORY = WEEKLY
08-01-2009 16:19:48 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-01-2009 16:19:48 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-01-2009 16:19:48 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-01-2009 16:19:48 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 16:19:48 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 16:19:48 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-01

08-01-2009 16:19:51 -- Initiate backup for "SRVAD" found on 192.128.10.250!
08-01-2009 16:20:01 -- SRVAD original powerState: poweredOn
08-01-2009 16:28:24 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-01" for SRVAD
08-01-2009 16:28:24 -- SRVAD has 1 VMDK(s)
08-01-2009 16:28:24 -- Backing up "datastore1 SRVAD/SRVAD.vmdk" to "NFS WEEKLY/SRVAD/SRVAD-2009-08-01/SRVAD.vmdk"
08-01-2009 16:55:08 -- Backup completed for datastore1 SRVAD/SRVAD.vmdk
08-01-2009 16:55:47 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-08-01" for SRVAD
08-01-2009 16:55:53 -- Backup completed for "SRVAD"!

============================== 08-01-2009 16:55:53 -- ghettoVCBg2 LOG END ==============================

============================== 08-01-2009 17:08:40 -- ghettoVCBg2 LOG START ==============================
08-01-2009 17:08:40 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-01-2009 17:08:40 -- CONFIG - VM_BACKUP_DATASTORE = NFS
08-01-2009 17:08:40 -- CONFIG - VM_BACKUP_DIRECTORY = WEEKLY
08-01-2009 17:08:40 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-01-2009 17:08:40 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-01-2009 17:08:40 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-01-2009 17:08:40 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 17:08:40 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 17:08:40 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-01

08-01-2009 17:08:43 -- Initiate backup for "SRVEXCHANGE" found on 192.128.10.250!
08-01-2009 17:08:52 -- SRVEXCHANGE original powerState: poweredOn
08-01-2009 17:11:37 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-01" for SRVEXCHANGE
08-01-2009 17:11:37 -- SRVEXCHANGE has 1 VMDK(s)
08-01-2009 17:11:37 -- Backing up "datastore1 SRVEXCHANGE/SRVEXCHANGE.vmdk" to "NFS WEEKLY/SRVEXCHANGE/SRVEXCHANGE-2009-08-01/SRVEXCHANGE.vmdk"
08-01-2009 19:15:12 -- Backup completed for datastore1 SRVEXCHANGE/SRVEXCHANGE.vmdk
08-01-2009 19:15:13 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-01" not found
08-01-2009 19:15:18 -- Backup completed for "SRVEXCHANGE"!

============================== 08-01-2009 19:15:18 -- ghettoVCBg2 LOG END ==============================

============================== 08-01-2009 19:20:59 -- ghettoVCBg2 LOG START ==============================
08-01-2009 19:20:59 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-01-2009 19:20:59 -- CONFIG - VM_BACKUP_DATASTORE = NFS
08-01-2009 19:20:59 -- CONFIG - VM_BACKUP_DIRECTORY = WEEKLY
08-01-2009 19:20:59 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-01-2009 19:20:59 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-01-2009 19:20:59 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-01-2009 19:20:59 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 19:20:59 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 19:20:59 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-01

08-01-2009 19:21:02 -- Initiate backup for "SRVTSE" found on 192.128.10.250!
08-01-2009 19:21:11 -- SRVTSE original powerState: poweredOn
08-01-2009 19:45:45 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-01" for SRVTSE
08-01-2009 19:45:45 -- SRVTSE has 2 VMDK(s)
08-01-2009 19:45:45 -- Backing up "datastore1 SRVTSE/SRVTSE.vmdk" to "NFS WEEKLY/SRVTSE/SRVTSE-2009-08-01/SRVTSE.vmdk"
08-01-2009 20:53:10 -- Backup completed for datastore1 SRVTSE/SRVTSE.vmdk
08-01-2009 20:53:10 -- Backing up "datastore1 SRVTSE/SRVTSE_1.vmdk" to "NFS WEEKLY/SRVTSE/SRVTSE-2009-08-01/SRVTSE_1.vmdk"
08-01-2009 22:48:32 -- Backup completed for datastore1 SRVTSE/SRVTSE_1.vmdk
08-01-2009 22:48:32 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-01" not found
08-01-2009 22:48:38 -- Backup completed for "SRVTSE"!

============================== 08-01-2009 22:48:38 -- ghettoVCBg2 LOG END ==============================

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Tue, 04 Aug 2009 13:49:29 GMT

Can you do the same test but with all 3 VMs on 1 backup vmlist and post your results.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Al3xandr3 — Mon, 10 Aug 2009 07:51:05 GMT

Hello,

i did the same test with 3 VMs on 1 backup vmlist. I post the result.

============================== 08-08-2009 11:28:24 -- ghettoVCBg2 LOG START ==============================
08-08-2009 11:28:24 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-08-2009 11:28:24 -- CONFIG - VM_BACKUP_DATASTORE = NFS
08-08-2009 11:28:24 -- CONFIG - VM_BACKUP_DIRECTORY = WEEKLY
08-08-2009 11:28:24 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-08-2009 11:28:24 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-08-2009 11:28:24 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-08-2009 11:28:24 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-08-2009 11:28:24 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-08-2009 11:28:24 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-08

08-08-2009 11:28:28 -- Initiate backup for "SRVAD" found on 192.128.10.250!
08-08-2009 11:28:38 -- SRVAD original powerState: poweredOn
08-08-2009 11:36:17 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-08" for SRVAD
08-08-2009 11:36:17 -- SRVAD has 1 VMDK(s)
08-08-2009 11:36:17 -- Backing up "datastore1 SRVAD/SRVAD.vmdk" to "NFS WEEKLY/SRVAD/SRVAD-2009-08-08/SRVAD.vmdk"
08-08-2009 12:04:06 -- Backup completed for datastore1 SRVAD/SRVAD.vmdk
08-08-2009 12:04:41 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-08-08" for SRVAD
08-08-2009 12:04:50 -- Backup completed for "SRVAD"!

08-08-2009 12:04:50 -- Initiate backup for "SRVTSE" found on 192.128.10.250!
08-08-2009 12:05:00 -- SRVTSE original powerState: poweredOn
08-08-2009 12:13:45 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-08" for SRVTSE
08-08-2009 12:13:45 -- SRVTSE has 2 VMDK(s)
08-08-2009 12:13:45 -- Backing up "datastore1 SRVTSE/SRVTSE.vmdk" to "NFS WEEKLY/SRVTSE/SRVTSE-2009-08-08/SRVTSE.vmdk"
08-08-2009 12:55:26 -- Backup completed for datastore1 SRVTSE/SRVTSE.vmdk
08-08-2009 12:55:26 -- Backing up "datastore1 SRVTSE/SRVTSE_1.vmdk" to "NFS WEEKLY/SRVTSE/SRVTSE-2009-08-08/SRVTSE_1.vmdk"
08-08-2009 14:39:29 -- Backup completed for datastore1 SRVTSE/SRVTSE_1.vmdk
08-08-2009 14:39:30 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-08" not found
08-08-2009 14:39:39 -- Backup completed for "SRVTSE"!

08-08-2009 14:39:39 -- ERROR - Unable to locate VM: SRVEXCHANGE

============================== 08-08-2009 14:39:39 -- ghettoVCBg2 LOG END ==============================

1st: OK.
2nd: Snapshot not removed.
3rd: VM not found.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

adrian888 — Tue, 11 Aug 2009 23:56:06 GMT

Could this be a time stamp issue?

I was backing up my domain controller, and it usually only takes 2-3 minutes with no issues. I was testing a new NFS share and it took almost an hour, and had the error that it could not find the snap shot. I usually only have snap shot issues with my larger VM's that take at least an hour or more to backup.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

dwhittaker — Wed, 12 Aug 2009 21:59:54 GMT

I'm just lending my voice to state that I have experienced the same phenomenon as Al3xandr3. I haven't yet tried to determine if the order of running individual backups affects the result or if it's a specific fault of one of the VM's. I have experienced it with multiple VMs in one list and also individually listed VMs with manual starts. With all of my VM's in a single list, in order:

(1st) Win2k TS (not the assigned names, there are no spaces): OK
(2nd) Win2k3 App: Snapshot not removed. Once I found that the snapshot not only wasn't removed but also didn't show up in the VI Client to be deleted and required manual creation of a snapshot to then show 2 snapshots and deleting them then.
(etc) The rest: Backup doesn't begin

When I find the logs I'll review and post them up, hopefully correlating the same errors.

Environment is vSphere Essentials (1 ESX, 1 vCentre VM, vMA4. My VMs ranged from 15GB to 45GB at the time and backup time was not excessive. Brand new setup, overprovisioned atm and plenty of resources and speed. No activity during the time of backups which are stored on an iSCSI NAS.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

grrltech — Fri, 14 Aug 2009 21:22:18 GMT

Are there any plans to add compression to this version of ghettoVCB like there is in generation 1? Compression is key to our off-site backup regimen.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Sat, 15 Aug 2009 02:22:36 GMT

ghettoVCB compression is currently at an experimental support, no plans for g2 yet. There are still some outstanding issues that I've yet had the cycles to investigate for ghettoVCBg2 and I'm hoping I'll have some time before VMworld else it'll have to be after.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

dwhittaker — Sat, 15 Aug 2009 14:32:09 GMT

Seems I am getting the same errors in the logs as Al3xandr3.

Backup log:

============================== 08-15-2009 05:27:09 -- ghettoVCBg2 LOG START ==============================
08-15-2009 05:27:09 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-15-2009 05:27:09 -- CONFIG - VM_BACKUP_DATASTORE = Thecus_iSCSI_BkUp
08-15-2009 05:27:09 -- CONFIG - VM_BACKUP_DIRECTORY = vMA_Backups
08-15-2009 05:27:09 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-15-2009 05:27:09 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-15-2009 05:27:09 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-15-2009 05:27:09 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-15-2009 05:27:09 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-15-2009 05:27:09 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-15

08-15-2009 05:27:13 -- Initiate backup for "CapServer_New" found on myESX.domain.dc!
08-15-2009 05:27:24 -- CapServer_New original powerState: poweredOn
08-15-2009 05:28:14 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-15" for CapServer_New
08-15-2009 05:28:14 -- CapServer_New has 1 VMDK(s)
08-15-2009 05:28:14 -- Backing up "COS_Datastore CapServer_New/CapServer_New.vmdk" to "Thecus_iSCSI_BkUp vMA_Backups/CapServer_New/CapServer_New-2009-08-15/CapServer_New.vmdk"
08-15-2009 05:53:48 -- Backup completed for COS_Datastore CapServer_New/CapServer_New.vmdk
08-15-2009 05:53:50 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-08-15" for CapServer_New
08-15-2009 05:54:00 -- Backup completed for "CapServer_New"!

08-15-2009 05:54:01 -- Initiate backup for "BrisSteelADS01" found on myESX.domain.dc!
08-15-2009 05:54:11 -- BrisSteelADS01 original powerState: poweredOn
08-15-2009 05:58:10 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-15" for BrisSteelADS01
08-15-2009 05:58:10 -- BrisSteelADS01 has 2 VMDK(s)
08-15-2009 05:58:10 -- Backing up "COS_Datastore BrisSteelADS01/BrisSteelADS01.vmdk" to "Thecus_iSCSI_BkUp vMA_Backups/BrisSteelADS01/BrisSteelADS01-2009-08-15/BrisSteelADS01.vmdk"
08-15-2009 06:14:03 -- Backup completed for COS_Datastore BrisSteelADS01/BrisSteelADS01.vmdk
08-15-2009 06:14:03 -- Backing up "COS_Datastore BrisSteelADS01/BrisSteelADS01_2.vmdk" to "Thecus_iSCSI_BkUp vMA_Backups/BrisSteelADS01/BrisSteelADS01-2009-08-15/BrisSteelADS01_2.vmdk"
08-15-2009 06:31:22 -- Backup completed for COS_Datastore BrisSteelADS01/BrisSteelADS01_2.vmdk
08-15-2009 06:31:22 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-15" not found
08-15-2009 06:31:37 -- Backup completed for "BrisSteelADS01"!

08-15-2009 06:31:37 -- ERROR - Unable to locate VM: CapServer2_New

============================== 08-15-2009 06:31:37 -- ghettoVCBg2 LOG END ==============================

.
.
.
.
.
.
Debug log (after the fact, snapshot was not there prior to backup above):

============================== 08-15-2009 07:25:08 -- ghettoVCBg2 LOG START ==============================
08-15-2009 07:25:08 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-15-2009 07:25:08 -- CONFIG - VM_BACKUP_DATASTORE = Thecus_iSCSI_BkUp
08-15-2009 07:25:08 -- CONFIG - VM_BACKUP_DIRECTORY = vMA_Backups
08-15-2009 07:25:08 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-15-2009 07:25:08 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-15-2009 07:25:08 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-15-2009 07:25:08 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-15-2009 07:25:08 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-15-2009 07:25:08 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-15

08-15-2009 07:25:12 --

DEBUG INFO CapServer_New

08-15-2009 07:25:12 -- DEBUG - Host Build: VMware ESX 4.0.0 build-171294
08-15-2009 07:25:12 -- DEBUG - Host: myESX.domain.dc
08-15-2009 07:25:12 -- DEBUG - Virtual Machine: CapServer_New
08-15-2009 07:25:12 -- DEBUG - VM ConfigPath: COS_Datastore CapServer_New/CapServer_New.vmx
08-15-2009 07:25:12 -- DEBUG - VMX File: CapServer_New.vmx
08-15-2009 07:25:12 -- DEBUG - BackupConfigPath: Thecus_iSCSI_BkUp vMA_Backups/CapServer_New/CapServer_New-2009-08-15/CapServer_New.vmx
08-15-2009 07:25:12 -- DEBUG - BackupPath: Thecus_iSCSI_BkUp vMA_Backups/CapServer_New/CapServer_New-2009-08-15
08-15-2009 07:25:12 -- DEBUG - VM Datastore: COS_Datastore
08-15-2009 07:25:12 -- DEBUG - VMDK(s):
08-15-2009 07:25:12 -- DEBUG - COS_Datastore CapServer_New/CapServer_New.vmdk
08-15-2009 07:25:12 --

DEBUG INFO CapServer_New

08-15-2009 07:25:12 -- WARN - Snapshot found for BrisSteelADS01, backup will not take place

08-15-2009 07:25:12 --

DEBUG INFO CapServer2_New

08-15-2009 07:25:12 -- DEBUG - Host Build: VMware ESX 4.0.0 build-171294
08-15-2009 07:25:12 -- DEBUG - Host: myESX.domain.dc
08-15-2009 07:25:12 -- DEBUG - Virtual Machine: CapServer2_New
08-15-2009 07:25:12 -- DEBUG - VM ConfigPath: COS_Datastore CapServer2_New/CapServer2_New.vmx
08-15-2009 07:25:12 -- DEBUG - VMX File: CapServer2_New.vmx
08-15-2009 07:25:12 -- DEBUG - BackupConfigPath: Thecus_iSCSI_BkUp vMA_Backups/CapServer2_New/CapServer2_New-2009-08-15/CapServer2_New.vmx
08-15-2009 07:25:12 -- DEBUG - BackupPath: Thecus_iSCSI_BkUp vMA_Backups/CapServer2_New/CapServer2_New-2009-08-15
08-15-2009 07:25:12 -- DEBUG - VM Datastore: COS_Datastore
08-15-2009 07:25:12 -- DEBUG - VMDK(s):
08-15-2009 07:25:12 -- DEBUG - COS_Datastore CapServer2_New/CapServer2_New.vmdk
08-15-2009 07:25:12 --

DEBUG INFO CapServer2_New

============================== 08-15-2009 07:25:12 -- ghettoVCBg2 LOG END ==============================

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

yp1 — Sat, 15 Aug 2009 17:45:21 GMT

I get "Hard power off, VMware Tools is not installed" when try to use power off before backup feature.
The tools are there and VIC says "OK" for the tools status.
Has the feature ever been tested?
Another bug?
I thought to use the power off option it as a workaround since I had never succeeded backing up "hot" due to a known bug with non-deleted snapshots, but it does not seem to work properly either.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Sun, 16 Aug 2009 19:40:14 GMT

This feature has been tested and has worked. Please provide the following pieces of information on your environment:

1) What version of ESX(i)?
2) Are you using vCenter? What version? Are you using vMotion?
3) What version of VIMA/vMA are you running?
4) Please execute the following script to verify your VMware Tools status on your VMs: getVMToolsStatus.pl

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

datenfalke — Mon, 17 Aug 2009 07:02:02 GMT

Hello,

I would like to use this nice script but I also have the problem that the first VM gets backuped, the snapshot does not get deleted and the following VMs cannot be found.

I am using VIMA 1.0.

Here are the logfiles:

============================== 08-16-2009 15:37:44 -- ghettoVCBg2 LOG START ==============================
08-16-2009 15:37:44 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-16-2009 15:37:44 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
08-16-2009 15:37:44 -- CONFIG - VM_BACKUP_DIRECTORY = VM_BACKUPS
08-16-2009 15:37:44 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
08-16-2009 15:37:44 -- CONFIG - ADAPTER_FORMAT = buslogic
08-16-2009 15:37:44 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-16-2009 15:37:44 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
08-16-2009 15:37:44 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
08-16-2009 15:37:44 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-16

08-16-2009 15:37:50 --

DEBUG INFO vps006

08-16-2009 15:37:50 -- DEBUG - Host Build: VMware ESX Server 3i 3.5.0 build-123629
08-16-2009 15:37:50 -- DEBUG - Host: xx.xx.xx.xx
08-16-2009 15:37:50 -- DEBUG - Virtual Machine: vps006
08-16-2009 15:37:50 -- DEBUG - VM ConfigPath: datastore1 vps006/vps006.vmx
08-16-2009 15:37:50 -- DEBUG - VMX File: vps006.vmx
08-16-2009 15:37:50 -- DEBUG - BackupConfigPath: datastore1 VM_BACKUPS/vps006/vps006-2009-08-16/vps006.vmx
08-16-2009 15:37:50 -- DEBUG - BackupPath: datastore1 VM_BACKUPS/vps006/vps006-2009-08-16
08-16-2009 15:37:50 -- DEBUG - VM Datastore: datastore1
08-16-2009 15:37:50 -- DEBUG - VMDK(s):
08-16-2009 15:37:50 -- DEBUG - datastore1 vps006/vps006.vmdk
08-16-2009 15:37:50 --

DEBUG INFO vps006

08-16-2009 15:37:50 --

DEBUG INFO vps003

08-16-2009 15:37:50 -- DEBUG - Host Build: VMware ESX Server 3i 3.5.0 build-123629
08-16-2009 15:37:50 -- DEBUG - Host: xx.xx.xx.xx
08-16-2009 15:37:50 -- DEBUG - Virtual Machine: vps003
08-16-2009 15:37:50 -- DEBUG - VM ConfigPath: datastore1 vps003/vps003.vmx
08-16-2009 15:37:50 -- DEBUG - VMX File: vps003.vmx
08-16-2009 15:37:50 -- DEBUG - BackupConfigPath: datastore1 VM_BACKUPS/vps003/vps003-2009-08-16/vps003.vmx
08-16-2009 15:37:50 -- DEBUG - BackupPath: datastore1 VM_BACKUPS/vps003/vps003-2009-08-16
08-16-2009 15:37:50 -- DEBUG - VM Datastore: datastore1
08-16-2009 15:37:50 -- DEBUG - VMDK(s):
08-16-2009 15:37:50 -- DEBUG - datastore1 vps003/vps003.vmdk
08-16-2009 15:37:50 --

DEBUG INFO vps003

08-16-2009 15:37:50 --

DEBUG INFO vps001

08-16-2009 15:37:50 -- DEBUG - Host Build: VMware ESX Server 3i 3.5.0 build-123629
08-16-2009 15:37:50 -- DEBUG - Host: xx.xx.xx.xx
08-16-2009 15:37:50 -- DEBUG - Virtual Machine: vps001
08-16-2009 15:37:50 -- DEBUG - VM ConfigPath: datastore1 vps001/vps001.vmx
08-16-2009 15:37:50 -- DEBUG - VMX File: vps001.vmx
08-16-2009 15:37:50 -- DEBUG - BackupConfigPath: datastore1 VM_BACKUPS/vps001/vps001-2009-08-16/vps001.vmx
08-16-2009 15:37:50 -- DEBUG - BackupPath: datastore1 VM_BACKUPS/vps001/vps001-2009-08-16
08-16-2009 15:37:50 -- DEBUG - VM Datastore: datastore1
08-16-2009 15:37:50 -- DEBUG - VMDK(s):
08-16-2009 15:37:50 -- DEBUG - datastore1 vps001/vps001.vmdk
08-16-2009 15:37:50 --

DEBUG INFO vps001

============================== 08-16-2009 15:37:50 -- ghettoVCBg2 LOG END ==============================

============================== 08-16-2009 15:41:25 -- ghettoVCBg2 LOG START ==============================
08-16-2009 15:41:25 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-16-2009 15:41:25 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
08-16-2009 15:41:25 -- CONFIG - VM_BACKUP_DIRECTORY = VM_BACKUPS
08-16-2009 15:41:25 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
08-16-2009 15:41:25 -- CONFIG - ADAPTER_FORMAT = buslogic
08-16-2009 15:41:25 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-16-2009 15:41:25 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
08-16-2009 15:41:25 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
08-16-2009 15:41:25 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-16

08-16-2009 15:41:29 -- Initiate backup for "vps006" found on xx.xx.xx.xx!
08-16-2009 15:41:43 -- vps006 original powerState: poweredOn
08-16-2009 15:41:45 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-16" for vps006
08-16-2009 15:41:45 -- vps006 has 1 VMDK(s)
08-16-2009 15:41:45 -- Backing up "datastore1 vps006/vps006.vmdk" to "datastore1 VM_BACKUPS/vps006/vps006-2009-08-16/vps006.vmdk"
08-16-2009 16:26:19 -- Backup completed for datastore1 vps006/vps006.vmdk
08-16-2009 16:26:19 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-16" not found
08-16-2009 16:26:27 -- Backup completed for "vps006"!

08-16-2009 16:26:27 -- ERROR - Unable to locate VM: vps001

08-16-2009 16:26:27 -- ERROR - Unable to locate VM: vps003

============================== 08-16-2009 16:26:27 -- ghettoVCBg2 LOG END ==============================

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

adrian888 — Thu, 20 Aug 2009 14:41:13 GMT

Is this a valid perl statement, as I have not found an example of using it anywhere else.

line 699 - $ref = $subRef if ($subCount);

I have changed it to the following, and it look good so far.

if ($subCount) {
$ref = $subRef;
}

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Thu, 20 Aug 2009 16:43:54 GMT

It's a valid Perl statement, test it out for yourself.

Here's a quick example:

[vi-admin@scofield]$ cat test.pl 
#!/usr/bin/perl -w

use strict;
use warnings;

my $subCount = 1;
my $subRef = "hello world";
my $ref = "not hellow world";

$ref = $subRef if($subCount);

print $ref . "\n";

Run it and you tell me what it prints

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

yp1 — Mon, 24 Aug 2009 10:17:07 GMT

I figured the problem with VM not being powered down was in the Tools being out of date, so sorry for the false alarm.
However, now the VM does not being powered back up after the backup. Even though I can see the code is there.
The script just quits after the last VMDK backup by saying "Backup completed", but without the "LOG END".
What could be wrong?

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

vipernicus42 — Thu, 03 Sep 2009 18:03:27 GMT

Hey William,

First off, thanks so much for making these scripts. Between ghettoShutdown.pl and ghettoVCBg2.pl (if I ever get it working) I'll have everything I need to run ESXi without worry!

That being said... I'm having some issues with getting ghettoVCBg2.pl working

Environment Setup:

vmbox1.domain.com ESXi 3.5u3 host (free license) which has one guest
- "vmguest1.domain.com" - a winXP "workstation"
vmbox2.domain.com ESXi 3.5u3 host (free license) which has one guest
- vMA-vmbox2.domain.com - a vMA 1.0 virtual appliance

This is the same setup I was using with my questions over in the ghettoShutdown.pl thread, and that's working very well now thanks!

Anyway, it -appears- to work, but when it's all done the only thing that appears in the backup folder is a single *.vmx file that's 2.03kb and that's it. The whole process takes no more than two minutes as well, which is quite a short time for something that's 10gb to go from one datastore to another (which are on separate physical hard drives in the same box) Here's a dump of the log

============================== 09-03-2009 09:49:48 -- ghettoVCBg2 LOG START ==============================
09-03-2009 09:49:48 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
09-03-2009 09:49:48 -- CONFIG - VM_BACKUP_DATASTORE = datastore2
09-03-2009 09:49:48 -- CONFIG - VM_BACKUP_DIRECTORY = VMBackups
09-03-2009 09:49:48 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
09-03-2009 09:49:48 -- CONFIG - ADAPTER_FORMAT = buslogic
09-03-2009 09:49:48 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
09-03-2009 09:49:48 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
09-03-2009 09:49:48 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
09-03-2009 09:49:48 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-09-03

09-03-2009 09:50:26 -- Initiate backup for "vmguest1" found on vmbox1.domain.com!
09-03-2009 09:50:45 -- vmguest1 original powerState: poweredOn
09-03-2009 09:50:47 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-09-03" for vmguest1
09-03-2009 09:50:47 -- vmguest1 has 1 VMDK(s)
09-03-2009 09:50:47 -- Backing up "datastore1 vmguest1/vmguest1.vmdk" to "datastore2 VMBackups/vmguest1/vmguest1-2009-09-03/vmguest1.vmdk"
09-03-2009 09:50:53 -- Backup completed for datastore1 vmguest1/vmguest1.vmdk
09-03-2009 09:50:56 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-09-03" for vmguest1
09-03-2009 09:51:06 -- Backup completed for "vmguest1"!

============================== 09-03-2009 09:51:06 -- ghettoVCBg2 LOG END ==============================

As you can see the log looks fine. There's plenty of space on datastore2 (like 100gb and the VM I'm trying to backup is 10GB).

Any ideas why it's not creating/copying anything to the backup folder other than a single tiny *.vmx file?

Jon

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

mvdheuvel — Mon, 07 Sep 2009 11:07:20 GMT

Hi William,

First of all, works great. Easy to configure and very fast free alternative.
I have a question, is it possible to have the status mailed to me.
Case of a backup error, I like to have a email.

Menno

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

martingreiner — Mon, 07 Sep 2009 12:59:55 GMT

Hi

I will try as described but i am hanging by creating a NFS-datastore in my ESXi/vMA
If I understand correctly the NFS-datastore have to be mounted on vMA (central)
But how to do ?

regards
Martin

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

angelone — Mon, 07 Sep 2009 14:05:49 GMT

Hi Menno,
you could pipe the cron output to mail.

Just append something like

| mail -s "ghettoVCBg2 <your vMA server>" someone@mydomain.com

at the end of the cron entry.

But this way you will get the mail everytime the cronjob runs, not just in case of errors.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

angelone — Mon, 07 Sep 2009 14:08:53 GMT

@martin

You have to mount the datastore on your ESXi to backup to it.
You can do this via the vCenter.

The only case in which you have to mount the datastore in your vMA is when you want to store the vmlist files in some central location.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

martingreiner — Mon, 07 Sep 2009 14:36:37 GMT

Following Log, whats going wrong here, NFS_datastore is mounted on esxi03 !

============================== 09-07-2009 07:28:01 -- ghettoVCBg2 LOG START ==============================

09-07-2009 07:28:05 -- CONFIG - USING CONFIGURATION FILE = vMA
09-07-2009 07:28:05 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
09-07-2009 07:28:05 -- CONFIG - VM_BACKUP_DATASTORE = NFS_datastore
09-07-2009 07:28:05 -- CONFIG - VM_BACKUP_DIRECTORY = esxi03_backups
09-07-2009 07:28:05 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
09-07-2009 07:28:05 -- CONFIG - ADAPTER_FORMAT = lsilogic
09-07-2009 07:28:05 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
09-07-2009 07:28:05 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
09-07-2009 07:28:05 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
09-07-2009 07:28:05 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-09-07
09-07-2009 07:28:05 -- Initiate backup for "vMA" found on esxi03.softline.at!
09-07-2009 07:28:14 -- vMA original powerState: poweredOn
09-07-2009 07:28:14 -- vMA has 1 VMDK(s)
09-07-2009 07:28:14 -- Backing up "datastore1 vMA/vMA.vmdk" to "NFS_datastore esxi03_backups/vMA/vMA-2009-09-07/vMA.vmdk"
09-07-2009 07:28:18 -- Backup completed for datastore1 vMA/vMA.vmdk
09-07-2009 07:28:18 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-09-07" not found
09-07-2009 07:28:21 -- Unable to purge "NFS_datastore esxi03_backups/vMA/GET https:esxi03.softline.atfolderesxi03_backupsvMA?dsName=NFS_datastore unsuccessful : 404 Not Found" due to rotation max
09-07-2009 07:28:21 -- Backup completed for "vMA"!

============================== 09-07-2009 07:28:21 -- ghettoVCBg2 LOG END ==============================

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

ermasadmin — Thu, 17 Sep 2009 16:29:00 GMT

William,
Phenomenal work and effort on this project, and thanks for all your support for the community! I have just executed a few initial trials, and like Shawn and martingreiner I am seeing a "404 Not Found due to rotation max" error. I have vMA and ESXi 3.5 build 184236 along with ghettoVCBg2.pl version 2.5.1. I was wondering if you had an idea what might be causing this error?
(debug output below):

============================== 09-17-2009 08:21:03 -- ghettoVCBg2 LOG START ==============================
09-17-2009 08:21:03 -- CONFIG - BACKUP_LOG_OUTPUT = ghettoVCBg2.log
09-17-2009 08:21:03 -- CONFIG - VM_BACKUP_DATASTORE = DSS1-bkupvm-eth4
09-17-2009 08:21:03 -- CONFIG - VM_BACKUP_DIRECTORY = ghettoVCB
09-17-2009 08:21:03 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
09-17-2009 08:21:03 -- CONFIG - ADAPTER_FORMAT = buslogic
09-17-2009 08:21:03 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
09-17-2009 08:21:03 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
09-17-2009 08:21:03 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
09-17-2009 08:21:03 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-09-17

09-17-2009 08:21:08 -- Initiate backup for "actitime" found on 192.168.50.7!
09-17-2009 08:21:24 -- actitime original powerState: poweredOn
09-17-2009 08:21:24 -- actitime has 1 VMDK(s)
09-17-2009 08:21:24 -- Backing up "DSS1-bond0 actitime/actitime.vmdk" to "DSS1-bkupvm-eth4 ghettoVCB/actitime/actitime-2009-09-17/actitime.vmdk"
09-17-2009 08:21:30 -- Backup completed for DSS1-bond0 actitime/actitime.vmdk
09-17-2009 08:21:30 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-09-17" not found
09-17-2009 08:21:34 -- Unable to purge "DSS1-bkupvm-eth4 ghettoVCB/actitime/GET https:192.168.50.7folderghettoVCBactitime?dsName=DSS1-bkupvm-eth4 unsuccessful : 404 Not Found" due to rotation max
09-17-2009 08:21:34 -- Backup completed for "actitime"!

09-17-2009 08:21:34 -- ERROR - Unable to locate VM: operator

============================== 09-17-2009 08:21:34 -- ghettoVCBg2 LOG END ==============================

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

adrian888 — Mon, 21 Sep 2009 14:51:44 GMT

Hi All,

I have been hacking this script to find out what the problem is with the snap shots not being deleted and have changed the call from the internal function to the external script snapshotmanager.pl.

#&remove_snapshot($vm_view, $snapshot_name, $vm_name);

my $res = `/home/vi-admin/bin/snapshotmanager.pl -server vc1 --operation remove --vmname $vm_name --username ******* --password ******** --snapshotname $snapshot_name --children 0`;

print LOG timeStamp('MDYHMS'), "Remove Snapshot $snapshot_name from VM $vm_name with results: $res";

It seems like you need to close and reopen the connections to get a refreshed listing of the snap shots? I tried using delays etc, but did not have any consistent results till I tried the external call. This new call has been working the past 2 weeks on the VM's that failed previously.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Tue, 22 Sep 2009 02:56:29 GMT

This has been on my to-do list, I'm wondering if the connection eventually times out while it's executing the backup ... I have few others that are running into the same issue with removing the snapshot, but I haven't had a chance to replicate the issue. Drop me your email via VMTN PM and I'll send out an email this weekend regarding the next steps and perhaps a live support chat.

Thanks for your patience

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

bmckerritnu — Tue, 22 Sep 2009 05:38:17 GMT

I had this error and traced it (via perl debugger) to the 'restricted version' error. I had the script working perfectly and all of a sudden it stopped with a message like yours. Turns out that while I was evaluating the product it allows writeable CLI. I never used the script for a while, but sometime later I grabbed free licenses and applied it to all of my ESX4i servers. I'm guessing that this is what has changed the CLI to be read-only and therefor ghetto no longer works ! Can anyone confirm this ?

<RANT>
If this is true then I will for sure be moving all my servers over to Xenserver 5.5 sure it has restrictions also but none as ridiculous as ESXi and the read only CLI. It does have Xenmotion (like vmotion) Xencenter (like vcenter) and a writeable CLI unlike the 'free' ESXi.
</RANT>

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Tue, 22 Sep 2009 06:00:53 GMT

Take a look at this article for your answer: http://vmetc.com/2009/03/31/esxi-u4-ends-free-version-read-and-write-access-from-the-rcli/

The API was never meant for full read/write, this was a bug in VMware's code between U2 and U3 which was remediated in U4 and subsequent release in vSphere. I can understand where VMware's coming from, though I may not agree ... but you pay for you what you get. You still have the ability to use the unsupported SSH console for automating various set of tasks and licensed version of ESXi gives you more than just vCLI access and that is what makes them a leader in this space.

Good luck.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

bmckerritnu — Tue, 22 Sep 2009 06:14:48 GMT

Thanks William,

I had read the article previously and it did not specifically mention ESX4i. I suppose I am disappointed because your excellent script worked during the evaluation and now it doesn't.

I have enabled 'tech support' mode on all 3 servers but I fear that it too one day soon will disappear.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Tue, 22 Sep 2009 13:48:54 GMT

The article was published well before vSphere came out, but again this was an internal bug that VMware had and the API was never intended to be free (both read/write), so once this was fixed it would be the case for ESXi 4.0 as well.

Take a look at this comparison chart for the various licensing versions: http://www.vmware.com/products/vsphere/buy/editions_comparison.html as you can see even the simplest licensed version provides much value added than just full r/w access to the API.

Regarding the unsupported console in ESXi, it's VMware's direction to go to the ESXi platform and eventually the classic ESX will no longer be around ( I wouldn't hold my breath anytime soon, but eventually). They may eventually remove the Busybox console as well but again, it'll take time but enjoy it for now.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Kabs123 — Tue, 06 Oct 2009 12:28:37 GMT

Hi William,

when i run the script, i get the following output:

print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 150
print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 300
print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 300
print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 300
print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 198

ESXi 4 Enterprise, vMA 4.0, backup: nfs

Thank you in advance.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

angelone — Tue, 06 Oct 2009 15:47:49 GMT

hi, I suppose the script can't write to the directoy or to the logfile you specified.
Try /tmp/ghettoVCB2.log that should work.

regards,
Martin

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Thu, 08 Oct 2009 04:20:38 GMT

I wanted to give an update to everyone that may be using this script.

The issues that have been mentioned are being actively worked on, as you know I do support many many other scripts and there's only so many hours in each day I do assure you, I've not forgotten about you guys/gals and the issues being reported. I do ask for you patience and I'll provide an update as soon as I have one.

Stay tuned ...

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Charon4you — Fri, 09 Oct 2009 06:44:21 GMT

Hi all here!

I/We have 3 ESX4 machines with a shared SAN datastore (all machines can see all SAN datastores). One machine has a "big" Local Storage and I want to backup remote VMs to the Local Storage using SAN, not over a network. Is it possible to configure ghettoVCBg2 to backup VMs this way?

Thanks a lot

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Thu, 15 Oct 2009 04:21:09 GMT

Hi everyone,

I'm currently in development (as time permits) on some new features for the ghettoVCBg2 script and I'm looking for some individuals that may be interested in taking part in a BETA. The requirement is pretty simple, willing to help test and provide feedback on the next version of the script (remember to test on non-production VMs)

If you're interested, please PM on VTMN with your email address and the script you would like to take part in and I'll be selecting a group users and send out email notification in the coming weeks.

Thanks

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Kabs123 — Wed, 21 Oct 2009 09:51:47 GMT

When i make a backup from a big VM with 300-500 GB, the VM does not get started automatically after backup prozess.
What could be the matter of this? With smaller VMs it works.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Wed, 21 Oct 2009 13:14:47 GMT

This is a known issue with larger VMs and will be fixed in a future release.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Kabs123 — Wed, 21 Oct 2009 14:15:05 GMT

OK, thank you for the fast answer.
Do you already know what causes this issue? Maybe i can fix it byself.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Wed, 21 Oct 2009 16:28:47 GMT

Nope the solution is actually pretty complicated, we're pretty much there but requires further testing which will be part of the BETA. I won't go into further details, if you're already emailed me then you'll know in the coming weeks if you're one of the few few to be on the BETA.

Thanks for your patience.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Kabs123 — Wed, 21 Oct 2009 17:06:31 GMT

All right, i´ve sent you already an email last week for taking part in the BETA. I would help you willingly.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

bbaas75 — Wed, 21 Oct 2009 23:37:24 GMT

Hi,

I'm using ESXi 4.0, have a few virtual machines running, and trying to get the backup done. The thing is that I created all virtual machines with Thin disks. However, there are in all those VM's two vmdk's, one called <machinename>-flat.vmdk, and one called <machinename>.vmdk. When I do some filecopying and rename the <machinename>.vmdk to <machinename>.vmdk.old and the <machinename>-flat.vmdk to <machinename>.vmdk the backup seems to work, although I cannot find the backupfiles somewhere on my system. See the attached logs, can anybody help me out?

============================== 10-21-2009 15:44:02 -- ghettoVCBg2 LOG START ==============================
10-21-2009 15:44:02 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DIRECTORY = backup
10-21-2009 15:44:02 -- CONFIG - DISK_BACKUP_FORMAT = thin
10-21-2009 15:44:02 -- CONFIG - ADAPTER_FORMAT = lsilogic
10-21-2009 15:44:02 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
10-21-2009 15:44:02 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 15:44:02 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-10-21

10-21-2009 15:44:07 -- Initiate backup for "TS3" found on 192.168.72.113!
10-21-2009 15:44:14 -- TS3 original powerState: poweredOn
10-21-2009 15:44:14 -- TS3 has 1 VMDK(s)
10-21-2009 15:44:14 -- Backing up "datastore1 TS3/TS3.vmdk" to "datastore1 backup/TS3/TS3-2009-10-21/TS3.vmdk"
10-21-2009 15:44:18 -- Backup completed for datastore1 TS3/TS3.vmdk
10-21-2009 15:44:18 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-10-21" not found
10-21-2009 15:44:19 -- Unable to purge "datastore1 backup/TS3/GET https:192.168.72.113folderbackupTS3?dsName=datastore1 unsuccessful : 404 Not Found" due to rotation max
10-21-2009 15:44:19 -- Backup completed for "TS3"!

============================== 10-21-2009 15:44:19 -- ghettoVCBg2 LOG END ==============================

============================== 10-21-2009 16:00:29 -- ghettoVCBg2 LOG START ==============================
10-21-2009 16:00:29 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DIRECTORY = backup
10-21-2009 16:00:29 -- CONFIG - DISK_BACKUP_FORMAT = thin
10-21-2009 16:00:29 -- CONFIG - ADAPTER_FORMAT = lsilogic
10-21-2009 16:00:29 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
10-21-2009 16:00:29 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 16:00:29 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-10-21

10-21-2009 16:00:35 -- Initiate backup for "TS3" found on 192.168.72.113!
10-21-2009 16:00:42 -- TS3 original powerState: poweredOff
10-21-2009 16:00:42 -- TS3 has 1 VMDK(s)
10-21-2009 16:00:42 -- Backing up "datastore1 TS3/TS3.vmdk" to "datastore1 backup/TS3/TS3-2009-10-21/TS3.vmdk"
10-21-2009 16:00:45 -- Backup completed for datastore1 TS3/TS3.vmdk
10-21-2009 16:00:47 -- Backup completed for "TS3"!

============================== 10-21-2009 16:00:47 -- ghettoVCBg2 LOG END ==============================

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Thu, 22 Oct 2009 04:13:28 GMT

I'm not exactly sure what you're trying to do? Was the backup successful? Are you saying that you executed a backup of 1 VM (TS3) and after the backup, you manually renamed the files on the ESX(i) host? From the looks of the log, you should have your backup VM stored in /vmfs/volumes/backup/TS3/TS3-2009-10-21/ and note that if you browse your datastore using the vSphere Client that the *-flat.vmdk will not display, only the descriptor.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

bbaas75 — Thu, 22 Oct 2009 11:18:44 GMT

Hi,

First I tried to do a backup, just of the VM TS3. That resulted in the following:

============================== 10-21-2009 15:44:02 -- ghettoVCBg2 LOG START ==============================
10-21-2009 15:44:02 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DIRECTORY = backup
10-21-2009 15:44:02 -- CONFIG - DISK_BACKUP_FORMAT = thin
10-21-2009 15:44:02 -- CONFIG - ADAPTER_FORMAT = lsilogic
10-21-2009 15:44:02 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
10-21-2009 15:44:02 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 15:44:02 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-10-21

10-21-2009 15:44:07 -- Initiate backup for "TS3" found on 192.168.72.113!
10-21-2009 15:44:14 -- TS3 original powerState: poweredOn
10-21-2009 15:44:14 -- TS3 has 1 VMDK(s)
10-21-2009 15:44:14 -- Backing up "datastore1 TS3/TS3.vmdk" to "datastore1 backup/TS3/TS3-2009-10-21/TS3.vmdk"
10-21-2009 15:44:18 -- Backup completed for datastore1 TS3/TS3.vmdk
10-21-2009 15:44:18 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-10-21" not found
10-21-2009 15:44:19 -- Unable to purge "datastore1 backup/TS3/GET https:192.168.72.113folderbackupTS3?dsName=datastore1 unsuccessful : 404 Not Found" due to rotation max
10-21-2009 15:44:19 -- Backup completed for "TS3"!

============================== 10-21-2009 15:44:19 -- ghettoVCBg2 LOG END ==============================

The backup directory was still empty. So, i tried to get info via Google, and saw something about two VMDK-files. So, I thought probably I can rename the TS3-flat.vmdk to TS3.vmdk. I did (made a saveset first) and then the second run:

============================== 10-21-2009 16:00:29 -- ghettoVCBg2 LOG START ==============================
10-21-2009 16:00:29 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DIRECTORY = backup
10-21-2009 16:00:29 -- CONFIG - DISK_BACKUP_FORMAT = thin
10-21-2009 16:00:29 -- CONFIG - ADAPTER_FORMAT = lsilogic
10-21-2009 16:00:29 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
10-21-2009 16:00:29 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 16:00:29 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-10-21

10-21-2009 16:00:35 -- Initiate backup for "TS3" found on 192.168.72.113!
10-21-2009 16:00:42 -- TS3 original powerState: poweredOff
10-21-2009 16:00:42 -- TS3 has 1 VMDK(s)
10-21-2009 16:00:42 -- Backing up "datastore1 TS3/TS3.vmdk" to "datastore1 backup/TS3/TS3-2009-10-21/TS3.vmdk"
10-21-2009 16:00:45 -- Backup completed for datastore1 TS3/TS3.vmdk
10-21-2009 16:00:47 -- Backup completed for "TS3"!

============================== 10-21-2009 16:00:47 -- ghettoVCBg2 LOG END ==============================

Still the backup directory was empty, but the log seems successfull. Can you point me in the right direction?

Regards,

Bas

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Fri, 23 Oct 2009 04:26:32 GMT

BETA email invitations will be going out this weekend ... keep an eye out

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Koldijk — Fri, 23 Oct 2009 06:48:21 GMT

I have installed this wonderfull script. Keep up the good work!

I am having trouble with crontab. My crontab (vi-admin) looks like:
vi-admin@vma ~$ crontab -l
0 0 * * * /home/vi-admin/ghettoVCBg2.pl --config_dir /home/vi-admin/backupConfigs --vmlist /home/vi-admin/virtual_machne_back_list_belangrijk
0 10 * * 7 /home/vi-admin/ghettoVCBg2.pl --config_dir /home/vi-admin/backupConfigs --vmlist /home/vi-admin/virtual_machne_back_list
0 5 * * 1-5 /home/vi-admin/ghettoVCBg2.pl --config_dir backupConfigs --vmlist /home/vi-admin/virtual_machne_back_list_branderweg

But cron doesn't run it at all. Did I missed something? When I use the lines at the shell the commands make great backups.
a tail of /var/log/cron gives:
~~vi-admin@vma ~~~$ sudo tail /var/log/cron
Password:
Oct 23 00:01:01 localhost crond14745: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 01:01:01 localhost crond14987: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 02:01:01 localhost crond15229: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 03:01:01 localhost crond15471: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 04:01:01 localhost crond15713: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 05:01:01 localhost crond15955: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 06:01:01 localhost crond16224: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 07:01:01 localhost crond16466: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 08:01:01 localhost crond16708: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 08:26:49 localhost crond2074: (CRON) STARTUP (V5.0)

Why cron isn't running my backups?

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

BasicVirtu — Tue, 27 Oct 2009 17:25:17 GMT

I am trying to backup vms in an ESX4 host that has mostly SAN based storage to an ESXi server with local storage over the network to a remote location. It's not clear to me whether I need to mount an NFS store on the ESX4 host or if the store still needs to be visible to the ESX4 host or if the vMA should take care of this as long as it is listed as being managed by the vMA. Do I still need to mount the NFS store manually on each ESX4 host so that $VM_BACKUP_DATASTORE that I'm backing up to is visible to that individual ESX4 host?

I think I'm just confused on the syntax for the datastore or how the script works with vMA, I verified that both hosts are being managed by vMA with sudo vifp listservers and I have tried several different names for the $VM_BACKUP_DATASTORE value. Everything in the logs says the backup completes but on the ESX4 host it keeps saying invalid datastore specified so it is clear it doesn't know what datastore I am trying to backup to. I have also tried specifying the ESXi server that I am trying to backup to with the --server command option but doesn't seem to make a difference.

I’ve tried several different values and no luck.

my $VM_BACKUP_DATASTORE = "/vmfs/volumes/backups";

my $VM_BACKUP_DATASTORE = "https://172.16.x.x/folder?dcPath=ha-datacenter&dsName=backups";

my $VM_BACKUP_DATASTORE = "172.16.x.x/folder?dcPath=ha-datacenter&dsName=backups";

my $VM_BACKUP_DATASTORE = "172.16.x.x";

Thanks again for all your work any help is apprieciated!

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Tue, 27 Oct 2009 17:51:26 GMT

The NFS datastore MUST be mounted on the ESX(i)host and must be visible to the host, vMA will instruct the various operations using the vSphere API to perform the backup to the NFS datastore or to any valid datastore for that matter.

Your syntax is for VM_BACKUP_DATASTORE is incorrect, you apparently did not go through the documentation throughly. The entry should just be the name of your datastore (e.g. datastore1)

If your NFS datastore is mounted on the ESX(i) host as backups then variable VM_BACKUP_DATASTORE should be set to backups

I would recommend going over the documentation and read through it carefully as much of your questions can be answered.

Thanks

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

BasicVirtu — Tue, 27 Oct 2009 17:53:33 GMT

Thank you that helps alot!

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Koldijk — Mon, 09 Nov 2009 12:33:47 GMT

Hi William,

Every night I am backing up my vm's with the script. with 3 of them the snapshot is not found and not deleted. For your information these Vm's are greater dan 30GB..
See the log file.

11-08-2009 00:52:06 -- CONFIG - USING CONFIGURATION FILE = UPDATES
11-08-2009 00:52:06 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-08-2009 00:52:06 -- CONFIG - VM_BACKUP_DATASTORE = vmbackup
11-08-2009 00:52:06 -- CONFIG - VM_BACKUP_DIRECTORY = vmware
11-08-2009 00:52:06 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
11-08-2009 00:52:06 -- CONFIG - ADAPTER_FORMAT = lsilogic
11-08-2009 00:52:06 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-08-2009 00:52:06 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-08-2009 00:52:06 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-08-2009 00:52:06 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-08
11-08-2009 00:52:06 -- Initiate backup for "UPDATES" found on 10.3.1.25!
11-08-2009 00:52:16 -- UPDATES original powerState: poweredOn
11-08-2009 00:52:18 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-11-08" for UPDATES
11-08-2009 00:52:18 -- UPDATES has 1 VMDK(s)
11-08-2009 00:52:18 -- Backing up "NFS updates/UPDATES_ESXi1.vmdk" to "vmbackup vmware/UPDATES/UPDATES-2009-11-08/UPDATES_ESXi1.vmdk"
11-08-2009 02:03:40 -- Backup completed for NFS updates/UPDATES_ESXi1.vmdk
11-08-2009 02:03:40 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-11-08" not found
11-08-2009 02:03:55 -- Backup completed for "UPDATES"!

And a other:
============================== 11-09-2009 12:29:04 -- ghettoVCBg2 LOG START ==============================

11-09-2009 12:29:09 -- Initiate backup for "Helpdesk" found on 10.3.1.24!
11-09-2009 12:29:21 -- Helpdesk has 1 VMDK(s)
11-09-2009 13:07:02 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-11-09" not found
11-09-2009 13:07:18 -- Backup completed for "Helpdesk"!

============================== 11-09-2009 13:07:22 -- ghettoVCBg2 LOG END ==============================

How is the Beta going? Can I help?

Regards,

Rob

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Tue, 10 Nov 2009 05:09:15 GMT

Hi Rob,

As mentioned in previous replies, the issue with backing up large VMs has been resolved in the next release of the script. I can't say much more but keep an eye out for the next version of the script soon

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

adrian888 — Thu, 19 Nov 2009 01:27:58 GMT

Hi,

to get the email to work I had to change line # 302
my $smtp = Net::SMTP->new($EMAIL_HOST ,Hello => 'vMA - ghettoVCBg2',Timeout => 30,);
to
my $smtp = Net::SMTP->new($EMAIL_HOST,Hello => 'mycompanydomain.com',Timeout => 30,);
or just do this
my $smtp = Net::SMTP->new($EMAIL_HOST);

I was always getting the error that it could not connect if I did not put my local company domain in.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Thu, 19 Nov 2009 03:49:12 GMT

Thanks for the info! I thought I had it working without specifying the entry and using a dummy string but I guess not .... well I've added an additional variable that'll specify the email domain and if you don't want to specify it, it'll default to localhost.localdomain as defined here: http://perldoc.perl.org/Net/SMTP.html

Glad to hear everything else is good

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

adrian888 — Thu, 19 Nov 2009 07:00:25 GMT

Great! Thanks for the quick fix.

Everything else is working great, no more snapshot problems! I also really like the selective backups!

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Koldijk — Thu, 19 Nov 2009 10:11:03 GMT

Hello William,

Thanks fore the new script! I have downloaded the new script and give the backup a GO.
Running bt hand gives me:
~~vi-admin@vma ~~~$ ./ghettoVCBg2.pl --config_dir /home/vi-admin/backupConfigs --vmlist helpdesk
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
~~vi-admin@vma ~~~$

the /tmp/ghettoVCBg2.log:

~~vi-admin@vma ~~~$ tail -f /tmp/ghettoVCBg2.log

11-19-2009 10:28:34 -- info: Initiate backup for Helpdesk found on 10.3.1.24
11-19-2009 10:28:44 -- debug: Helpdesk original powerState: poweredOn
11-19-2009 10:28:44 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2009-11-19" for Helpdesk
11-19-2009 10:28:54 -- info: Helpdesk has 1 VMDK(s)
11-19-2009 10:28:54 -- debug: backupVMDK: Backing up "NFS Helpdesk/Helpdesk.vmdk" to "vmbackup vmware/Helpdesk/Helpdesk-2009-11-19/Helpdesk.vmdk"
11-19-2009 10:28:54 -- debug: backupVMDK: Signal copyThread to start
11-19-2009 10:28:54 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
11-19-2009 10:28:54 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
11-19-2009 10:28:54 -- debug: CopyThread: Start backing up VMDK(s) ...
11-19-2009 10:43:57 -- debug: backupVMDK: Backup progress: Elapsed time 15 min
11-19-2009 10:59:00 -- debug: backupVMDK: Backup progress: Elapsed time 30 min
11-19-2009 11:04:22 -- debug: copyTask: send copySuccess message ...
11-19-2009 11:04:22 -- debug: copyTask: waiting for next job and sleep ...
11-19-2009 11:05:02 -- debug: backupVMDK: Successfully completed backup for NFS Helpdesk/Helpdesk.vmdk Elapsed time: 36 min
11-19-2009 11:05:18 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2009-11-19" for Helpdesk
11-19-2009 11:05:21 -- debug: checkVMBackupRotation: Starting ...
11-19-2009 11:05:40 -- debug: Purging vmbackup vmware/Helpdesk/Helpdesk-2009-11-13--7 due to rotation max
11-19-2009 11:05:40 -- info: Backup completed for Helpdesk!

11-19-2009 11:05:40 -- debug: Main: Disconnect from: 10.3.1.24

11-19-2009 11:05:40 -- debug: Main: Login by vi-fastpass to: 10.3.1.25
11-19-2009 11:05:43 -- debug: Main: Disconnect from: 10.3.1.25

11-19-2009 11:05:43 -- debug: Main: Login by vi-fastpass to: 10.3.1.26
11-19-2009 11:05:45 -- debug: Main: Disconnect from: 10.3.1.26

11-19-2009 11:05:45 -- debug: Main: Calling final clean up
11-19-2009 11:05:45 -- debug: cleanUP: Thread clean up starting ...
11-19-2009 11:05:45 -- debug: cleanUp: Send exit to copyThread
11-19-2009 11:05:45 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit
11-19-2009 11:05:45 -- debug: copyTask: die ...
11-19-2009 11:05:45 -- debug: cleanUp: Join passed
11-19-2009 11:05:45 -- info: ============================== ghettoVCBg2 LOG END ==============================

Do you have a idea? The backup is running fine, with a large disk

Thanks!
kind regards,
Rob

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Thu, 19 Nov 2009 17:44:34 GMT

It sounds like one of the logging variables is not configured correctly, can you please output one of the VM backup policy? The only supported logging option is either "info" or "debug" and hopefully that's defined properly in all your configs.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Koldijk — Fri, 20 Nov 2009 06:36:16 GMT

William,

Thanks, you where right. I still used "verbose".
I'll change that right away.

Regards,

Rob

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

deniseche — Fri, 20 Nov 2009 10:58:03 GMT

Nice Script and Nice work !
I had trouble with previous version.
Now it's works fine ! I can backup all my vm and i receive a report status in my mailbox everytime !

Thanks a lot William.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Fri, 20 Nov 2009 15:57:51 GMT

Yea that was changed in this release with just having two logging options. Glad to see you got it working

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Fri, 20 Nov 2009 15:58:09 GMT

Awesome to hear!

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

boelama — Sat, 21 Nov 2009 09:39:11 GMT

hi all ,
I just installed this fine tool to evaluate is as main backup tool
(after being disapointed of other commercial tools)

i was wondering if it would be possible to add a feature to execute a script add the end of each VM backup
something like

POST_BACKUP_SCRIPT=/home/vi-admin/myscript.sh
POST_BACKUP_SCRIPT_ONESX=true

this would copy the script to the ESX/ESXi server and execute it after each backup.
i would provide 2 parameters :
- path of backup location
- name of VM

we could use this to compress the files in a rsync friendly way
so we can transfer the "big" files to a offsite storage for save keeping

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

roadfox — Sat, 21 Nov 2009 10:47:42 GMT

Like the idea of post script. Another idea might be to add another option --vm vmname so i could create a backup script like:

./ghettoVCBg2.pl --vm vm_to_backup
my_post_script vm_to_backup
./ghettoVCBg2.pl --vm next_vm_to_backup
my_post_script next_vm_to_backup
....

so one could then choose if he wants to use --vmlist or just --vm

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Sat, 21 Nov 2009 17:17:42 GMT

This kind of defeats the purpose of the script not requiring any SSH connections to either the classic ESX or unsupported busybox console on ESXi, else a user could just use [ghettoVCB.sh|ghettoVCB.sh - Free alternative for backing up VM's for ESX(i) 3.5 and 4.0+ which does support compression.

If you're looking for such functionality, I would recommend you checking out VMware's vDR or Veeams Backup/Recovery (which I've heard great things about, especially supporting change block tracking). There are no compression support within the VMware APIs ... minus change block tracking which I'm sure is not a trivial task to implement into a backup solution.

Having said that, you're more than welcome to modify/tweak the script to fit your environment but this is something that will not be supported.

Thanks for the comments.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Sat, 21 Nov 2009 17:20:58 GMT

You also fail to understand the point of having either a global configuration for all VMs to be backed up or individual VM policy ...by having specifying individual VM to be backed up on the command line, you've introduced additional overhead which can be solved by using individual VM backup policy.

If the suggestion of allowing some type of post script to execute, this config would be stored in the VM policy file and you would still have 1 global vm list to backup but the specific params to each VM would be store in a configuration file which is why -c option is for.

Again, you can take a look at my previous reply, this defeats the purpose of this specific script not utilizing SSH console on either ESX(i) and you're more than welcome to modify/tweak the script to fit your environment but this is something that will not be supported.

Thanks for the commands.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

boelama — Sun, 22 Nov 2009 07:56:32 GMT

It was not my intention to break the "no SSH" requirements. i'm not yet on speed on what is possible with all
API's but my tough where going in the way of coping the script to the ESX(i) server and execute it there because
there is access to the backup files .

thinking longer on the subject , i guess that it would be hard to create scripts that can work in ESX/ESXi environment.
especially because ESXi is a very stripped busybox environment and VMWare would like it to be misused.

luc

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

roadfox — Sun, 22 Nov 2009 09:34:28 GMT

What i fail to understand is what is option -c for, can't find a description for it.
My problem is that at the moment i have to maintain a configuration in the config dir and a vmlist file for each vm. My recomendation would just avoid this:

"Note: --vmlist is still a required parameter with the exception that backup policy files must exist for each VM in the list else the VM will be ignored in the backup process."

What i would like to achieve is that we just need to maintain one configuration file per backup policy (not per vm, thats to much work) and then call ghettoVCBg2 with the right policy for the right vm.

The postscripts im using do not connect to the ESXi host, they do manipulations in the NFS filer which is the datastore.

RE: ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

lamw — Sun, 22 Nov 2009 17:48:26 GMT

If you don't understand the -c which specifies a configuration directory for your individual VM backup policy, you may want to go through the documentation carefully and I've provided some example output to show you each of the options.

With your logic, you're effectively duplicating the number of files that need to be created and maintain to accomplish the exact same task that is already available in the script today.

There are two core use cases that I've designed the script to handle:

1) Global configuration where all VMs are backed up in the same manner (1 vmlist, no need for -c option)
2) Individual VM backup policy, where you MUST create individual policy on a per VM basis, as each VM differs from one another (1 vmlist, -c option required)

What i would like to achieve is that we just need to maintain one configuration file per backup policy (not per vm, thats to much work) and then call ghettoVCBg2 with the right policy for the right vm.

That is effectively option #2 and again running any type of post/pre scripts is not supported as this defeats the purpose of the script not requiring any type of SSH connectively to the ESX(i) host as mentioned in the last replies.

Again, you're more than welcome to add/modify the script to fit your needs, its definitely doable but will not be supported.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".