ghettoVCB.sh - Free alternative for backing up VM's for ESX(i) 3.5, 4.x, 5.x, 6.x & 7.x

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

romgo75 · ‎04-23-2010

Hello,

I found a bug in the script. This bug cause the script to be unable to delete the backup folder.

The default naming convention is :

VM_BACKUP_DIR_NAMING_CONVENTION="$(date +%F_%k-%M-%S)"

this definition induce a space in the folder name from 1am to 9am

I replaced it by :

VM_BACKUP_DIR_NAMING_CONVENTION="$(date +%F_%H-%M-%S)"

And everything seems to be fine.

lamw · ‎04-23-2010

romgo75,

Thanks for bringing this to my attention, you're right, must have fat fingered that entry. The issue has been resolved and I've just uploaded the latest version. Please download the latest version and let me know if you run into any other issues.

Thanks

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

aliog · ‎04-23-2010

Its my first visit to this thread for some time, as the ghettoVCB script has been running perfectly on my ESXi 3.5 setup for months. We are now in the process of migrating to ESX 4 so I thought I would drop by.

Nice to see more improvements have been added since my last visit. I am testing the latest version on our new setup and all seems to be working great so far.

Thanks again lamw

tbsky · ‎04-23-2010

Dear lamw:

I have used this script over 1 year. it's really great. I like the new idea to seprate config file and program, so I try the new script today. however the script failed when I put "ENABLE_NON_PERSISTENT_NFS=1" to the external config file.

after checking the script, I found the script create the nfs mount at the "sanityCheck" stage. but the external config have not read yet at that stage, it will read later. so the script failed. hope it can be fixed so we can enjoy the

external settings with "ENABLE_NON_PERSISTENT_NFS=1"

and could you add "monosparse" disk format? although it didn't show in the vmkfstools command help, but that format exists at least from 3.5 to current 4.0. compare to "2gbsparse", "monosparse" is much easier to handle for rsync or rdiff-backup for second phase backup. I have used monosparse since 3.5. and it is really good.

thanks again for your kindly help!!

vlho · ‎04-23-2010

Hi lamw

thank you for new option: global configuratin file,

however I have same problem as tbsky.

Backup does not function if:

- ENABLE_NON_PERSISTENT_NFS=1 is defined in global configuration file

and

- NFS datastore does not mount manual before start of script

I think that script does not mount NFS datastore if script is running with parameter -g

I tested that script mounts NFS storage OK if ENABLE_NON_PERSISTENT_NFS=1 is defined in main script

Thanks

Feature request:

Could you extend the script for sending any log file by email in VMware vSphere?

lamw · ‎04-23-2010

Roger that, I'll take a look when I get chance, little busy this weekend but try to get something out by Sunday.

Email is not part of the scope, do a search online for email implementation. I know there are some hacks for classic ESX but not sure about ESXi and the unsupported console. I would recommend if you need this, you can dump the logs on NFS datastore and have a remote system pick them up and email. I know you can also hack around the local logger to have it direct the output to syslog and have that system handle it. In either scenario, it's not going to be handled by the script.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

lamw · ‎04-24-2010

The bug has been resolved, please give the latest version a try and let me know.

FYI - Also enhanced the script to allow for a VM exclusion list

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

mattiarambelli · ‎04-26-2010

Hi, i'm having a strange problem with ghettovcb/esxi4:

the first backup went well, but i haven't snapshots associated with the VM and ghettovcb fails for the "snapshot found" reason.

I looked in the iSCSI disk where the vm is and it's structured like there's a backup, but the snapshot manager doesn't see it, nor it's removed by vim-cmd command.

Have someone of you had my same problem with iSCSI disks?

If i enable the poweroff backup, will it backup well even if there are the vm-000000n disks?

thank you for your help

Mattia Rambelli

tbsky · ‎04-26-2010

Dear lamw:

the script now runs fine. but it show an error "./ghettoVCB.sh: line 798: cannot create : nonexistent directory".

after tracing, that error comes from the "logger" function. when it is called by "reConfigureGhettoVCBConfiguration()". it wants to log to $ but the variable didn't exist at that stage. it seems the chicken-egg problem. maybe "logger" function can just log to stdout if $ is empty?

and I find the script now need the "adapter" info. can we just skip that part if we need to keep the original adapter, no mater what type(buslogic or ide or ...) it is?

thanks a lot for your help!!

lamw · ‎04-26-2010

Odd, I ran through few of my tests and did not run into the problem using the latest version of the script from 04/24/2010

I've just updated where the global configuration gets set, can download the latest version and give it a try. If you still continue to run into issues for whatever reason, can you provide a little background on the configuration you have setup? (e.g. options you're using and if this is on ESX or ESXi/etc.) - Please check out FAQ #1 if you have more questions.

Regarding the adapter, that has always been a required parameter, if you did not explicitly set it, it relies on the default which is buslogic. I'm assuming you haven't been modifying the script?

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

chrb · ‎04-26-2010

Where is the -a "backup all VMs" option mentioned above? It does not seem to be in the actual ghettoVCB.sh:

./ghettoVCB.sh -a

./ghettoVCB.sh: line 729: usage: not found

There is no -a listed in options:

-f List of VMs to backup

-c Configuration directory for VM backups

-l File to output logging

-d Debug level info (default: info)

chrb · ‎04-26-2010

Ok forget this, the site must have served up an old version of the script - I have re-clicked and got the new version with -a.

tbsky · ‎04-26-2010

Dear lamw:

sorry I didn't make it clear. maybe you don't use global configuration file, so you didn't encounter the error.

I use global cofiguration file to backup every vm. my backup command is "ghettoVCB.sh -f backup_list -g ghettoVCB.conf". with that config, the newest script will read global config at sequence below

line 145: reConfigureGhettoVCBConfiguration "$" line 279: logger "info" "CONFIG - USING GLOBAL GHETTOVCB CONFIGURATION FILE = $"

line 38: if ........... then

at line 38, LOG_TO_STDOUT is not exist yet, so it go to:

line 50: echo -e "$ -- $: $" >> "$" $ is not exist either, so script give out error message

I suggest maybe modify line 38 to check empty $ to prevent chick-egg problem.

and my ghettoVCB.conf:

DISK_BACKUP_FORMAT=2gbsparse

VM_BACKUP_DIR_NAMING_CONVENTION=image

VM_BACKUP_ROTATION_COUNT=1

POWER_VM_DOWN_BEFORE_BACKUP=0

ENABLE_HARD_POWER_OFF=0

SNAPSHOT_TIMEOUT=30

ENABLE_COMPRESSION=0

ADAPTER_FORMAT=lsilogic

VM_SNAPSHOT_MEMORY=0

VM_SNAPSHOT_QUIESCE=1

ENABLE_NON_PERSISTENT_NFS=1

UNMOUNT_NFS=1

NFS_SERVER=10.1.1.1

NFS_MOUNT=/share/nfs

NFS_LOCAL_NAME=nfs_storage_backup

NFS_VM_BACKUP_DIR=backup

and about "adapter" type. old script didn't use "-a" when doing vmkfstools, it just respect the original adapter type. that's the behavior I want. because I just want my backup to use the same adapter type as original, no matter it is buslogic, lsilogic, or ide.

about modification, now I only modify the script a little to use "monosparse" instead of "2gbsparse".

I can live with that I think. it seems few people need "monosparse" backups..

thanks again for your kindly help!!

lamw · ‎04-26-2010

Okay, I've moved the output of using the global configuration further down into the script. All that output is used for is for me to know that someone is using a global conf and mainly around for troubelshooting. The fix should resolve the problem and if not, please let me know.

I've actually had requests about specifying the adapter type and probably was added in one of the subsequent release. It may have been in the very early version of the script that adapter type was not something you can specify.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

MatthiasLuraTec · ‎04-27-2010

Hi guys

I've got a new problem:

Failed to clone disk : The system cannot find the file specified (25)

Thats the log entry.

I've searched the web and found that if this issue occurs I have to take the spaces from the filenames.

So far so good, I have done this in every filename, .vmdk, .vmx , .vmxf but the error still occurs.

The VM boots up again it can still work but I cant take a backup.

What can I do?

Thanks for your replies

Regards

tbsky · ‎04-27-2010

Dear lamw:

now the script works perfect for me. thanks again for your great help!!

lamw · ‎04-27-2010

Awesome, glad to hear!

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

lamw · ‎04-27-2010

Please read FAQ #1

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

scowse · ‎04-27-2010

I have seen no evidence that the "adapter type" setting has ever changed the adapter type on a backed up vm.

Anyone seen this do anything?

I do not actually want it to change but there seems to be a lot of confusion about this setting and it's effect.

tohe · ‎04-28-2010

Hi,

yesterday I tried the new version of ghettoVCB with the exclusion list feature. This worked for me on one of two servers. On that one where it did not worked the script tried to backup all VMs although I defined one VM not to backup.

What can be the reason for that problem?

One other problem I have on that server. It is an error I get for one of the VMs "2010-04-27 23:30:04 -- info: Error: failed to locate and extract VM_ID for ..." What can cause this error?

Are there any naming conventions for the VMs? I ask because we have names with "/" and "(...)"?

Thanks, Tom

chrb · ‎04-28-2010

Removing snapshot hangs:

Clone: 100% done.

2010-04-28 09:16:42 -- info: Removing snapshot from Web_TESTBOX6 ...

It should take about 10 seconds but I leave it for 20 minutes before quitting. I see this has already been reported several times. Is there any solution?

I take it that david pyer's solution above (387 $ vmsvc/snapshot.removeall $ > /dev/null 2>&1) will remove all snapshots rather than just the one made for backup?

Edit: Also, it would be nice if a single host backup failing like this didn't end the backup when using -a... if problems with backing up a single host were handled better so the script would continue on and still backup the other hosts. At the moment I am waiting an hour for a failure, and hosts following the ones that hangs are not being backed up.

lamw · ‎04-28-2010

This may be related to FAQ #18 and #29. Are you running ESXi and using and NFS datastore to backup your VMs? If you are in fact hitting these issues related to the "NFS" issue, there's not much that can be done since the backups do complete for the existing VM but something causes the script to hang and everything has pointed back to either the ESXi host or the NFS server.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

lamw · ‎04-28-2010

I suspect it's most likely due to your naming convention for your VMs, perhaps the "/" is causing some issues. In general, I don't recommend using odd characters for your VM names. Even though Windows allows it, I prefer to use things like "-" or "_" or even maybe "." to separate out a name.

I would need to test to see if this is the case, but it should be trivial for you to test by properly renaming the VM not only the display name but the actual files backing the VM and re-run your backup. You can also try running this in dryrun mode and see if it's able to properly parse our your VM, I would expect that it may not show up in the dryrun as it's not able to search for the VM due to the special chars.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware KB - How do snapshots work?

If you find this information useful, please award points for "correct" or "helpful".

chrb · ‎04-29-2010

GhettoVCB.sh hangs on one specific VM persistently when attempting to remove the snapshot. The weird thing is that this VM has snapshots and should not be being backed up anyway. VMX_DIR has:

 
 # ls -l /vmfs/volumes/datastore4/TESTBOX6
-rw-------    1 root     root         8589934592 Mar 11 17:38 TESTBOX5-000001-flat.vmdk
-rw-------    1 root     root                503 Mar 11 17:38 TESTBOX5-000001.vmdk
-rw-------    1 root     root         8589934592 Mar 11 17:38 TESTBOX5-000002-flat.vmdk
-rw-------    1 root     root                503 Mar 11 17:38 TESTBOX5-000002.vmdk
-rw-------    1 root     root         8589934592 Mar 11 17:33 TESTBOX5-000004-flat.vmdk
-rw-------    1 root     root                503 Mar 11 17:33 TESTBOX5-000004.vmdk
-rw-------    1 root     root         8589934592 Apr 29 08:45 TESTBOX5-000005-flat.vmdk
-rw-------    1 root     root                503 Apr 29 08:35 TESTBOX5-000005.vmdk
-rw-------    1 root     root          268435456 Mar 12 10:56 TESTBOX5-425ed04f.vswp
-rw-------    1 root     root          269643751 Mar 11 17:12 TESTBOX5-Snapshot7.vmsn
-rw-------    1 root     root         8589934592 Mar 11 16:54 TESTBOX5-flat.vmdk
-rw-------    1 root     root               8684 Apr 29 02:15 TESTBOX5.nvram
-rw-------    1 root     root                470 Mar 11 17:05 TESTBOX5.vmdk
-rw-------    1 root     root               2358 Apr 29 08:35 TESTBOX5.vmsd
-rw-------    1 root     root               3039 Apr 29 08:35 TESTBOX5.vmx
-rw-------    1 root     root                263 Apr 21 15:28 TESTBOX5.vmxf
-rw-------    1 root     root              63270 Mar 11 16:54 vmware-11.log
-rw-------    1 root     root             165654 Mar 11 16:54 vmware-12.log
-rw-------    1 root     root             105954 Mar 11 16:54 vmware-6.log
-rw-------    1 root     root             240822 Mar 11 16:54 vmware-7.log
-rw-------    1 root     root              62127 Mar 11 16:54 vmware-8.log
-rw-------    1 root     root             105384 Mar 11 16:54 vmware-9.log
-rw-r--r--    1 root     root             309577 Apr 29 08:35 vmware.log

GhettoVCB.sh checks for a snapshot by using "grep -delta" but as you can see from the listing the snapshot does not have a -delta file. (If it helps, I think this VM was probably cloned from another, maybe using VMware Converter).

More generally, I am curious as to the technical reasons as to why snapshots should not be used long-term. For testing we use boxes which are snapshotted after an OS install, so we can install software over the top of it, test, and then revert. Also for testing we will have multiple product configs saved as snapshots derived from the same initial install. This enables quickly changing between different configs for faster testing. Is there some technical reason why this is a bad thing to do with VMware?

tohe · ‎04-29-2010

Hi William, you are right, the problem was the "()". I removed them and did a dryrun and it seems to work.

Thank you for that usefull hint!

lamw · ‎04-29-2010

So my personal stance on snapshots, is that it does have it's place but many times it's not properly understood and is abused as an easy way to "revert" back in time. The problem is many users don't understand the implication when a snapshot is created, especially on a live running VM and what happens to that snapshot as time goes on.

I would invite you take a look at the following links:

The process & space required when committing/deleting multiple snapshots

Performance Impact when using VMware snapshots

The way VMware snapshots are implemented, they're really meant for short duration of time. Unless you're very clear on the changes and even then, you really need some solid monitoring on your VM's disk + datastore. If you have a VM that's been allocated to be 20GB and you take a snapshot while it runs, a new "delta" disk is created which starts out at 16MB, that's how much it grows. The original disk is now left in a "read-only" state and no longer grows. All changes going forward now will write to the delta disk, now lets say you do a bunch of changes and install new applications and start writing and you grow the disk ... this new "delta" can actually get up to the size of the original disk if you introduce huge amounts of change and let's say it gets to 15GB, we now take another snapshot. We now have a new delta disk that continues to grow/etc. You can see how easily w/o proper monitoring that you can take multiple snapshots and potentially fill up your datastore.

This example is just with one VM but you can see how bad this can be with bunch more VMs. The other piece of it, as deltas get larger, when you commit the changes, it also takes quite a bit of time an generates quite a bit of I/O on the backend which can impact performance. If you take a look at Duncan's article on how snapshot commit/delete are done, they require so much more additional space than the snapshot size as it requires to commit every snapshot before it actually deletes it.

So it's not that snapshots are bad thing, but its not well understood and generally users run into all sorts of problems when they start to use snapshots without understanding how they really work. I've seen several posting on the VMTN forums where users end up with 500GB or 1TB snapshots that grew for a year! and commits that took few days to a few weeks.

This is why I do not promote the use of snapshots for long durations of time, they have their place like you've mentioned for critical patches/updates. Heck even VMware's VUM utility allows you to snapshot a VM prior to patching and then removing the snapshot after x-days, but it's a temporarily thing and not mean to be long lived.

Now regarding your VM, I'm not sure what state it was in ... but usually when you properly clone a VM, at least from vCenter, it'll consolidate the snapshot to your destination (e.g. the newly cloned VM will won't have snapshots and will just carry the latest changes)

To verify if you truly have snapshots, you may want to go to vSphere Client and check the snapshot manager to see if it see's any snapshots. It's not uncommon that the vSphere Client may not see the snapshots but you still have them and also it's not uncommon to end up with orphaned snapshots. I'm not an expert in figuring out the whole snapshot tree and what links to what and what can be deleted ... but in generally ,if you're not sure. You should be able to create a new snapshot using the vSphere Client and then do a "delete all" and that should commit all the changes and hopefully clean up any leftovers. If there are still snapshot files, then they may actually be orphaned.

Hopefully this helps you understand how VMware snapshots work and their impact

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

http://pastebin.com/xgiG3dNa

If you find this information useful, please award points for "correct" or "helpful".

MatthiasLuraTec · ‎04-30-2010

Hi lamw

i have posted my logoutput to pastebin.com

here is the link

Regards

Matthias

tohe · ‎04-30-2010

Hi William,

first I want say thank you for this great script. I´am using this now since two months and it works great!

With the new version I have only one problem, it does not recognize the VM_BACKUP_ROTATION_COUNT option. This option is set on 2 backups (VM_BACKUP_ROTATION_COUNT=2).

I saw today that I have 4 backups in the folders of my vmware appliances. The first one was created with an "older" version of ghettoVCB, so the naming is different (Servername-2010-04-26--1) and the other three are in the new naming scheme with the scheduled time of the backup (Servername-2010-04-27_20-00-01).

So the script does not delete the old backups. What can cause this problem?

lamw · ‎04-30-2010

Yes, this was a known bug that caused a space to be created within the output name. This has resolved in one of the updates, please manually delete the backup with the spaces or manually rename and download the new script and you should not run into this problem.

The latest version is always updated within the documentation and if you look at the title now, it includes the last revision date. All changes are documented in the change log, I would recommend that you take a look prior to posting to see if certain bugs/features have been added.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

lamw · ‎04-30-2010

Could you also include a dryrun output as well? It's interesting that the errors is that it can't find the specified disks. You may also need to provide me your directory structure to this VM + .vmx file to compare, since that is where the script is pulling the paths to the VMDK(s).

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

http://pastebin.com/hHczvj1M

If you find this information useful, please award points for "correct" or "helpful".

tohe · ‎04-30-2010

I´am using already the latest version of the script. I have downloaded this on monday the 26th and installed it after.

MatthiasLuraTec · ‎04-30-2010

Hi lamw,

thanks for the fast reply.

Here is the link to pastbin.com

But now I have used the new version from your script.

Maybe I haven't got the problem (?!) but I can't test a real run yet!

On tuesday next week I can paste the real run log.

Regards

lamw · ‎04-30-2010

Actually there were two updates that day. The problem that you're referring to was fixed in the latest version, you can easily verify by greping out "date" and you'll see that there's a difference in the way the date is generated. You mentioned earlier you were using an old version of the script which created the "space" and that has been resolved, please download the latest version and let me know if you still run into the problems. I would also either delete OR update the current backups so they match the existing format, else the rotation will have issues.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

chrb · ‎05-03-2010

William,

Thank you for the detailed reply regarding snapshots.

I have another problem. VMs with whitespace in the VM name are not being removed during rotation. A fix diff follows:

--- ghettoVCB.sh.orig   2010-05-03 09:52:52.845261356 +0100
+++ ghettoVCB.sh        2010-05-03 10:07:25.342935259 +0100
@@ -390,6 +390,8 @@
        LIST_BACKUPS=$(ls -t "${BACKUP_DIR_PATH}")
        BACKUPS_TO_KEEP=$(ls -t "${BACKUP_DIR_PATH}" | head -"${VM_BACKUP_ROTATION_COUNT}")

+       OLD_IFS=$IFS
+       IFS="$(echo -e "\n\r")"
         for i in ${LIST_BACKUPS};
         do
                 FOUND=0
@@ -405,6 +407,7 @@
                        rm -rf "$BACKUP_DIR_PATH/$i"
                 fi
         done
+       IFS=$OLD_IFS
 }

 ghettoVCB() {

lamw · ‎05-03-2010

Thanks for reporting this, if you've read the documentation, you know my stance on using "spaces" in a name When you ran this test, was it on ESX or ESXi?

I'll get this updated tonight and post the new version.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

MatthiasLuraTec · ‎05-04-2010

Hi lamw,

now, with the actualized script the logoutput is only:

" 2010-05-04 09:01:03 -- debug: Failed to acquire lock, another instance of script may be running, giving up on /tmp/ghettoVCB.lock"

Are the vmdks locked?

what can I do?

Regards

lamw · ‎05-04-2010

As the message states:

another instance of script may be running

This is a new feature introduced which prevents multiple instances of the script launching. In the past this has bitten certain users who miss-configured their cron entry, this has nothing to do with VMDK locking

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

maxsupremicus · ‎05-04-2010

I wrote a summarize script in AWK that summarizes the output log. Not sure anyone will find it helpful, but here's the script:

-

# cat summarize-esxi-log.sh

#!/bin/awk -f

/Initiate backup/ {

error="";

printf "%-20s ", $NF;

};

/Failed/ {

error=substr($0,match($0,"Failed"))

};

/Backup Duration/ {

printf "duration: %s %s", $(NF-1), $NF

};

/(Successfully|Error)/ {

if ($5 == "Successfully" && length(error) == 0)

printf "\tSUCCESS";

else

printf "\tFAILED";

if (length(error) > 0)

printf " : %s\n", error;

else

printf "\n"

};

/Snapshot found for .*, backup will not take place/ {

printf "%-20s no backup => has snapshot\tFAILED\n", substr($8, 1, length($8)-1)

};

-

When you run one of the ghetto backup log files through this awk script, you get this formatted output which I have a job email to me daily:

# ./summarize-esxi-log.sh logs/zeta-weekly-201005012332.log

delaware duration: 17.80 Minutes SUCCESS

connecticut duration: 33.57 Minutes SUCCESS

scrod duration: 10.70 Minutes SUCCESS

earth duration: 52.73 Minutes SUCCESS

uranus duration: 67.03 Minutes FAILED : Failed to clone disk : No space left on device (1835017).

georgia duration: 17.18 Minutes SUCCESS

pike duration: 34.17 Minutes SUCCESS

rattler duration: 15.50 Minutes FAILED

python duration: 15.55 Minutes FAILED

neptune duration: 20.47 Minutes SUCCESS

newjersey duration: 21.07 Minutes SUCCESS

I find it easier to quickly glance at this report and see if anything needs addressed.

Enjoy!

tohe · ‎05-05-2010

Hi William,

the problem still exists, also with the updated script the old backups will not be deleted.

What else can cause the problem?

lamw · ‎05-05-2010

Do you have spaces in your VM names? This was recently reported, but I'm still waiting for confirmation that it suggest fix is resolved prior to posting.

Could you provide a directory structure listing of the backup directory and post it here?

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

tohe · ‎05-05-2010

Hi William,

yes we use spaces in the names. Sorry I did not read it before (maybe I did not understand well because english is not my native language).

I can´t give you a listing of the backup directory, because the names too much detailed, such like domain names and so on ... sorry.

lamw · ‎05-05-2010

That's okay, then you're running into the problem that one of the users reported earlier on. I'll post a fix shortly and you can verify if it resolves the problem.

Thanks

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

chrb · ‎05-05-2010

Yes, spaces in names sometimes suck, but VMware allows it and I can't control what other people do.

The problem I ran into with VMs with spaces in the names not being removed was with esxi 4.0.

lamw · ‎05-05-2010

np. I've just updated the script and did a quick verification. Thanks for reporting the bug

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

VMware Code Central - Scripts/Sample code for Developers and Administrators