| Host Version | Script to use |
| ESX 3.5/4.0+ and ESXi 3.5+ | ghettoVCB.sh |
| ESXi 4.0 | ghettoVCB4i.sh |
# DO NOT MODIFY PAST THIS LINE #
chmod +x ghettoVCB.sh
[root@himalaya scripts]# ./ghettoVCB.sh
Usage: ghettoVCB.sh [VM_FILE_INPUT]
[root@himalaya scripts]# ./ghettoVCB-enhance.sh example_virtual_machine_backup_list
Powering off initiated for Quentin, backup will not begin until VM is off...
VM is still on - Iteration: 1 - waiting 3secs
VM is still on - Iteration: 2 - waiting 3secs
VM is off
################## Starting backup for Quentin ... #####################
Destination disk format: VMFS thick
Cloning disk '/vmfs/volumes/himalaya-local-SAS.VMStorage/Quentin/Quentin_1.vmdk'...
Clone: 100% done.
Destination disk format: VMFS thick
Cloning disk '/vmfs/volumes/himalaya-local-SAS.VMStorage/Quentin/Quentin.vmdk'...
Clone: 100% done.
Powering back on Quentin
#################### Completed backup for Quentin! ####################
Start time: Tue Jan 6 16:09:35 PST 2009
End time: Tue Jan 6 16:12:12 PST 2009
Duration : 2.62 Minutes
Completed backing up specified Virtual Machines!
~ # ./ghettoVCB-enhance.sh backup
Powering off initiated for UCSB-ENGINEERING, backup will not begin until VM is off...
VM is still on - Iteration: 1 - waiting 3secs
VM is still on - Iteration: 2 - waiting 3secs
VM is still on - Iteration: 3 - waiting 3secs
VM is still on - Iteration: 4 - waiting 3secs
Hard power off occured for UCSB-ENGINEERING, waited for 12 seconds
VM is off
################## Starting backup for UCSB-ENGINEERING ... #####################
Destination disk format: VMFS thick
Cloning disk '/vmfs/volumes/dlgCore-FC-LUN200.Templates/UCSB-ENGINEERING/UCSB-ENGINEERING.vmdk'...
Clone: 100% done.
Powering back on UCSB-ENGINEERING
#################### Completed backup for UCSB-ENGINEERING! ####################
Start time: Sat Jan 10 00:57:37 UTC 2009
End time: Sat Jan 10 01:01:55 UTC 2009
Duration : 4.30 Minutes
Completed backing up specified Virtual Machines!
vm1
vm2
vm3
Forgot to mention, you can also find more scripts/resources located at:
http://engineering.ucsb.edu/~duonglt/vmware/
Hi Shaveht,
This script can not be locally executed on VMware VIMA, but can be remotely kicked off from VIMA or any other UNIX/Linux, even a Windows host using plink.
The process is as follows:
You store the "ghettoVCB.sh" on shared storage (FC/iSCSI SAN / NFS), including the list of Virtual Machines you would like to backup. You can then create a cron entry that will kick off the backup process say every Friday at 12am. To ensure this does not require a user to enter a password, you'll need to utilize public/private SSH Key Authentication. This would then ssh into the ESX/ESXi host(s) and execute this script passing in the list which is also stored on the shared storage and the backup process would be kicked off.
This allows you to centralize your backup process without having to create separate cron entries on each of the ESX and ESXi hosts and storing a copy on each system. This does add an additional system to the process and if you know this VIMA host will always be accessible to the ESX/ESXi cluster, you should be fine.
An entry to the cron for the command would look like the following:
ssh root@A.B.C.D "/vmfs/volumes/SOME_DATASTORE/ghettoVCB.sh /vmfs/volumes/SOME_ADMIN_DATASTORE/virtual_machine_backup_list"
Hi lamw,
Thanks for the explanation.
in effect i was confused about how to use it.
thanks again.
Hi,
I have 7 blade servers with approx 50 VM's, is it possible to run this script from just one esxi host and have it back up all the VM's that are running on it locally and also the VM's running on the other blades?
Take a look at the my previous comment. I would not suggest setting this up on an ESXi host, probably an administrative linux/windows or even VIMA host.
Nice script and Thanks for the same. The backup runs fine after few issues being faced in the start. I have the following queries.
1. Is there a way I can backup the vm's without shutting them down?
2. What is the ideal restore procedure? (assuming Its going to be a disk-to-disk restoration.)
Thanks
tux
Hi tuxfan2183,
1) The power down feature was recently added, if you want to backup your VM(s) while they're live, just ensure the flag "POWER_VM_DOWN_BEFORE_BACKUP" is set to 0=no, 1=yes. By default this option is set to 0 which is to backup while the VM is running
2) There are multitude of options, depending where and how the backups are being stored and what your backup scenario requires. You can either mount the backup datastore and register the VM either by browsing the VI Client datastore or manually using CLI tool like "vmware-cmd -s register" or "vim-cmd" for ESXi. If you do not want to directly mount your backup but rather restore back to your source ESX/ESXi host, you can just manually run "vmkfstools" copy from your backup datastore to your source datastore and then register your VM. Again, this will be environment specific, but those are your options and this can be easily scripted or done manually as this is a backup process which requires additional care.
Hope this answered your questions.
Thanks william. You are the best
does this only back up 1 vmdk file per virtual machine? I ran it and it backed up teh vmdk of a vm i have but not another vmdk that is in the same directory
No, it should backup all "valid" VMDK(s) that is associated with a VM. Yon can verify this by going into the VM's "edit virtual machine settings" and ensure that all disks are listed within the hardware tab.
When you use the VIC to delete a disk and selecting the option "Remove from virtual machine", this will only remove the disk from being accessible to the VM, but the actual disk will still be located on the datastore and within the .vmx file, the disk will have a present value of "false". If you want to truly delete the VM's disk, you need to select the other option, so be sure before doing so.
Let me know if you have any other questions
There are certain things which I as a user thought could add more value to this script. When taking backup to compress and store the backup in a compressed format in the remote location. makes sense?
That is a good suggestion, but unfortunately there are many options in terms of compression format/type/etc and I would leave that to the end user to implement if required.
Why dont you try to setup a solaris 10 or opensolaris box and use a compressed ZFS file system via NFS to ESXi host?
Its extremely easy - and no expensive hardware required.
i found the problem. here is the 2nd disk in the .vmx file. I can almost guaruntee you this wasn't modified by hand because I don't think the person who created it has no idea how to.
scsi0:1.filename = "datastore_1.vmdk"
the line in your script that finds the disks is
VMDKS=`grep scsi "${VMX_PATH}" | grep fileName | awk -F "\"" '{print $2}'`
it's a case issue. maybe that should be changed to a "grep -i fileName"
Yea it should be -i, but from the various .vmx file I've viewed in the past for both 3.0 and 3.5 it's always been "fileName". What version of ESX are you running btw? This is the first I've heard of this issue
esx server 3i 3.5.0 build 123629
i just reproduced this by adding a second hard disk with VIC to the virtual machine and it puts it in the vmx with filename not fileName
Hello,
I'm trying your script.
Thanks for your work.
Just a question relative to performance.
In fact i was using vcbMounter on a local datastore and on an nfs server.
Backup time for the same machine:
with vcbmount on local datastore: 7 min
with vcbmount on NFS: 13 min
With your script on local datastore: 5 min
With your script on NFS: 180 min
Have you got an explanation for this or any clue?
Regards
Tom
my first run of this script was really slow over nfs too, around 25GB per hour. i blamed that on my cheap nas device.
i don't know what vcbmounter is, but does it copy with a straight copy or does it do a clone disk like this script does?
I'm not familiar with the vcbMounter tool, YMMV with different environments and I've seen various postings about the different performance with NFS and it all depends on your configuration and the type of NFS server you're running. It should not differ with over 100+min so I'm not exactly sure what type of VM you're backing up/etc. Does this occur on all VM(s) or just this one?
Also and FYI, the VMware VCB tools that come installed by default on the ESX host are only available with the version of ESX w/Service Console, this is not part of the ESXi releaese.
Interesting, perhaps this is only with ESXi. I'll double check with my ESXi 3.5u3 and see if this is also the case, if so I'm surprised this is the first time it's been caught. I know there's quite a few people using this script on unlicensed version of ESXi. Thanks for the info
when I run the command sh ./ghetto.sh vm_list from path / vmfs/volumes/datastore1/ of esxi 3.5 returns the following error:
. / ghetto.sh:. / ghetto.sh: 76: Syntax error: word unexpected (expecting "do").
What could be the problem?
Thank you.
I'm not exactly sure what the issue could be. I suggest re-downloading the script and ensure the script has been set to execute. If you download to a Windows system, you might have some ^M characters if you tried to edit prior to executing on ESX/ESXi host. Make sure you edit it on the host themselves to ensure you don't run into this issue.
You can always run:
sh -x ghetto.sh vm_list
Thanks for your answer, the script has been set to execute and the file has been downloaded and modified with ubuntu desktop.
This is the result of the command:
/vmfs/volumes/4975d95d-45fc3458-7ff4-001b212e4341 # sh -x ghetto.sh vm_list
+
: not foundghetto.sh: 6:
+ VM_BACKUP_VOLUME=/vmfs/volumes/datastore2/backupvm/
+
: not foundghetto.sh: 9:
+ ENABLE_2GB_SPARSE=0
+
: not foundghetto.sh: 12:
+ VM_BACKUP_ROTATION_COUNT=3
+
: not foundghetto.sh: 15:
+ date +%F
+ VM_BACKUP_DIR_NAMING_CONVENTION=2009-01-22
: not foundghetto.sh: 17:
+
: not foundghetto.sh: 18:
+ POWER_VM_DOWN_BEFORE_BACKUP=0
+
: not foundghetto.sh: 23:
+ ENABLE_HARD_POWER_OFF=0
+
: not foundghetto.sh: 26:
+ ITER_TO_WAIT_SHUTDOWN=4
+
: not foundghetto.sh: 31:
+
: not foundghetto.sh: 36:
+ ENABLE_NON_PERSISTENT_NFS=0
+
: not foundghetto.sh: 38:
+ UNMOUNT_NFS=0
+
: not foundghetto.sh: 41:
+ NFS_SERVER=172.30.0.195
+
: not foundghetto.sh: 44:
+ NFS_MOUNT=/nfsshare
+
: not foundghetto.sh: 47:
+ NFS_LOCAL_NAME=nfs_storage_backup
+
: not foundghetto.sh: 50:
+ NFS_VM_BACKUP_DIR=mybackups
: not foundghetto.sh: 52:
+
: not foundghetto.sh: 53:
+
: not foundghetto.sh: 55:
+
: not foundghetto.sh: 57:
+ DEVEL_MODE=0
+
: not foundghetto.sh: 59:
+ basename ghetto.sh
+ SCRIPT_PATH=ghetto.sh
VM_FILE_INPUT\n ghetto.sh
VM_FILE_INPUT
+ exit
: not foundghetto.sh: 63: exit
+ }
: not foundghetto.sh: 64: }
+
: not foundghetto.sh: 65:
+ local BACKUP_DIR_PATH=vm_list
+ local BACKUP_VM_NAMING_CONVENTION=
+ ls -tr vm_list
: No such file or directory
+ LIST_BACKUPS=
+
: not foundghetto.sh: 70:
ghetto.sh: ghetto.sh: 76: Syntax error: word unexpected (expecting "do")
/vmfs/volumes/4975d95d-45fc3458-7ff4-001b212e4341 # sh -x ./ghetto.sh vm_list
+
: not found: ./ghetto.sh: 6:
+ VM_BACKUP_VOLUME=/vmfs/volumes/datastore2/backupvm/
+
: not found: ./ghetto.sh: 9:
+ ENABLE_2GB_SPARSE=0
+
: not found: ./ghetto.sh: 12:
+ VM_BACKUP_ROTATION_COUNT=3
+
: not found: ./ghetto.sh: 15:
+ date +%F
+ VM_BACKUP_DIR_NAMING_CONVENTION=2009-01-22
: not found: ./ghetto.sh: 17:
+
: not found: ./ghetto.sh: 18:
+ POWER_VM_DOWN_BEFORE_BACKUP=0
+
: not found: ./ghetto.sh: 23:
+ ENABLE_HARD_POWER_OFF=0
+
: not found: ./ghetto.sh: 26:
+ ITER_TO_WAIT_SHUTDOWN=4
+
: not found: ./ghetto.sh: 31:
+
: not found: ./ghetto.sh: 36:
+ ENABLE_NON_PERSISTENT_NFS=0
+
: not found: ./ghetto.sh: 38:
+ UNMOUNT_NFS=0
+
: not found: ./ghetto.sh: 41:
+ NFS_SERVER=172.30.0.195
+
: not found: ./ghetto.sh: 44:
+ NFS_MOUNT=/nfsshare
+
: not found: ./ghetto.sh: 47:
+ NFS_LOCAL_NAME=nfs_storage_backup
+
: not found: ./ghetto.sh: 50:
+ NFS_VM_BACKUP_DIR=mybackups
: not found: ./ghetto.sh: 52:
+
: not found: ./ghetto.sh: 53:
+
: not found: ./ghetto.sh: 55:
+
: not found: ./ghetto.sh: 57:
+ DEVEL_MODE=0
+
: not found: ./ghetto.sh: 59:
+ basename ./ghetto.sh
+ SCRIPT_PATH=ghetto.sh
VM_FILE_INPUT\n ghetto.sh
VM_FILE_INPUT
+ exit
: not found: ./ghetto.sh: 63: exit
+ }
: not found: ./ghetto.sh: 64: }
+
: not found: ./ghetto.sh: 65:
+ local BACKUP_DIR_PATH=vm_list
+ local BACKUP_VM_NAMING_CONVENTION=
+ ls -tr vm_list
: No such file or directory
+ LIST_BACKUPS=
+
: not found: ./ghetto.sh: 70:
./ghetto.sh: ./ghetto.sh: 76: Syntax error: word unexpected (expecting "do")
/vmfs/volumes/4975d95d-45fc3458-7ff4-001b212e4341 #
It's kind of hard to tell what's going on with the output. What system are you running this on? Which version of the script did you download? Can you also list the contents of your vm_list
Can you also try the following:
Change
also, you didn't remove the first line with #!/bin/bash did you? or you're not using some wierd shell are you?
I downloaded version ghettoVCB-enhance.sh. The contents of the list is as follows:
windows xp
ubuntu server 32bit
this change VM_BACKUP_VOLUME=/vmfs/volumes/datastore2/backupvm had no effect.
Aalso the change of value DEVEL_MOdE variable had no effect.
Are you running this on ESX or ESXi? I also noticed you're executing this script on one of your datastores, it should not matter but can you try to move the script to another location, perhaps /root. I just downloaded both scripts and they executed fine using the default and just modifying the backup path, the issue seems to be in your environment but I'm not sure what might be causing this issue.
Could you also run the following on the shell:
echo $SHELL
vmware -v
I have not touched the first line of the script and the shell I use is integrated into the system vmware ESXi 3.5.I try to move the script in the root.
The command echo $SHELL returns:/bin/ash
The command vmware -v returns:VMware ESX Server 3i 3.5.0 build-123629
Moving the file in the root me back the message: Operation not permitted
Okay so looks like you're running the the latest ESXi 3.5u3, which is the same release I just finished testing as well and I don't see any issues. Since you're running on ESXi, there will not be a directory /root, just /.
It's going to get pretty difficult to troubleshoot exactly what the issue is, for some reason the system isn't liking the script. If you understand what the script is doing or have some scripting experience, you can setup some basic echo statements and see exactly where the script stops execution. From the looks of it, it seems to get through all the user defined variables but halts right before the actual code starts. If you can narrow down on the junk of code that it fails on, perhaps I can assist you further.
Also, your two VM(s) have the following displayName?
# DO NOT MODIFY PAST THIS LINE #
exit
# DO NOT MODIFY PAST THIS LINE #
Hello,
I don't know really the difference between vcbmount and snapshots.
I know that vcbmount work on vmdk files to compact them.
The NFS server is a FreeNAS on a 2x3Ghz xeon with one disk of 300Gb 15K U320.
I'm using ESX 3.5.
I don't think it's related to nfs configuration or server performance because for the same vm and for the same nfs server the vcbmount take 13 min and your script take 180 min
And yes it's the same problem for all machines
I'm still investigating on this
Regards
Tom
Can you tell me what variables need to be changed (other than NFS_SERVER and VM_BACKUP_DIR) in order for this script to work?
Also, how can I create a script that will initiate a snapshot of each VM, store them in the NFS (for daily backups) and another script that will do a full backup of each VM?
We are using ESXi 3.5u3
Thanks
Please take a look at the documentation for your questions, it should answer majority of them.
I am not an expert in scripting and general Linux environment, I am a beginner.
Yes, my two VM(s) have that name.
Also by entering the command exit the error is always in the same line in the script
The "exit' you inserted, allowed me to see where the issue is occuring. It's telling to the script to basically exit once it hits that line. This means the error is caused during the variable declaration section before it even starts to execute.
This brings me to believe the file may have some additional characters that may not be part of the original script/etc, perhaps characters from a Windows system. Did you download this from your Ubuntu box and scp the script to your ESXi host?
Can you run the following:
wc -l ghettoVCB-enhance.sh
wc -c ghettoVCB-enhance.sh
I'm sorry, can you provide a link for the documentation?
Thanks
If you're posting in this section of the forum, you're already looking at the documentation. Please go over the instructions carefully, I do ask that new users read through the documentation before posting a question as you can answer most of your questions this way. You'll see that both your use cases can be solved by using this script and may require slight modifications, again this script is intended to help backup VM(s) that may reside on ESX/ESXi 3.5+. This script will not cover all processes/requirements for a backup or recovery scenario, I leave it to the discretion of the end user to understand what this script is doing and making modifications that fit your operating environment and needs.
You're welcome to post follow-up questions if you're still unsure of how the script operates or question about the functionality of the script. Hope this helps
Problem solved: the file ghettoVCB-enhance.sh downloaded from the link higher contains the end of each line the notorious ^M, while the other file ghettoVCB.sh is clean
Thanks to all.
William,
I was wondering if you could recommend teh best way for recovering a vm from a backup like this. Like say your esxi server blew up one day and you put a brand new one in it's place and hooked it up to the nas you had bacccked up your vms to.
most of hte instructions out there involve manually creating a vm in vic like the one you used to have, deleting it's hard drive, cloning the backup disk into your vm dir with vmkfstools -i and then adding it to your virtual machine.
since all the vm information is stored in a .vmx it seems like it should be easier to restore a vm, is it just that there aren't vm creation tools at the command line that can be used?
There are many scenarios in terms of recovery and this really depends on your requirements/environment. If you look at the simplest case where you have a new ESX/ESXi host and you present the NFS datastore which contains the backup VM(s) to the host. All you really need to do is just browse the datastore using the VIC and click into each of your VM(s) and right click and add to inventory. You should see your VM registered to the host and you can power them VM on. This assumes that want your recovered VM(s) running on your backup datastore, it's probably best to definitely copy the VM to your new host, register and power on. Going off of that, yes you'll need to copy the .vmx and copy the VMDK(s) associated with that VM and you should be using vmfkstools -i as you've suggested. Again once you understand this process manually, it's relatively easy to automate and script this process. Again, this process will vary from user to user and based on your requirements, you'll come up with a solution hat fits your environment.
Good morning,
I created the script backupvm in /usr/local/bin running:
#!/bin/sh
ssh root@192.168.100.165 "sh /vmfs/volumes/datastore1/ghettoVCB.sh /vmfs/volumes/datastore1/vm_list"
#invio promemoria
echo backup vm posmailbe del `date ‘+%d-%B-%Y’`|mail -s “backup vm posmailbe” \ aaaaaa@iol.it
I also assigned the permission of execution.
I changed the file cron as follows:
americo@ubuntuserver32:/usr/local/bin$ sudo crontab -l
When setting up a cron entry, make sure the paths to all the commands use are the fullpath.
Please take a look at this thread to ensure they are: http://communities.vmware.com/message/1151164#1151164
I believe the vmkfstools is not being passed in my script as the full path, so you'll probably have to fix at least that command.
Hello, I have a question: Is it possible to backup additional added disks too? We have the OS on disk1 and the files on disk2. Does this backupscript support this. I could not find anything and tying it only made a backup of disk1.
Another thing I would like to ask is if you know any troubles using this script wiht ESX Version 3.02.
Thank you very much - also for the hard work writing this script!
Regards
Yes, please download the latest script ghettoVCB-enhance.sh which will backup all valid VMDK(s) presented to the VM. This was added awhile back but I guess I must have forgotten to update the doc. I'll make sure I do that tonight and also remove the old instance of the script.
I just read in the comments about the grep -i fileName change...It now works ...
That's really great work you've done! Respect!
Thank you
Sorry one more stupid question - does it also work on ESX3.02? I see the note "You*'re not running ESX 3.5+ ....
Or are there known issues using the script with 3.02.
Thank you!
Hi ollivetti,
Sorry, this script only supports ESX 3.5+, the requirement is stated in the title and also noted in the body of the document. This was intentional since majority of the users were also using this on ESXi and the only common toolset between ESXi and ESX is vimsh. You probably can slightly modify it to support older version of ESX, as vimsh does exists going back to 3.0, but I have not tested that far back. You can uncomment the section of the code that blocks it from executing if you're not running either ESXi or ESX 3.5+.
lamw,
I was finally able to get the NFS setup working with WSFU (in case anyone's pulling their hairs out, make that when you configure User Name Mapping, you map the LOCAL ADMIN account (not a domain admin) to "root")
I tested with a live running VIMA and it did a full backup of the VM.
I want to modify the script to do regular snapshots, not clone snapshots. Where/What do I have to change on the ghettoVCB-enhanced.sh script?
Thanks
I'm not sure what you mean by regular snapshots, I'm aware of only one type and the snapshot can be taken with/without the state of the memory. Please look into script on where the snapshot is being taken, I believe it's documented in the script if you go through it.
What would be the procedure to restore the ghettoVCB backups?
the procedure i use is
If you just need to restore a vm to a state it was during the backup, and you still have your existing vm:
1. delete the hard drive of the vm from within vic, choosing to remove tne files from the disk as well.
2. clone the disk from backup to the directory of your vm with
vmkfstools -i /path/to/backup/disk.vmdk /path/to/vm/
3. add the hard drive to your vm choosing to use an existing virtual disk, select the /path/to/vm/disk.vmdk that you just made
If you need to restore the vm to another host or simply deleted it all together
1. make a directory /vmfs/volumes/whatever/vm-name
2. copy the .vmx file from backup to this dir
3. clone the disk from backup to the directory of the vm
vmkfstools -i /path/to/backup/disk.vmdk /vmfs/volumes/whatever/vm-name/
4. Browse the datastore in VIC, find the .vmx file, right click and select add to inventory.
not sure if theres a better way of doing it but it works for me.
How were you able to create a directory? When I try "mkdir whatever" I get: Operation not permitted.
I am logged in as "root". Am I missing something?
Thanks
Hi, first of all thank you again for your response! I see you have quite a lot of questions to answer. One thing I would like to ask because I have problems is: I am using 3.02 and don't have vmware-vim-cmd so I use vmware-cmd or shall I use vimsh -n -e to execute commands. Unfortunately the execution stops here: echo "Error: failed to extract VM_ID for ${VM_NAME}!" Does it mean that the script is looking for der VirtualMachine ID in /tmp/vms_list.
Regards
Please take a look at the vimsh documentation to understand how it works.
http://engineering.ucsb.edu/~duonglt/vmware/#vmware_vimsh
Hello
Thank you for this script! Really good work. The only thing that does not seem to work for me is backing up more than one vmdk in a vm. I tried several times with no luck. Do I have to do something special to get it to work?
Thank you for an answer.
Are you running ESXi? If so, you might need to change the line "grep fileName" to' grep -i fileName".
Multi-VMDK(s) are supported, so long they're valid and presented to the guestOS.
Yes, I am running ESXi. If I run the command:
VMDKS_FOUND=`grep scsi "${VMX_PATH}" | grep -i fileName | awk -F " " '{print $1}'`
on a cmd-sh, it gives me both scsi-vm-hdd... so thats all ok. But when I run your script, only the first (50G) HDD is backed-up, the second HD (300GB) not.
Do you have any ideas what could cause this problem?
Thank you very much!
you have to modify the script to have grep -i as well, it only uses grep without the -i which only returns matches to the exact case fileName. esxi servers sometimes have it as filename (lower case n)
Thank you for the response.
Of course I also modified the script... But still got only one vm-hdd backuped. I tried to troubleshoot in the script and what I found is the following. After the line:
grep "${SCSI_ID}.present" "${VMX_PATH}" | grep "true" > /dev/null 2>&1
it gives me the Value "1" for the second vm-hdd, which probably means it is not present??? But it is my data-disk which IS present in the vm... what do I have to modify?
Thank you.
if you get a value of 1 for that particular line, it means that within the .vmx file for the ${SCSI_ID}.present = "true". This means that the VMDK was either removed from the VM but not deleted from the filesystem or it was removed and deleted from the filesystem. In either case, the reference is left in the .vmx file and the flag is just changed from true to false.
To verify this, you can look at your .vmx file, and if the code is returning a 1, it means that disk is not present to the VM which means it should not be showing up in the VI Client when you go into it's properties unless the .vmx file has been manually modified without the ESX/ESXi knowing about it.
That seems to be strange... if you look at the content of my vmx-file:
/vmfs/volumes/497b9254-df6992a6-9fc0-0090270f57c5/Win2K3 TS # cat Win2K3\ TS.vmx
config.version = "8"
virtualHW.version = "4"
floppy0.present = "true"
nvram = "Win2K3 TS.nvram"
deploymentPlatform = "windows"
virtualHW.productCompatibility = "hosted"
tools.upgrade.policy = "useGlobal"
powerType.powerOff = "default"
powerType.powerOn = "default"
powerType.suspend = "default"
powerType.reset = "default"
displayName = "Win2K3 TS"
extendedConfigFile = "Win2K3 TS.vmxf"
numvcpus = "2"
scsi0.present = "true"
scsi0.sharedBus = "none"
scsi0.virtualDev = "lsilogic"
memsize = "1024"
scsi0:0.present = "true"
scsi0:0.fileName = "Win2K3 TS.vmdk"
scsi0:0.deviceType = "scsi-hardDisk"
ide0:0.present = "true"
ide0:0.clientDevice = "FALSE"
ide0:0.deviceType = "cdrom-image"
ide0:0.startConnected = "TRUE"
floppy0.startConnected = "false"
floppy0.clientDevice = "true"
ethernet0.present = "true"
ethernet0.networkName = "VM Network"
ethernet0.addressType = "generated"
guestOSAltName = "Microsoft Windows Server 2003, Standard Edition (32-bit)"
guestOS = "winnetstandard"
uuid.location = "56 4d 61 2a b1 4c 7f ca-b4 81 00 31 27 13 e1 0d"
uuid.bios = "56 4d 61 2a b1 4c 7f ca-b4 81 00 31 27 13 e1 0d"
toolScripts.afterPowerOn = "true"
toolScripts.afterResume = "true"
toolScripts.beforeSuspend = "true"
toolScripts.beforePowerOff = "true"
ide0:0.fileName = "/vmfs/volumes/497b9254-df6992a6-9fc0-0090270f57c5/Windows Server 2003/Windows Server 2003.Enterprise.German.ISO"
floppy0.fileName = "/dev/fd0"
scsi0:0.redo = ""
ethernet0.generatedAddress = "00:0c:29:13:e1:0d"
tools.syncTime = "FALSE"
sched.swap.derivedName = "/vmfs/volumes/497b9254-df6992a6-9fc0-0090270f57c5/Win2K3 TS/Win2K3 TS-af7ea147.vswp"
ethernet0.generatedAddressOffset = "0"
hostCPUID.0 = "0000000a756e65476c65746e49656e69"
guestCPUID.0 = "0000000a756e65476c65746e49656e69"
userCPUID.0 = "0000000a756e65476c65746e49656e69"
hostCPUID.1 = "00010676000208000008e39dbfebfbff"
guestCPUID.1 = "0001067800010800000802010febbbff"
userCPUID.1 = "0001067600020800000802010febbbff"
hostCPUID.80000001 = "00000000000000000000000120100000"
guestCPUID.80000001 = "00000000000000000000000000100000"
userCPUID.80000001 = "00000000000000000000000120100000"
evcCompatibilityMode = "FALSE"
scsi0:1.present = "TRUE"
scsi0:1.deviceType = "scsi-hardDisk"
scsi0:1.filename = "Win2K3 TS_1.vmdk"
scsi0:1.mode = "persistent"
scsi0:1.redo = ""
****************************************
Here you can see that scsi0:0 as well as scsi0:1 is present and true.... And I am working with this device every day, so it IS present...
Do you have any idea? Thank you very much for your help, I am really out of any guess....
Look at what the script is doing .... it's grepping for "true" ... looks like your .vmx has "TRUE", see the problem.
I'm curious on why some of the user's .vmx files are slightly different from the one's I've worked with. I'm not sure if this is due to any specific version or user's are manually modifying their files. I've always seen the right hand side use lower case values, but I have seen it few times in older releases where it's all caps. I'll try to find some time this weekend to fix those two issues, it should be using grep -i.
That should fix your issue
Thank you, it's now working like a charme!
But does somebody know how to start this script automatically with some kind of "cronjob" in ESXi? I tried the idea with "autobackup.sh" and included the "cron" directory in this script, but it does not seem to work... (Always get the following error:
diff: Can't stat '/tmp/auto-backup.237536.dir/var/spool/cron/crontabs/root': No such file or directory
which means this temporary directory/file has not been created by the script...
Thank you!
I did it by adding the following lines to /etc/rc.local
/bin/echo "0 19 5 * * /vmfs/volumes/vmback/ghettoVCB.sh" >> /var/spool/cron/crontabs/root
/bin/busybox crond -b
I believe these changes do not persist do a reboot? I remember talking to someone about this and on reboot these changes are reverted to their original configurations, including the cron entry. Is that correct?
the auto-backup.sh script which runs like every hour or so i think backs up a number of files in /etc to be persistant upon reboot. this is why things like modifying inetd.conf to enable ssh still work after reboot. /etc/rc.local is another one that changes will be saved. that is why i put the code in there to add the crontab entry on boot.
Ah good to know, I'm aware of the backup but I thought someone had mentioned earlier in another forum thread that files are not persisted through a reboot. That is good to hear, I'll try to find some time to update the doc stating that fact. Thanks
Hello
Yes, I can confirm the Backups are persistent when you write the stuff in rc.local. All runs without problems.
Thank you for all the good tipps!
First, thank for the great script lamw!
Everything works well when I do ghetto from SAN to local hard drive, but when I try gheto to NFS I get
"DiskLib_Check() failed for source disk The system cannot find the file specified (25)" after cloning and snapshot remove lasts about 2 mins with lot of "...missing *-delta.vmdk" in ESX's /var/log/messages. There is no difference with Windows or Linux NFS server - backed vmdk looks fine but I don't know is this only warning or something to worry about. It's same story with all my ESXs.
Usually when you get those warnings, it means it was not able to find the source VMDK for whatever reason. I know some user's have had issues in the past where their VMDK(s) were stored in other directories or if they had multiple VMDK(s), earlier versions of the script did not handle that use case.
The latest script should be validating that certain VMDK(s) exists prior to trying to take a backup, though if you've manually edited the .VMX file, there could be some inconsistencies in the .vmx itself. You can always try the script with a -x flag which gives detail verbosity of what's happening while the script execute and see if you can anything that's off. If it works for SAN to Local, then it should also work for NFS unless you some miss-configurations on your NFS Server (you can always manually take a snapshot and manually run vmkfstools copy manually to make sure it's working with your NFS Server)
(e.g.)
sh -x ghettoVCB.sh my_back_vm_list.txt
Thanks, I'll try.
Another thing; currently I do three kind of backups from san: daily to local hard drive (raid1), dayly to nfs (raid1) and weekly to tape. Since there is not enough space on local hard drives to do rotation, ghetto returns error every second day (and delete files then, so backup is ok tommorow). Rotation count is 0.
What can I do to make this backup daily?
Well nothing is stopping the script do run daily backups, that is up to the end user's and the setup of their cron schedule. If you're looking for more elaborate backup processes such as transfer from SAN to local, then to NFS, etc, etc. You'll need to script that process, this script is intended to help provide a backup solution for your VM(s), it's left open for any modifications/specific requirements one may have for their environment.
William,
Thanks for sharing your great scripts and knowledge.
Everyone,
Thanks for the great forum. I've learned a lot here.
We are using ESXi with an NFS datastore on Openfiler 2.3.
I've modified William's script to make ESXi "thin" disk format backups, as it was not too easy to add backups or clones of VMs back to the VM inventory when they were in 2gbsparse disk format, and I did not want to lose the disk space "thick" backups would have required.
The backups and restore/clone seem to work fine with thin format, but I feel it seems too easy. Am I missing something?
Is there any reason why I should not use William's ghettoVCB.sh script to do thin backups rather than thick or 2gbsparse?
(I realize the 2gbsparse format is more portable to other filesystems, other versions of VMware, etc.)
Thanks,
-David
First off, this is really some great work. Thank you for posting this.
But, unfortunately I am having some issues. I downloaded the file yesterday 2-12-2009, onto my ubuntu machine. I opened it with gedit and it had the ^M throughout the file. I dont know if Ubuntu did that or the file up there has them but either way I fixed it with dostounix. So I set it up and ran it on a VM with 1 vmdk file and it worked like a charm. HDD to HDD on a 10 GB VM takes consistently less than 5 mins, which is great.
I noticed another user had a similar issue too, I may have to look at the case when the VMDK(s) are stored in another datastore other than the root VM directory. I'm curious when it tries to backup the other drives, is the location correct? In terms of how it's able to backup VM(s) while it's still running is with a snapshot. When you take a snapshot, the main VMDK is changed to readonly and a new delta file is created which will contain all changes after the snapshot has been taken. This then releases a file lock on the main disk which allows you to backup the disk. Once the snapshot has been removed (committed), all changes will merge back with the primary disk. Then on the next backup, it'll do the same and those changes will be captured.
I'll take a look at the issue sometime next week ... need to spend sometime with the gf this weekend =)
The location is correct that it comes up with. It matches the same string thats in the vmx file so it almost knows its there it just cannot copy it.
Also is there a way to backup one vmdk to one location and a second vmdk to another, even though they are in the same VM?
I thought it was using the snapshot feature, but I wasnt sure. I appreciate the update, and have a good weekend! 
hm ... that's interesting. If the snapshot was taken successfully, then both those disks should be unlocked. You can always manually do what the script is doing to just verify that it should work. In terms of your second question, you would need to modify the script to duplicate it to two destinations. You can always just re-run the script with a second destination as a work around.
Has anybody come up with a reliable method of monitoring this script for success/failure? Here's an idea off the top of my head for Windows users who use Kaseya/LPI for monitoring. I wouldn't have the expertise to write something like this but maybe somebody else would see value in this.
How do I run it with 2 destinations, wont it copy the first vmdk each time. So I would only end up with one of the 2 vmdks in 2 places?
Im not sure where in the script I should try manually.
Thanks for the help.
You'll have to add that logic into the script if you're looking to split your destination copy.
Many thanks for your great script!!
To everyone:
Everything works great except the output to a log file. I have a scheduled task on a windows 2003 server machine to run batch file which executes ghettoVCB.sh via plink.exe. Thanks to Raj Perumal for his blog guide.
The command looks like this:
plink.exe root@192.168.100.200 -pw mypassword "nohup ./vmfs/volumes/vmfs-storage/ghettoVCB.sh /vmfs/volumes/vmfs-storage/vmlist > /vmfs/volumes/vmfs-storage/backuplog.txt &"
After a successful backup the backuplog.txt contains not a single line and the file size is 0 btyes.
When I execute the command in a putty session the output to the backuplog.txt works great.
Any ideas whats wrong?
I have a problem.....
It all works find but VM_BACKUP_ROTATION_COUNT=1 does not do any thing.
I have now
.15_SpiceWorks-2009-02-17
.15_SpiceWorks-2009-02-16
as backups and if I run it now I get
.15_SpiceWorks-2009-02-18
any ideas?
Looks like something was modified in the script because the rotation count is kept track by a special extension "--X" ... I don't see that in your output, did you change something in the script?
Config part
VM_BACKUP_VOLUME=/vmfs/volumes/489326a8-7adbcdf8-a9f4-00188bf6e6f5
POWER_VM_DOWN_BEFORE_BACKUP=0
Code part
LIST_BACKUPS=$(ls -Atr ${BACKUP_DIR_PATH})
This is due to that my VM names are .15_Name_XXXX
BTW, wnat can I comment out in order to run all but the actual cloning of the folder?
OK, I recopied the original code.
this is my output.
~ # /ghettoVCB.sh /backup_vms
Ah, then I think you can just remove the -A param, as that ignores "." Any reason why you're naming convention is in that form? I would probably change that since that usually interpreted as hidden directories (just a thought).
I'm not sure what you mean by actual cloning of the folder? If you're asking about the rotation count, you'll need to set that = 1, it does not make sense to set it to 0.
You should give the script a read through, the logic should not be too difficult to follow and you're welcome to ask further questions. If you can manually run through this process, then the script will make more sense.
Again, this is probably due to your naming convention, so it's unable to rename the directory since it's looking for a "--X' extension. Please take a look at the previous reply for a fix or modify the way you VM directory.
Just changed the name of the VM. No .15_ as prefix.
Running the code.....
Lets see.
What I ment about cloning the folder... was that I would like to coment the part that actualy does the backup until I get the rotation count working
BTW, VM_BACKUP_DIR_NAMING_CONVENTION="$(date +%F)" is what is set for the naming.
is Date: 11/01/2008 the last version of the file? I see updates later than that
That is actually the date I started the project. The latest script will always be available in the download section
To your other comemnt, that is the naming convention that is appended to the backup directory, this can be anything but I thought having the date as part of the backup directory might be useful. This can be changed to any string you would like.
mmmm
now its creating
SpiceWorks-2009-02-18--1
I will rename to 02-17--1 and run again.
YES!!!!!!!!!!!!!!!!!!!!!!
Thanks!!!!!!!!
Now I need to just find a way to get the files of the ESXi server.
Glad to hear. Try setting up an NFS Server, it's pretty simple and this will allow you to get the VM(s) off the host to other backup or tape storage.
When I am running the script I am getting the following error:
DiskLib_Check() failed for source disk The system cannot find the file specified (25).
I turned on tracing on the script and discovered it is happening as a result of this command:
vmkfstools -i /vmfs/volumes/datastore2/OpenFiler//vmfs/volumes/4947b0ac-5b9a4e28-9b21-0011092a9380/OpenFiler/OpenFiler_2.vmdk /vmfs/volumes/Disk1/vmbackups/OpenFiler/OpenFiler-2009-02-19//vmfs/volumes/4947b0ac-5b9a4e28-9b21-0011092a9380/OpenFiler/OpenFiler_2.vmdk
Any ideas what is causing this?
Realy nice Work, but I found an Izzue 
When you have multiple vmdks on different datastores in one vmx thd filename contains the fullpath to the vmdk
So the Script could not save this vmdk because it is not in the VMX_DIR Path.
I hope you understand this.
Same issue Im having. Im trying to modify the script but no success so far.
Nope, sorry you lost me.
I do have a couple of different VMs on different datastores, but I got this error when I tried to back up just one of them.
Are you saying that all my VMs need to be on the same datastore for the script to work?
Or am I just not understanding this...
My situation is my VM's have 2 vmdk files. They are both on different datastores. When I run the script only the vmdk that is on the same datastore as the vmx gets backed up. The vmdk on the other datastore does not and it gives me the same error.
Oh wait, I think I (might) get it!
So, yes the VM I am backing up also has a couple of disks which do reside on different data store.
Fortunately I do not actually need to back those up (they are storage for openfiler which I am spanning across a software RAID).
I really just need to backup the main disk associated with the VM.
So I am guessing I can just ignore this error.
If you do happen to extend the script to support this, you might consider allowing the user to specify which disks to back up and which to ignore...
Jepp, this is because the filename of the vmdk on the diffrent Datastore contains the fullPath.
scsi0:0.fileName = "Fileserver.vmdk"
scsi0:1.filename = "/vmfs/volumes/4982de6f-6ea7343b-c3c9-00221986180a/Fileserver/Fileserver_1.vmdk"
So you could not easily use the VMDK Name as source or target for copy.
You have to check first the path an the make 2 arrys VMDKS and VMDKFULLPATH
All, this has been acknowledged by a few as an issue, the current workaround is to only backup VM(s) that reside on the same datastore.
I've been occupied with other projects recently and when I get some time I'll take a look and provide a fix. If anyone has implemented a a working solution, feel free to post or send me a PM.
Anyone having issues downloading the ghettoVCB.sh from this page? I get it and it doesn't run. It shows a "Bad Interpreter" File Not Found error. It seems to be corrupt or have some control characters in it (^M). I can't see them in vim or vi and I only downloaded to a windows xp system, didnt edit it at all on windows.
It seems the ghettoVCB.sh on http://engineering.ucsb.edu/~duonglt/vmware/ and here on VMware site are different revisions, but the one from the ucsb site is working OK.
TIA.
Sorry about that, I must have forgotten to update the engineering account. They should be the same, I've uploaded the current version and can you try the download on the VMTN doc again?
If you do have a problem with the trailing "^M" line termination you can do the following to get a good version of the script:
Alright it seems the file from the vmware site still has the ^M somehow, tried the sed command and it was OK.
The file from the ucsb site is updated and Good still.
Thanks.
How to change the ghettoVCB.sh to delete the old backup before execute new backup?
Thanks.
I'm looking at the issue and basically it boils down to the SOURCE and DESTINATION path, the VMDKS array will contain either the local VMDK disk names with respect to the VM's folder which you'll need to append the VMX_PATH variable or if the disk is stored in another datastore, it'll contain the full path to the VMDK. A quick fix would to check the variable that goes through each for loop iteration to see if it starts with /vmfs/volumes, if so then you know it's a disk that's located in another datastore and setup the variable accordlying. Running through some print statements, I noticed an issue with the DESTINATION which also needs similar logic.
Though right now the issue from my basic test is the following: I have one Windows VM with 1 disk call it Quentin.vmdk which is the unique descriptor describing one of Quentin's disk that is stored in Datastore A. I know add a second disk and store i in Datastore B, I commit the changes and then look at the disk name from both the VIC and on the filesystem and what do I notice?
The name of the .vmdk is the same as disk 1, this will be an issue when backing up the disks to your destination as these filenames are no unique and will most likely over write one another. I then added a third disk two the VM and stored it in Datastore B and the .vmdk now has a unique name appended with _1.
The observation, at least on ESX w/SC (haven't tested on ESXi) if you have a VM with 1 disk and you add another and you're going to store it on another datastore the name will not be unique as it'll default to the VM's display name. This will cause a problem when backing up ... I'm trying to think of what changes would need to occur but it seems that it'll most likely requiring the modification of the .vmdk name's to the destination which may or may not be something end users want.
I'm still pondering over the issue right now over lunch ... if anyone has any suggestions I'm open, but this may be a feature that may not be supported due to the way the disk's are not uniquely labeled across datastores.
I think what would make sense would be to create folders for the datastores names and put the files there.
This would also make it a lot easier to restore since one can easily tell what when where...
Make sense?
Or more technically, retain the path relative to /vmfs/volumes
Unfortunately that does nothing for the current situation other than additional directories and a disconnect between what's listed in the .vmx file and the additional work a user would have to remove the disk reference and re-map on a backup/recovery. The solution would be too specific as some user's may want to mount the backup datastore directory whereas others would want to manually copy from the recovery datastore but not directly register the VM(s) from backup.
The more I think about this, the only decent solution is to automatically rename and update the backup VMDK(s) as they're being backed up and set them to be referenced locally within the VM's directory. This then allows a user to either mount the VM directly if he/she would need to recovery or restore from backup and have all VMDK(s) referenced properly. The only difference with VM(s) that were initially stored across multiple datastores will not be preserved (the ability to recovery may be more important than how the disks were initially stored). For further use cases, that'll be something end-users will need to customize for their enviornment.
Even if you retain the path ... what's to say a user can't bring up the original datastores, they're recovering to another ESX/ESXi host with another datastore or local. This really goes hand in hand with recovery which is going to be unique for each environment. This script will not be able to handle all those use cases, hence it's up to the end users to customize as needed for their enviornment. We're trying to generalize the process so that such customization is available if necessary.
How about the following as a compromise:
To preserve the functionality and familiarity of the script with user's that have already implemented a recovery/restore solution, I will continue to backup all VMDK(s) that are stored locally with the VM to the destination folder as you're aware of. The added fix would then support VMDK(s) that are not stored in the same datastore as the VM but in backing those VMDK(s) up to take into consideration of non-unique VMDK names provided by VMware, I would create an additional directory which will be the unique UUID of the original VMDK destination and then it would be up to the end user's to provide a solution on recovery/restoration.
This will preserve the original .vmx file as I have currently in the script and if a user chooses to restore a VM to it's initial configuration, the only added step would be to vmkfstools copy any VMDK(s) that were not stored with the VM to it's correct datastore. A user would also be able to directly mount the backup datastore but then would be required to manually update the location of the VMDK(s) that were not stored with the original VM ( this would have to be done even if the current script supported VMDK(s) across multiple datastores as your recovery .vmx file will not contain the correct paths )
Script has been updated, please let me for those that have multiple VMDK(s) across multiple datastores if this works. Thanks
It seemed to work good.
Now I have an backup of the vmdk which where on the second datastore
First of all.. thank you William and all others who have contributed to this. The script is sweet. Much faster than other scripts that use other backup methods. I am however getting speeds that are unusually slow for file transfer on an internal LAN.
I am running ESXi embedded on my host and I have a Allegro NFS Demo running on a physical server 2008 box (attempted to configure Unix on server 2008 using native Windows tools with no luck; authentication issues). The export is located on my local server 2008 storage. It's a 1.5TB Barracuda. Both boxes have Intel Gigabit Nics and are connected VIA a Linksys Gigabit switch. I tested my transfer speeds using wireshark at 15.8MBit/sec while running this script. I tested windows transfer from my guest VM to my Windows 2008 server at ~320MB/s. I know it's not my network configuration because of the results I get with Windows. Is allegro just slow as mollases? Server 2008 and Allegro Hate eachother? I know Server 2008 hates Unix (unless it's solaris 10), but thats another story.. What numbers should we expect/shoot for? What are other seeing backing up to NFS storage?
Feeling down? Need a good laugh? 100GB of data split between two VM's..
Start time: Wed Feb 25 00:16:28 UTC 2009
End time: Wed Feb 25 11:56:51 UTC 2009
Duration : 700.38 Minutes
Completed backing up specified Virtual Machines!
Quick question.
How do I restore form the Backup?
I currently store the Backup on the same Datastore.
Do I just add to Inventory and run from there? or do I move the files to the original Folder?
What happens with multiple vmdks?
mmm
I just did a backup of a server that has its OS on one DS and the data on a second DS.
The only vmdk that was backedup was the data disk. the the actual OS vmdk was not.
Please make sure you're using the latest version.
Restore can be either direct mount from your backup datastore and just register or replicate from your backup datastore using vmkfstools. I would recommend that latter, that way you don't use your backup for use but replicate it to the host for recovery/restore. The recovery/restoration process is something that is more manual and can be scripted but the requirement will be based around your requirements and should be implemented based on that.
no comment?
not really, I'm not familiar with NFS off of Windows, there are many configurations that could cause the speeds you're seeing. If you look in the forums, you'll find various comments on good/bad speeds. If you're running an NFS Server on VM, you may see speed decreases than on a physical. Your export options can also affect the speeds (e.g. sync vs async). There have been many individuals getting very good speeds and many that get less than up to par/expected. Your best bet is to ask the community for their configurations but can also be due to the hardware you're using... unfortunately when you're discussing performance, ymmv depending on your setup.
The script has nothing to do with your performance if that's what you're after.
hello all
I am backing up about 4 different vm-images each night with the script. But somethimes the snapshots are not removed after backing up and so the next time the backup of this vm will fail.
VM(s) w/Snapshots are not supported because snapshots in general should not be kept for long periods of time. They're meant to be used for quick testing and all changes should be merged back into the main VMDK as soon as possible. If you're looking to send email, a quick search online you'll find many solutions available including using default utilities like (mail or mutt) if you're on a UNIX/Linux system.
Thank you for your response. But somethimes the snapshots created by ghettovcb are not removed correctly and so the next backups are not performed anymore.
Is there a way to remove every snapshot automatically (in the script) before the new backups are performed? That would ensure that the backups run even if there are "old" snapshots.
Interesting, I haven't heard of any issues of snapshots not being removed. Are there any previous snapshots on the VM when running the backup? If so, that may cause some issues.
Currently with the method of using the vimsh there is not an option to remove all snapshots, you also don't want to automate that process of committing older snapshots without checking the state of those snapshots. Committing snapshots should be a manual process to ensure you don't revert or make any changes to the system that the Application/System owner is not aware of. Again, the script enforces the rule that if it finds any prior snapshots it will not continue. You should not be using snapshots for long durations and should commit the changes as soon as possible especially if you're looking to backup the VM(s).
I have a directory with my virtual disc and a snapshot created from ghettoVCB (not removed after the backup). When I try to:
${VMWARE_CMD} vmsvc/snapshot.remove ${VM_ID}
then I get a segmentation fault. So that seems to be the problem the snapshots are not correctly removed after the script terminates.
Any idea what could be the reason?
Hello lamw,
since your script is running now for a few weeks, I got two days ago this error:
on esxi1:
Clone: 100% done.
Powering back on ps1
bash: /vmfs/volumes/iscsi1backupdsa1/backups//ps1/ps1-2009-02-26: bad number
/vmfs/volumes/iso/backup/ghettoVCB-enhance-iscsi1.sh: /vmfs/volumes/iso/backup/ghettoVCB-enhance-iscsi1.sh: 323: Syntax error: /vmfs/volumes/iscsi1backupdsa1/backups//ps1/ps1-2009-02-26+1
on esxi2:
Clone: 100% done.
Powering back on exchange3
bash: /vmfs/volumes/iscsi1backupdsa1/backups//exchange3/exchange3-2009-02-26: bad number
/vmfs/volumes/iso/backup/ghettoVCB-enhance-iscsi1.sh: /vmfs/volumes/iso/backup/ghettoVCB-enhance-iscsi1.sh: 323: Syntax error: /vmfs/volumes/iscsi1backupdsa1/backups//exchange3/exchange3-2009-02-26+1
Plink error: Remote command failed
(Exit code 2)
I can see that your script was not removing the backup files: Backup rotation is set to: 2 and there are 3 directories now in the folder.
It looks like that only the last vm of my machinelist had this problem. I did not change anything in your script an machinelist since weeks.
Hope you have a answer.
Sorry, but where i can't find ghettoVCB-enhance.sh script?
Please download the current version of the script and manually try again and see if you run into the same issues, if not, then it could be some issues when remoting in with plink.
The current download is the most up to date version of the script, please look over the documentation if you have further questions.
Hello lamw
I updated to the latest version, but I got the same problem.
See:
I think you might have had some bad backups that were not rotated out properly which is causing some issues. Can you please move any previous backups whether they're good/bad out of that directory or purge them completely. Try a fresh backup
I deleted the complete folder with the backups and tried 3 backups only with this problematic machnine and they were running. So I delted also the other folders with
problematic machine. I have to wait now for a few days to see if everything is working again.
Just like you I wanted to log to a file.
This is what I came up with
http://communities.vmware.com/docs/DOC-9653
Thanks for the post, the logging feature will be available in an upcoming release. Stay tuned
First of all, i wish to thank for great script.
I have one suggestion. It is possible to add functionality to backup power off machines which have snapshots?
In some kind applications (like development) night downtime does not matter. But snapshots does. Thank you.
Hi hremul,
As mentioned in the previous posts, snapshots should not be kept for long durations. They can grow and left uncheck could cause issues on your datastores being filled up. If you're looking to backup VM(s) that have snapshots, best practice is to commit those changes and merge the delta files back to the VM then backup the virtual machine. There some complexity with the way the script handles removing snapshots if you're going to try to backup a VM with snapshots and partially it's a limitation with the vimsh and keeping track of the snapshots. It's best that you don't backup VM(s) with snapshots, but if you do, you'll need to make modifications to the script and that is an unsupported feature.
Hi lamw,
Thanks for a great script! I have one question though. Is it possible to change the script in such way that it can copy the backups to a datastore on a remote esxi host using something like scp?
You could append some scp commands and using paired SSH keys to remotely copy the backups to external storage or another host. There have been some threads regarding the implementation of scp on the unsupported console and the performance/speed in copying large files. You might want to take a look at a free tool by Veeam fastscp, I have not used it personally but I've heard many good things about their product. That might be a better approach if you need to copy files in/out of ESXi.
You also have the option to leverage vifs.pl from the VI Perl Toolkit default utilities, but again these are customizations that'll need to be added to your custom version of the script.
Thanks for your reply. Fastscp is a nice tool, but it is only a file copy-tool. It does not handle running vm's because it does not take snapshots.
There is also veeam-backup 3.0 now with support for esxi. The draw back of this tool is that it does not backup to a target esxi host, only local storage or san or iscsi storage. Veeam-backup makes use of the console of esx to make incremental backups of vm's and esxi of course does not have an esx-like console, so that is why it cannot use esxi as a target for its backups.
And then there is also an other tool, vm-explorer, which does do backups to other esxi hosts, but this tool does not run as a service. It is intended to be run as a user only.... For me that is not an option for a backup tool. I could run the tool with a service wrapper, to run it as a service, but i cannot test this scenario with the trial license because there is this annoying popup window, you have to click away every time you start the tool.
So it looks like i have to customize your script. Wish me luck
Jeroen
Compressing the backup is more than a good suggestion... here is why...
The VM filesystem is stored as a sparse file. When you copy that data to the network storage, you are un-sparsing it, unless you encapsulate the data into a compressed file, using compression that knows how to properly restore the sparse data.
So my 20g vm, that is really only using 270m of disk space in it's sparsely allocated file, will grow instantly to 20g using your backup system. If you restore it, or use the backup to clone in, you are no longer sparse... Instead you suddenly have a big file full of zeros. And those zeros take up network bandwidth.
The backup is not accurate if it fails to preserve the state of the stored data, and this includes the sparseness.
Sparse files are strange beasts, and programs like cp do not handle them correctly most of the time, so the moment you "forget" to handle the file correctly, it inflates to a non-sparse state. In a sparce file, an unallocated block of 0's is not stored on disk, rather the index points to null, which represents an unallocated data block assumed to be zeros. If you cat the file, the filesystem produces those 0's, but they don't really exist on disk. If you copy the file without an awareness of the sparse allocation, you end up with a huge file full of unneeded zeros... and this takes up network bandwidth as well as disk io. When you unsparse a file, there is a heavy performance cost and storage cost.
While some might say "use a compressed filesystem on the NFS server"... That does not deal with the whole issue. On a compressed filesystem you are still storing unneeded 0's, and when you move the file to a non-compressed filesystem, all that space is consumed, along with the disk io needed to write it all out. So you go from a vm that takes twenty seconds to move from one server to another using vmware's tools, to one that takes 3 hours to move. And if you depend on your network bandwidth to do work, now you have a whole lot less of it... not to mention the time lost. I in a day can move hundreds of vm's that only take 20 seconds to transit across the network... but that number falls to 8 when they take 3 hours each.
If you have 3 vm's to back up, it's a minor issue, especially if you do the backup when you sleep. But if you store the backup with the intent to clone it during work hours, it's is a big hit on productivity.
The backups need to preserve the sparseness, a sparse aware compression tool needs to be used. There are not a lot of compression tools on your ESXi system, there is one. So the suggestion that there are many options is wrong. And the suggestion that this is not an issue is misguided. And using a compressed filesystem to store the data on is also not addressing the main issue.
If you don't know what a sparse file is, you can read about it at:
http://administratosphere.wordpress.com/2008/05/23/sparse-files-what-why-and-how/
This does need to be addressed for this tool to be useful the the larger audience.
It is a great tool by the way, don't let my sparse rant take away from the fact that this tool is wonderful. My point is that compression in the utility is not as optional as you would like to think. Compression and correct sparse handling are critical.
Hi there and thank you for a wonderful script and contribution to the community!
I have two ESXi host machines that are colocated at a data center. There is a 3rd Linux backup computer with NFS on it, and I want to be able to backup the VMs running on these hosts on a regular basis. I have a 'public' WAN network (the VLAN provided by the data center that is public to the world) and a private LAN that is not private. All machines have dual NICs - one for the public WAN and one for the private LAN.
I would like to have it so that my NFS server runs the backups of all VMs using the LAN for this. I'm also sensitive to security concerns, particularly brute force attacks on SSH which are showing up routinely. As a result, all public servers on the WAN have their SSH ports changed from default ports to non-default ports, and Fail2Ban scripts installed to detect brute force login attacks on these boxes. This has been very successful so far.
I cannot install a firewall in this environment because the cost of doing so is prohibitive since I have about 20 WAN IP addresses on the VLAN. I may do this in the future when I work out how to set this up with pfSense but for now, its prohibitive as an option. Consequently every server that is WAN facing has their own local IPTables firewalls in place.
I understand from reading the posts and documentation on the ghettoVCB scripts that it appears to run from within a ESXi host which is made available through a SSH port enable on the box. As much as I would like to be able to do this, its not possible due to security concerns. I can't have default port 22 open like this. So is there a way that I can install it on my NFS server for the LAN only, and not have any of the ESXi boxes set with SSH at all?
Or if not, is there anyway to apply some 'best practice' security approach to enabling SSH on the ESXi hosts that allow for the use of non-default ports, and some form of script that could be run to thwart brute force attacks on the boxes?
Thanks again for your wonderful work, and I look forward to any comments or suggestions.
Myles
Hi Myles,
Thanks for the comments.
I wanted to get a few things cleared up and this may help you decide on how you go forward with implementing a backup solution. ghettoVCB can run on both ESX/ESXi as you know and specifically on ESXi, you'll need to enable the unsupported SSH console. This is not required for the backup to work but you'll need a way to execute the script from a remote system and currently SSH is the only method to communicate with the host to run the script.
You should also note that the unsupported SSH Console is unsupported by VMware, if you have a support contract this could potentially void the agreement. Best practice is not to enable the console, but there are certain things that is limited with ESXi if you were to just rely on RCLI or VIMA to manage and sometimes an admin may need to login to the console. It's best that you secure your environment and ensure that your hosts are not directly accessible to the public WAN network and only hosts within your environment can communicate to your ESXi host(s).
Hopefully this clears up a few things and good luck on your implementation.
Hi,
I am using ESXi 3.5 Update4 with the ghettoVCB.sh script downloaded from this website.
I trying to backup 3 machines: Ubuntu 8.10, Windows 2003 STD, Windows 2008 STD, on all 3 machines are the latest VMWare tools installed to a NFS share which is mounted.
I have tried to backup all 3 machines with putty connected to the ESXi server.
su root ./ghettoVCB.sh virtual_machine_backup_list
The problem now is that I would like to backup the machines while they are running (Online)
POWER_VM_DOWN_BEFORE_BACKUP=0
ENABLE_HARD_POWER_OFF=0
2 of the 3 machines are shut down immediately (not gracefully, its like power off) when I start the backup script.
Can anyone tell me where the problem can be?
Thanks in advance for any suggestions!
Your stated configurations and your expectations slightly contradict each other? Are you trying to perform the backup while the VM(s) are running? If so, the VM(s) should not be shutting down, unless there is some other issue with the VM(s) or host. If you're trying to backup the VM(s) and have them power down prior to backup, then you'll need to enable the appropriate flags. If you enable both flags to be 1 and you do not have VMware Tools installed on the guest, then the host will hard power off the VM after a certain duration.
Please clearly describe your scenario and setup.
Thanks for your quick answer, the 2 machines where the problem with the hard shutdown occurs have the current VMWare Tools installed, the two machines are also Powered On and running before starting the backu script.
hi:
thanks for your great script!
Hi,
I'm confused about the versions of this script. The version I downloaded today from the link in the "attachments" section of this page (or from "http://engineering.ucsb.edu/~duonglt/vmware/") is 11/01/2008:
#!/bin/bash
hi:
the author didn't modify the change log. but they are the latest version and can detect existing snapshot. i have tried that.
the attachment and the script at author's website is different files. but after filtering all the white spaces, they are the same.
Hi sky_li,
If you're running hight transactional databases, it's generally not recommended to use snapshots for backups due to transactions occurring within a DB. There is something called VSS which is available within VMware Tools that makes the guest aware that a snapshot is going to occur and it forces all data to be flushed from memory to disk, I've not used this personally and I believe it may only be available when VMware VCB is called, Duncan has an article at http://www.yellow-bricks.com/2008/07/26/vss-snapshots/
I would probably stay away from taking VMware snapshots and rely on database snapshots.
In terms of vimsh wrapper for creating snapshots, you won't find much information on any of the commands as these are not officially supported by VMware even though it provides a slew of advanced functionality.
You can always run the following for some basic info:
[root@himalaya scripts]# vmware-vim-cmd vmsvc/snapshot.create
Insufficient arguments.
Usage: snapshot.create vmid [snapshotName] [snapshotDescription] [includeMemory]
Creates a snapshot for the vm.
Hi Guido,
As tbsky stated, the latest script is currently available for download on this page. The date within the script may be a little miss-leading, it's actually the date that I started develoment of the script. The chanelog on the other hand is accurate in terms of the changes that have taken place since the release of the script and again the latest download will always be available on the VMTN document download page.
I'll ensure that both the script on the VMTN and on our website is the same, sorry for any confusion
Hi Guido,
As tbsky stated, the latest script is currently available for download on this page. The date within the script may be a little miss-leading, it's actually the date that I started develoment of the script. The chanelog on the other hand is accurate in terms of the changes that have taken place since the release of the script and again the latest download will always be available on the VMTN document download page.
I'll ensure that both the script on the VMTN and on our website is the same, sorry for any confusion
hello lamw
I ve got a server to backup which has only space for one backup at a time. Now your
script does first the backup and then rotates/deletes the old backups. This means, in
some time during the backup there are to backups on the disc...
Would it be possible to just move the "rotate"-function to the beginning of your script
and do the backup after rotating/deleting the old backups? Or can you see any problems?
Thank you,
ibhp
hi lamw:
i am just curios. "quiesced windows snapshot" seems become complete at 3.5 update 2 with sync+vss driver. but the link you give me tell us stay away from exchange,ad,sql. then i don't know what this "quiesce" function for.
also there seems some version of snapshot command line have "quiesce" option, like the link mentioned:
http://www.esxguide.com/esx/index.php?option=com_mambowiki&page=createsnapshot&page_heading=&show_pageheading=&allowanonymoususers=
but it seems the new command line version didn't have the "quiesce" parameter, only "memory" option is left. i wonder if the snapshot is quiesced automatically now, or the option is because "quiesced" may bring problems.
someone said that if you do a "quiesced snapshot", ms-sql log will tell you.i haven't any real ms-sql server running in the vm now. maybe i can check that in the future.
The reason I put the rotate function after the backup is to ensure we have a good backup prior to rotation, especially in the case of just rotating once. I would say please ensure that you have sufficient space for all backups
Yea I'm not too familiar with the expected behavior of VSS other than asking the VM to flush its data from memory to disk. I just know from reading on the VMTN forums from other experts it's not recommended to using VMware snapshots on VM(s) like databases and if you do, you'll need to look into VSS which is only available per VMware VCB (I think). You may need to run a few tests to see the behavior and I would actually recommend posting a question regarding this on the main VMTN forum and I'm sure an expert familiar with the process can provide further insight.
Hi lamw,
First of all, thx for this great script! I have a question though and hope that the following might also be possible with your script: can you specify a vmdk within a vm that you want to backup? I would like to backup only one vmdk of a vm that is on the same datastore. So basically I want to exclude all the other vmdk's that are attached to this vm and stored on a different datastore. Is this possible?
Thank you,
Pietcur
Specific VMDK disk(s) backups are not supported, only full backups. You're more than welcome to modify the script to fit your needs.
hi:
i am considering put all the windows pagefile and linux swap file to a separate vmdk file. so i can skip backing up them in the future. would you consider implement this feature? or would you consider receive patch?
thanks for your help!!
Thanks for the script.
Shouldn't the script bypass the creation of a snapshot if the guest is in a powered down state? Seems like extra work to take a snapshot and then merge the snapshot back into the base when the machine is already static.
Also, I got the following error while backing up a guest that was powered off:
/vmfs/volumes/datastore1/scripts # ./ghettoVCB.sh single_vm
################ Taking backup snapshot for Windows XP 32bit Base ... ################
DiskLib_Check() failed for source disk Insufficient permission to access file (38).
Destination disk format: VMFS thick
Cloning disk '/vmfs/volumes/Development/Windows XP 32bit Base/winxp-32bit-base.vmdk'...
Clone: 16% done.
The clone appears to have worked, however out of all the guests that I backed up, that was the only one that echoed the DiskLib_Check() message.
I notice that the script also doesn't appear to take into consideration whether or not a vmdk is thinly provisioned. (as previously mentioned by ddigioia)
I could alter the command:
vmkfstools -i "${SOURCE_VMDK}" "${DESTIONATION_VMDK}"
to
vmkfstools -i "${SOURCE_VMDK}" -d thin "${DESTIONATION_VMDK}"
But:
a) is there a way for the script to identify if a vmdk is thin, and if so add '-d thin'
b) is there any danger in storing thin clones? maybe why the script doesn't already do the previous
You know I think this was something I had missed when dealing with the option of allowing a user to configure the shutdown of a guest, I think I might have forgotten to take into consideration the current state of the VM. You're right, this may be a bug. Majority of the user's I've spoke to that have used the script are running live backups and would not run into the issue, but it's a good point and I'll ensure the next update of the script is fixed.
Thanks for bringing it to my attention
At the time, we were not looking at using thin provision backups, it does not hurt and provides optimized storage consumption (hence probably why vSphere officially supports it now)
Your modifications will work and thinly provision disks upon backup, also a note when you're backing up to NFS datastore, it'll automatically thin provision the disk for you AFAIK
a) You can look at the descriptor file and you'll find a line that says: ddb.thinProvisioned = "1"
b) In terms of backup, there is are no dangers, better storage utilization. In terms of running thin provision VM(s), you just need to be aware of your storage utilization and with ESX 3.5, there are not sufficient monitoring on the datastores to notify you as VM(s) start to fill up, with vSphere there are better notifications/reports.
In an upcoming release of the script all 3 backup types will be supported zeroedthick, thin and 2gb sparse
Thanks for the comments
hi:
please consider add "monolithic sparse" also. with that you get a small,single vmdk file.
backup tools like rdiff-backup can compare two versions for a file and only backup the difference.
with "monolithic sparse" single vmdk file, backup tools can easily backup many versions of vms with little harddisk space.
Hello community
I have a win2k3 server running on a ESXi-host with 2 virtual harddrives attached. Now one time the backup runs correct (backing up both virtual harddrives) and the other time it backs-up only the first virtual harddrive (every second time only one virtual hdd is backed-up). This looks like this:
Failure:
Destination disk format: VMFS thick
Cloning disk '/vmfs/volumes/First_750GB_HD/Win2K3 TS/Win2K3 TS_1.vmdk'...
Destination disk format: VMFS thick
Cloning disk '/vmfs/volumes/First_750GB_HD/Win2K3 TS/Win2K3 TS.vmdk'...
Clone: 100% done.
*******************************
Success:
Destination disk format: VMFS thick
Cloning disk '/vmfs/volumes/First_750GB_HD/Win2K3 TS/Win2K3 TS_1.vmdk'...
Clone: 100% done.
Destination disk format: VMFS thick
Cloning disk '/vmfs/volumes/First_750GB_HD/Win2K3 TS/Win2K3 TS.vmdk'...
Clone: 100% done.
*******************************
Any idea what this could be caused from?
Thanks for any help!
Hi everyone,
We're proud to announce the release of ghettoVCBg2 the next gen of ghettoVCB. You can find more information located at: http://communities.vmware.com/docs/DOC-9843
Hope you guys like it!
I've just upgraded my ESXi from 3.5 to 4.0 and the old ghetto script doesn't work.
Infact if I connect with putty and launch the script:
./ghettoVCB.sh weekly
I receive
-ash: ./ghettoVCB.sh: not found
I can't use the new script ghetto2 because I have the free version of ESXi
Can someone help me?
ghettoVCB has not been tested/certified for ESX(i) 4.0, we've been pretty busy and just have not had the time to test. This holds true for variety of other scripts and some will fail due to changes in vSphere release. In general, I think the script should work and it does not utilize anything that is not already available in ESXi 4.0, make sure the permissions on your script is set to executable. I'll try to do a quick run tonight and post back.
The error is telling you that ghettoVCB.sh doesn't exist in your current directory.
Was your ghettoVCB.sh script on a datastore or on one of the OS file systems? If it wasn't on a datastore, the version 4 upgrade probably wiped it.
Copy it back to a datastore and execute it from there and see if it will work with v4.
Okay, just got ESXi 4.0 installed as a VM on our ESX 4 (physical) and I see the problem, looks like there was some change to their shell environment and if you didn't know the busybox console runs on ash ... kinda like bash. It looks like it has a problem when you try to do the following:
./ghettoVCB.sh
-ash: ./ghettoVCB.sh: not found
#!/bin/ash
ash ./ghettoVCB.sh
Ok now I can launch the script but I have these errors:
/vmfs/volumes/48c6a14e-37156b78-c647-001a925eca7c # ash ./ghettoVCB.sh weekly
Error: failed to extract VM_ID for 2008Termi!
Error: failed to extract VM_ID for al
Assiste!
Error: failed to extract VM_ID for zaVista2!
Start time: Wed May 27 06:51:45 UTC 2009
End time: Wed May 27 06:51:48 UTC 2009
Duration : 3 Seconds
Completed backing up specified Virtual Machines!
I ran a quick test earlier and realized some of the code may need to be updated. I have not had a chance to dig further on what needs to be modified.
Have you had a chance to try out ghettoVCB4i.sh to see if the issues are resolved?
Thanks for the great script. I finally have an option for backing up my ESXi (free) 4 VMs. I downloaded ghettoVCB4i.sh and attempted to use it but encountered a problem where all of my VMs with the letter 'n' in the name where causing an error. This is similar to the issue described by freez267 above. I don't fully understand why but commenting out the line
IFS=$'\n'
appeared to resolve the issue for me. I am not sure what this line does.
Thanks
Tony Russell
That's odd... it should work, I've verified on our ESXi 4 system.
I am running this on ESXi 4. I have tried to follow the directions very closely but I keep getting a Permission Denied error every time I try to actually run the script. Any suggestions?
Did you change the permissions on the script to have executable permissions?
chmod +x ghettoVCB4i.sh
I continue to have this error
Error: failed to extract VM_ID
On the VMs that have 'n' in the name. For example is impossible to backup the VM '2008Terminal' and 'AssistenzaXp' but no problem with 'W7RC'
I just created some dummy VM(s) using the names you've specified and it looks like the environment is not treating $'\n' as a newline char but the literal char 'n'. This cause the loop to break off once it sees the first char 'n' and a VM with the name: "2008Terminal" will be chompped into "2008Termin" and "al" which neither of which will match, hence it fails. If you specify a list of VM(s) w/o 'n' then the backup is successful. This is pretty odd and I'll have to look into it some more.
You said if you remove the IFS=$'\n', everything works?
That is exactly what happened in my case. If I commented out that line, it worked fine.
Looks like the newline needs to be defined in the following manner:
IFS='
'
Hi,
Thanks for a fantastic script. Works well, but I have a problem with the rotation. Anyone with a clue why it won't work? See output from a sh -x command (last bit):
Cloning disk '/vmfs/volumes/HB-ESXi2-scsi1/HB-Label/HB-Label.vmdk'...
Clone: 100% done.
+ IFS=
+ ! 0 -eq 1
+ /bin/vim-cmd vmsvc/snapshot.remove 256
+ echo Removing snapshot from HB-Label ...
+ ls /vmfs/volumes/HB-ESXi2-scsi1/HB-Label
+ grep -q delta
+ sleep 3
+ ls /vmfs/volumes/HB-ESXi2-scsi1/HB-Label
+ grep -q delta
+ sleep 3
+ ls /vmfs/volumes/HB-ESXi2-scsi1/HB-Label
+ grep -q delta
+ checkVMBackupRotation /vmfs/volumes/Thecus/Backup/HB-Label /vmfs/volumes/Thecus/Backup/HB-Label/HB-Label-2009-06-07
+ local BACKUP_DIR_PATH=/vmfs/volumes/Thecus/Backup/HB-Label
+ local BACKUP_VM_NAMING_CONVENTION=/vmfs/volumes/Thecus/Backup/HB-Label/HB-Label-2009-06-07
+ ls -tr /vmfs/volumes/Thecus/Backup/HB-Label
+ LIST_BACKUPS=HB-Label-2009-05-31
HB-Label-2009-06-05
HB-Label-2009-06-07
+ -z 4
+ TMP_DIR=/vmfs/volumes/Thecus/Backup/HB-Label/HB-Label-2009-05-31
+ echo /vmfs/volumes/Thecus/Backup/HB-Label/HB-Label-2009-05-31
+ TMP=/vmfs/volumes/Thecus/Backup/HB-Label/HB-Label-2009-05-31
+ /vmfs/volumes/Thecus/Backup/HB-Label/HB-Label-2009-05-31 = /vmfs/volumes/Thecus/Backup/HB-Label/HB-Label-2009-06-07
+ /vmfs/volumes/Thecus/Backup/HB-Label/HB-Label-2009-05-31 -ge 4
sh: /vmfs/volumes/Thecus/Backup/HB-Label/HB-Label-2009-05-31: bad number
+ echo /vmfs/volumes/Thecus/Backup/HB-Label/HB-Label-2009-05-31
+ BASE=/vmfs/volumes/Thecus/Backup/HB-Label/HB-Label-2009-05-31
/vmfs/volumes/Thecus/Script/HB-Label/ghettoVCB.sh: /vmfs/volumes/Thecus/Script/HB-Label/ghettoVCB.sh: 340: Syntax error: /vmfs/volumes/Thecus/Backup/HB-Label/HB-Label-2009-05-31+1
/vmfs/volumes/46df116f-b561be9a/Log #
I had a similiar Problem here an a ESX Server 3.5:
At first: Script works well from the Root-Shell...
But, when i Plan the Script via Crontab, the Script starts, make's the Snapshot, Backup the *.vmdk and *.vmx File in a newly created Folder BUT at the End of the Backup the Script don't rename the Folder in correct manner.
Script stopped working. I have to rename the Folders manually in correct Name und Numbers and have to remove the Snapshot manually from ESX Server's VI Client.
The Script only Backups the First VM in the Backup List and after Backup stopped working !
mfg
Ritchy059
I've running an ESXi 4.0 Server.
The backup does function well. But there is one problem wich is descibed by other persons above. It is the empty log file. Is there a solution for it or not? At the moment I see through all folders if the backup success. But that can't be the final solution.
Further the current version of ghettoVCB4i.sh runs great if there are no whitespaces in the vm name. Otherwise the rename after backup fails and because of that the backup rotation fails too. So I've removed all whitespaces in VM names like lamw describes as best practice
Thanks lamw for his great work... really really usefull script!!
For those interested in logging, maybe this helps you:
Run the script like this
sh ghettoVCB.sh vmlist.txt 2>&1 | busybox logger -t ghettoVCB -p 1
and then, push ESXi logs to a remote syslog-ng server.
Does this backup method copy the vmdk to the same vmfs volume and to get it offsite you would have to download the backup vmdk file from the datastore?
You could just mount a NFS volume and select that as the destination for the backup. That's what we do. Works great.
Quick question... I am running ghettoVCB.sh on ESXi 3.5 and its working great on most of my VMs. However I have a VM that seems to have a snapshot associated with it. I checked in the datastore, and there was a 'delta' file there. I powered down the VM and then deleted the delta file. Powered it back on, and then attempted to run ghettoVCB.sh. It again wouldn't do the backup because it said that there was a snapshot there. I checked it again, and the delta file had returned...
I know this isn't a specific question of ghettoVCB, but how do I get rid of these snapshots so I can do a clean backup of a live VM?
You shouldn't delete deltas by hand. Instead, remove snapshots from VIC.
Deltas track changes to the main vmdk, but information about snapshots are kept in vmsd files.
Unfortunately I only have my one ESXi box at home with 1 HDD in it and my desktop so I have no NFS volume to mount 
So for me, does that mean that this script just backs up the running vmdk and I would need to manually download it?
If that is the case the primary function of the script would be to reduce downtime as otherwise I would need to power down the VM and copy the vmdk correct?
Great script! Thank you so much... We are backing up 4 ESXi 3.5 servers successfully with about 6VMs each.
There is only one problem to backup a Debian Linux 60GB virtual harddisk. Snapshot seems to be OK. Backing up the disk (one of two) failed after some time with the following error message.
Taking backup snapshot for DEASX401 ... ################
Destination disk format: VMFS thick
Cloning disk '/vmfs/volumes/datastore1/DEASX401/DEASX401_2.vmdk'...
Clone: 79% done.Failed to clone disk : Insufficient permission to access file (38).
Destination disk format: VMFS thick
Cloning disk '/vmfs/volumes/datastore1/DEASX401/DEASX401.vmdk'...
Clone: 100% done.
Removing snapshot from DEASX401 ...
Completed backup for DEASX401! ####################
Start time: Wed Jul 1 06:45:27 UTC 2009
End time: Wed Jul 1 09:21:51 UTC 2009
Duration : 156.40 Minutes
Completed backing up specified Virtual Machines!
Might it be possible that the disk is to big? Is there any timeout?
Here some more information:
./ghettoVCB.sh is running with root account!
backup disk mouted vie NSF from a Win2003R2 server
share is an external 1GB USB disk
nsf shared folder is windows compressed (can save until 75% of original space)
Regards,
Bjoern
How can i edit it to run on VIMA?