Ghetto Tech Preview - ghettoVCB-restore.sh - Restoring VM's backed up from ghettoVCB to ESX(i) 3.5 and 4.0+

VERSION 1 Published

Created on: Aug 23, 2009 11:28 PM by lamw - Last Modified: Aug 25, 2009 11:13 PM by lamw

This script performs a restore of virtual machines backed up using ghettoVCB. Tasks are performed directly within the service console of the ESX(i) server involved in the restore process. This is currently a Ghetto Tech Preview that includes basic restore functionality. Two main use cases are supported in this release:

1) Restore a VM that contains ALL VMDKs on one datastore
2) Restore a VM that has VMDKs located on multiple datastores

In all cases, restored VMs will have VMDKs that reside on the SAME datastore chosen for the restore process. Please ensure that there is sufficient space on the target datastore before attempting a restore operation. In the near future, restoration of VMs backed up using the compression feature of ghettoVCB will be implemented.

Features

Support for logging output
Support for various debugging output
Allow spaces in VM(s) backup list (not recommended and not a best practice)
Support for restore in the following formats: ZEROEDTHICK (default behavior), 2GB SPARSE, THIN or EAGERZEROEDTHICK

Requirements:

VMs backed up using ghettoVCB.sh on ESX(i) 3.5 or 4.0
SSH console access to ESX/ESXi 3.5 or 4.0 host

Setup:

1) Download ghettoVCB-restore.sh to either your ESX or ESXi
2) Change the permissions on the script to ensure it can be executed:

chmod +x ghettoVCB-restore.sh

Usage:

[root@himalaya ~]# ./ghettoVCB-restore.sh
###############################################################################
#
# ghettoVCB-restore for ESX/ESXi 3.5u2+ & 4.x+
# Author: William Lam
# http://www.engineering.ucsb.edu/~duonglt/vmware/
# Created: 08/18/2009
# Last modified: 08/25/2009
#
###############################################################################

Usage: ./ghettoVCB-restore.sh -c [VM_BACKUP_UP_LIST] -l [LOG_FILE]

OPTIONS:
   -c     VM backup list
   -l     File ot output logging

(e.g.)

Output will go to stdout
        ./ghettoVCB-restore.sh -c vms_to_restore

Output will log to /tmp/ghettoVCB-restore.log
        ./ghettoVCB-restore.sh -c vms_to_restore -l /tmp/ghettoVCB-restore.log

Standard input for script is a file that contains:
1) Full path to the backed up VM
2) Full restore path
3) Restoration disk format
Reminder: When creating this file on a non-Linux/UNIX system, one may introduce ^M characters that will cause the script to misbehave. To ensure that this does not occur, please create the file on the ESX/ESXi host.

Here is a sample of what the file should look like:

[root@himalaya ~]# cat vms_to_restore
#"<DIRECTORY or .TGZ>;<DATASTORE_TO_RESTORE_TO>;<DISK_FORMAT_TO_RESTORE>"
# DISK_FORMATS
# 1 = zeroedthick
# 2 = 2gbsparse
# 3 = thin
# 4 = eagerzeroedthick
# e.g.
# "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/VCAP/VCAP-2009-08-18--1;/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage;1"
"/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/VCAP/VCAP-2009-08-20--1;/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage;1"
"/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/sideswipe/sideswipe-2009-08-20--1;/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage;1"
"/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/stupid ass vm with spaces/stupid ass vm with spaces-2009-08-20--1;/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage;2"
"/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/vMA-resize/vMA-resize-2009-08-23--1;/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage;3"

Comments in the input file is acceptable so long as the intended line is preceded by a #. The above sample VM restore file, vms_to_restore, describes the following backup:

VM to restore	Datastore to restore to	VMDK format
/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/VCAP/VCAP-2009-08-20--1	/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage	zeroedthick
/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/sideswipe/sideswipe-2009-08-20--1	/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage	zeroedthick
/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/stupid ass vm with spaces/stupid ass vm with spaces-2009-08-20--1	/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage	2gbsparse
/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/vMA-resize/vMA-resize-2009-08-23--1	/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage	thin

Sample Execution:

Execute restore with output going to stdout (restore the first two VMs listed from above):

Input file:

[root@himalaya ~]# cat vms_to_restore
"/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/VCAP/VCAP-2009-08-20--1;/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage;1"
"/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/sideswipe/sideswipe-2009-08-20--1;/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage;1"

[root@himalaya ~]# ./ghettoVCB-restore.sh -c vms_to_restore
################## Restoring VM: VCAP  #####################
Start time: Sun Aug 23 22:31:26 PDT 2009
Restoring VM from: "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/VCAP/VCAP-2009-08-20--1"
Restoring VM to Datastore: "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage" using Disk Format: "zeroedthick"
Creating VM directory: "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/VCAP" ...
Copying "VCAP.vmx" file ...
Restoring VM's VMDK(s) ...
Updating VMDK entry in "VCAP.vmx" file ...
Destination disk format: VMFS zeroedthick
Cloning disk '/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/VCAP/VCAP-2009-08-20--1/VCAP.vmdk'...
Clone: 100% done.
Registering VCAP ...
End time: Sun Aug 23 22:32:32 PDT 2009
################## Completed restore for VCAP! #####################

################## Restoring VM: sideswipe  #####################
Start time: Sun Aug 23 22:32:32 PDT 2009
Restoring VM from: "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/sideswipe/sideswipe-2009-08-20--1"
Restoring VM to Datastore: "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage" using Disk Format: "zeroedthick"
Creating VM directory: "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/sideswipe" ...
Copying "sideswipe.vmx" file ...
Restoring VM's VMDK(s) ...
Updating VMDK entry in "sideswipe.vmx" file ...
Destination disk format: VMFS zeroedthick
Cloning disk '/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/sideswipe/sideswipe-2009-08-20--1/sideswipe.vmdk'...
Clone: 100% done.
Updating VMDK entry in "sideswipe.vmx" file ...
Destination disk format: VMFS zeroedthick
Cloning disk '/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/sideswipe/sideswipe-2009-08-20--1/4a31aece-e34d675a-72cf-003048d9586a/sideswipe.vmdk'...
Clone: 100% done.
Updating VMDK entry in "sideswipe.vmx" file ...
Destination disk format: VMFS zeroedthick
Cloning disk '/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/sideswipe/sideswipe-2009-08-20--1/4a31aece-e34d675a-72cf-003048d9586a/sideswipe_1.vmdk'...
Clone: 100% done.
Updating VMDK entry in "sideswipe.vmx" file ...
Destination disk format: VMFS zeroedthick
Cloning disk '/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/sideswipe/sideswipe-2009-08-20--1/sideswipe_1.vmdk'...
Clone: 100% done.
Registering sideswipe ...
End time: Sun Aug 23 22:34:46 PDT 2009
################## Completed restore for sideswipe! #####################


Start time: Sun Aug 23 22:31:26 PDT 2009
End   time: Sun Aug 23 22:34:46 PDT 2009
Duration  : 3.33 Minutes


---------------------------------------------------------------------------------------------------------------

Execute restore with output going to log file /tmp/ghettoVCB-restore.log (restore the last two VMs listed from above):

Input file:

[root@himalaya ~]# cat vms_to_restore
"/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/stupid ass vm with spaces/stupid ass vm with spaces-2009-08-20--1;/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage;2"
"/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/vMA-resize/vMA-resize-2009-08-23--1;/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage;3"

[root@himalaya ~]# ./ghettoVCB-restore.sh -c vms_to_restore -l /tmp/ghettoVCB-restore.log
Logging output to "/tmp/ghettoVCB-restore.log" ...

cat /tmp/ghettoVCB-restore.log

Display debug information level 1:

[root@himalaya ~]# ./ghettoVCB-restore.sh -c vms_to_restore -d 1

################ DEBUG MODE ##############
Virtual Machine: "VCAP"
VM_VMX: "VCAP.vmx"
VM_ORG_FOLDER: "VCAP-2009-08-20--1"
VM_FOLDER_NAME: "VCAP"
VMDK_LIST_TO_MODIFY:
scsi0:0.fileName = "VCAP.vmdk"
scsi0:0.fileName  = "VCAP-0.vmdk"
##########################################


################ DEBUG MODE ##############
Virtual Machine: "sideswipe"
VM_VMX: "sideswipe.vmx"
VM_ORG_FOLDER: "sideswipe-2009-08-20--1"
VM_FOLDER_NAME: "sideswipe"
VMDK_LIST_TO_MODIFY:
scsi0:0.fileName = "sideswipe.vmdk"
scsi0:0.fileName  = "sideswipe-0.vmdk"
scsi0:1.fileName = "/vmfs/volumes/4a31aece-e34d675a-72cf-003048d9586a/sideswipe/sideswipe.vmdk"
scsi0:1.fileName  = "sideswipe-1.vmdk"
scsi1:8.fileName = "/vmfs/volumes/4a31aece-e34d675a-72cf-003048d9586a/sideswipe/sideswipe_1.vmdk"
scsi1:8.fileName  = "sideswipe-2.vmdk"
scsi3:12.fileName = "sideswipe_1.vmdk"
scsi3:12.fileName  = "sideswipe-3.vmdk"
##########################################


Start time: Sun Aug 23 23:04:49 PDT 2009
End   time: Sun Aug 23 23:04:49 PDT 2009
Duration  : 0 Seconds

---------------------------------------------------------------------------------------------------------------

Display debug information level 2:

[root@himalaya ~]# ./ghettoVCB-restore.sh -c vms_to_restore -d 2
################## Restoring VM: stupid ass vm with spaces  #####################
==========> DEBUG MODE LEVEL 2 ENABLED <==========
Start time: Sun Aug 23 23:05:11 PDT 2009
Restoring VM from: "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/stupid ass vm with spaces/stupid ass vm with spaces-2009-08-20--1"
Restoring VM to Datastore: "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage" using Disk Format: "2gbsparse"
Creating VM directory: "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/stupid ass vm with spaces" ...
Copying "stupid ass vm with spaces.vmx" file ...
Restoring VM's VMDK(s) ...
Updating VMDK entry in "stupid ass vm with spaces.vmx" file ...

SOURCE: "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/stupid ass vm with spaces/stupid ass vm with spaces-2009-08-20--1/stupid ass vm with spaces.vmdk"
        ORIGINAL_VMX_LINE: -->scsi0:0.fileName = "stupid ass vm with spaces.vmdk"<--
DESTIONATION: "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/stupid ass vm with spaces/stupid ass vm with spaces-0.vmdk"
        MODIFIED_VMX_LINE: -->scsi0:0.fileName  = "stupid ass vm with spaces-0.vmdk"<--
Updating VMDK entry in "stupid ass vm with spaces.vmx" file ...

SOURCE: "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/stupid ass vm with spaces/stupid ass vm with spaces-2009-08-20--1/4a31aece-e34d675a-72cf-003048d9586a/stupid ass vm with spaces.vmdk"
        ORIGINAL_VMX_LINE: -->scsi0:1.fileName = "/vmfs/volumes/4a31aece-e34d675a-72cf-003048d9586a/stupid ass vm with spaces/stupid ass vm with spaces.vmdk"<--
DESTIONATION: "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/stupid ass vm with spaces/stupid ass vm with spaces-1.vmdk"
        MODIFIED_VMX_LINE: -->scsi0:1.fileName  = "stupid ass vm with spaces-1.vmdk"<--
Updating VMDK entry in "stupid ass vm with spaces.vmx" file ...

SOURCE: "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/stupid ass vm with spaces/stupid ass vm with spaces-2009-08-20--1/4a31aece-e34d675a-72cf-003048d9586a/stupid ass vm with spaces_1.vmdk"
        ORIGINAL_VMX_LINE: -->scsi0:2.fileName = "/vmfs/volumes/4a31aece-e34d675a-72cf-003048d9586a/stupid ass vm with spaces/stupid ass vm with spaces_1.vmdk"<--
DESTIONATION: "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/stupid ass vm with spaces/stupid ass vm with spaces-2.vmdk"
        MODIFIED_VMX_LINE: -->scsi0:2.fileName  = "stupid ass vm with spaces-2.vmdk"<--
Registering stupid ass vm with spaces ...
End time: Sun Aug 23 23:05:11 PDT 2009
################## Completed restore for stupid ass vm with spaces! #####################

################## Restoring VM: vMA-resize  #####################
==========> DEBUG MODE LEVEL 2 ENABLED <==========
Start time: Sun Aug 23 23:05:11 PDT 2009
Restoring VM from: "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/vMA-resize/vMA-resize-2009-08-23--1"
Restoring VM to Datastore: "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage" using Disk Format: "thin"
Creating VM directory: "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/vMA-resize" ...
Copying "vMA-resize.vmx" file ...
Restoring VM's VMDK(s) ...
Updating VMDK entry in "vMA-resize.vmx" file ...

SOURCE: "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/vMA-resize/vMA-resize-2009-08-23--1/vMA-resize.vmdk"
        ORIGINAL_VMX_LINE: -->scsi0:0.fileName = "vMA-resize.vmdk"<--
DESTIONATION: "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/vMA-resize/vMA-resize-0.vmdk"
        MODIFIED_VMX_LINE: -->scsi0:0.fileName  = "vMA-resize-0.vmdk"<--
Registering vMA-resize ...
End time: Sun Aug 23 23:05:11 PDT 2009
################## Completed restore for vMA-resize! #####################


Start time: Sun Aug 23 23:05:11 PDT 2009
End   time: Sun Aug 23 23:05:11 PDT 2009
Duration  : 0 Seconds

---------------------------------------------------------------------------------------------------------------

Change Log:

Known Issues:

Attachments:

vms_to_restore_sample.txt (317 bytes)
ghettoVCB-restore.sh (15.3 K)

Tags: 3.5, backup, esx3.5, ghettovcb, restore

Average User Rating

(1 rating)

Comments (11)

Sep 10, 2009 9:48 PM hvle says:

nice example, but what is up with the script name? "stupid ass vm with space"

Sep 23, 2009 2:18 PM knutfut says:

It works. I wiped my VM using VI Client (a bit scary), then I used this script to restore a previous backup. After a while it automatically popped up in the VI client.

Very nice. Thanks a lot.

Sep 23, 2009 10:12 PM lamw says: in response to: knutfut

Awesome to hear!

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

Oct 1, 2009 4:32 AM banthorpe says: in response to: lamw

when I run the restore script I get the following....complains about support for .tgz ???

~ # ./ghettoVCB-restore.sh -c vms_to_restore
Support for .tgz not supported - "/vmfs/volumes/backup/OCSSE_DR/OCSSE_DR-2009-10-01" will not be backed up!

Start time: Thu Oct 1 11:37:31 UTC 2009
End time: Thu Oct 1 11:37:31 UTC 2009
Duration : 0 Seconds

~ #

Oct 1, 2009 7:15 AM lamw says: in response to: banthorpe

As noted in this document, compression restore is not supported yet.

Oct 21, 2009 12:42 PM inwallav says:

I've tried walking away so I could look at this from a fresh mind, but I'm still getting a path problem on restore.

Could I get some input on maybe some syntax problems with this?? I'm really sure about the path. I've tried the aliased (as shown) and actual (with all the letters/numbers) for the final directory where the datastore resides.

I've run it from the root directory and in the datastore directory just for giggles. No luck.
restore.txt
#"<DIRECTORY or .TGZ>;<DATASTORE_TO_RESTORE_TO>;<DISK_FORMAT_TO_RESTORE>"
"/vmfs/volumes/macpro/vm/IBMvfusion;/vmfs/volumes/datastore1:datastore1;1"
Output
~ # ./ghettoVCB-restore.sh -c restore.txt
ERROR: Unable to verify datastore locateion: "/vmfs/volumes/datastore1:datastore1"! Ensure this exists

Thanks for looking

Brian

Oct 21, 2009 9:20 PM lamw says: in response to: inwallav

Remember to note that this script will ONLY work with backups taken with ghettoVCB.sh ... I only bring this up to ensure you're able to use the script and secondly, the format of your restore configuration doesn't match up from the sample template provided.

The first param /vmfs/volumes/macpro/vm/IBMvfusion should look something like /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/VCAP/VCAP-2009-08-18--1 where it points to the directory of a specific VM backup, not the root directory of the VM backups but a specific backup.

The second param /vmfs/volumes/datastore1:datastore1 is the datastore to restore to, I don't know if this is correct or a typo? Did you mean to say _/vmfs/volumes/datastore1_?

The third param 1 should just be the disk format and that looks fine to me.

Pleae double check your input and make sure /vmfs/volumes/datastore1:datastore1 does exists

Oct 22, 2009 9:20 AM inwallav says: in response to: lamw

Found the clue in this:
"Did you mean to say _/vmfs/volumes/datastore1_?"

I took verbatim what was in the original restore text like this:
"/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/VCAP/VCAP-2009-08-18--1;/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage;1"

"/vmfs/volumes/macpro/vm/IBMvfusion;/vmfs/volumes/datastore1:datastore1;1"

but this worked:
"/vmfs/volumes/macpro/vm/IBMvfusion;/vmfs/volumes/datastore1;1"

I'm assuming somewhere along the line the ":Storage" part worked at one time?? Or I was using it incorrectly.

Thanks for your help and VERY SOLID blogging/reply posts in all the ESX forums.

Brian

Oct 22, 2009 9:30 PM lamw says: in response to: inwallav

The example I gave was for our development environment and the datastore I was testing this with had the following display name: "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage", ":Storage" is not some string that needs to be appended You just need to provide the name of your datastore that you're restoring to.

Glad everything works

Oct 27, 2009 5:01 PM inwallav says: in response to: lamw

banthorpe,

I also got that message about "Support for .tgz not supported..." even though I had not chosen any file that was tgz'd.

What I found was that I had the wrong case (Macpro instead of macpro) on one of my nfs datastores. I was using essentially a wrong path. The script couldn't find the directory and halted. Just took some squinting on my part to see my error.

I added my own comment to the script in case I forget in the future too

Brian

Nov 16, 2009 9:52 AM xerolame says:

I cannot even run this script.
The script fails to create a log file even.
It is spouting off errors in the actual script involving lines 30 / 31

http://pastebin.com/m15e0794c has more information. I tried to throw in some diagnostic info as well.

Search VMware Communities Advanced Search

Actions

View as PDF

Incoming Links

vGhetto Script Repository

More by lamw

Up to Documents in VI: VMware ESX™ 3.5

Ghetto Tech Preview - ghettoVCB-restore.sh - Restoring VM's backed up from ghettoVCB to ESX(i) 3.5 and 4.0+

Created on: Aug 23, 2009 11:28 PM by lamw - Last Modified: Aug 25, 2009 11:13 PM by lamw

Table of Contents:

Description:

Requirements:

Setup:

Usage:

Sample Execution:

Change Log:

Known Issues:

Actions

Incoming Links

More by lamw

Post a comment

Communities

Community Resources

VMworld

Get Support

Other Links

Ghetto Tech Preview - ghettoVCB-restore.sh - Restoring VM's backed up from ghettoVCB to ESX(i) 3.5 and 4.0+

Created on: Aug 23, 2009 11:28 PM by lamw - Last Modified: Aug 25, 2009 11:13 PM by lamw

Table of Contents:

Description:

Requirements:

Setup:

Usage:

Sample Execution:

Change Log:

Known Issues:

Actions

Incoming Links

More by lamw

VMware Developer

Developer Sample Code

VMworld Sessions & Labs

Purchase PSO Credits Online

Community Hardware Software

VMware vSphere

Post a comment

Communities

Community Resources

VMworld

Get Support

Other Links