Just for fun! == ghettoVCBg3 == Just for fun! (Supports both Windows & Linux platform)

    Table of Contents:

      • Description
      • Requirements
      • Setup
      • Configurations
      • Usage
      • Sample Execution 
        • Backup VMs on Linux system
        • Backup VMs on Windows system
      • FAQ


    !!!! DISCLAIMER - PLEASE READ FAQ!!!! ==  THIS IS JUST FOR FUN AND WILL NOT BE SUPPORTED WHETHER IT WORKS OR NOT  FOR YOU  == !!!! DISCLAIMER - PLEASE READ FAQ!!!!

     

    Description:

    Awhile back I challenged myself to see if it was possible to create a  100% free scripted backup solution leveraging VMware's vSphere API and  freely available tools. The caveat being that I would not need to resort  to using the Service Console on classic ESX nor unsupported Busybox  (Tech Support Mode) console in ESXi and not violating any EULA  agreements. After a few days of diging around and looking at some of the  features/limitations/holes in the various APIs and tools, I manage to  come up with a solution that leveraged VMware's: vSphere API 4.0, vCLI  Toolkit 4.0, VIX 1.6.2 Toolkit and OVF Tool 1.0 to create a 100% free  scripted backup solution that would work across a Windows and Linux  platform.

    One important differnce with this script is the backups that are taken will NOT be stored on a datastore that ESX(i) host can talk to, but actually on  the system in which runs this script. This allows users to backup to any  type of backup datastore (LOCAL, NFS, CIFS, TAPE, etc.) reachable by  either a Windows or Linux platform. Also note the backup will be output  to OVF format

    Requirements:


    • ESX or ESXi 4.0 (free or licensed version)
    • vCLI 4.0 installed on either a Linux or Windows system (vMA 4.0 has this installed by default)
    • VIX 1.6.2
    • OVF Tool 1.0
    • Linux or Windows or vMA running the above 3 toolkits

     

     


    Setup:

    1) Download ghettoVCBg3.tar.gz to either a Linux or Windows system that supports the requirements listed above

    2) Extract the contents of the tarball using tar OR if you're on Windows use winrar to extract

    [vi-admin@scofield skunkworks]$ tar -zxvf ghettoVCBg3.tar.gz
    ghettoVCBg3/
    ghettoVCBg3/g3-linux.conf
    ghettoVCBg3/ghettoVCBg3.pl
    ghettoVCBg3/vmlist
    ghettoVCBg3/g3-windows.conf
    



    3) The script should be in a directory called ghettoVCBg3 or to whatever directory naming structure you extracted the contents into

    4. Download and install vCLI 4.0

    5. Download and install VIX 1.6.2

    6. Download and install OVF Tool 1.0

     


    Configurations


    Included in the tarball, exists two configuration file depending on the platform that'll be used to run the backup g3-linux.conf for UNIX/Linux systems and g3-windows.conf for Windows system.

    Here is what the g3-linux.conf looks like:

    [vi-admin@scofield ghettoVCBg3]$ cat g3-linux.conf
    LOG_LEVEL=debug
    LOCAL_TEMP_DIR=/tmp
    BACKUP_DIR=/tmp/backups
    PATH_TO_VIFS=/usr/bin/vifs
    PATH_TO_VMRUN=/usr/bin/vmrun
    PATH_TO_OVFTOOL=/usr/bin/ovftool
    



    Here is what the g3-windows.conf looks like:

    [vi-admin@scofield ghettoVCBg3]$ cat g3-windows.conf
    LOG_LEVEL=debug
    LOCAL_TEMP_DIR=C:\tmp
    BACKUP_DIR=C:\backups
    PATH_TO_VIFS=C:\Program Files\VMware\VMware vSphere CLI\bin\vifs.pl
    PATH_TO_VMRUN=C:\Program Files\VMware\VMware VIX\vmrun.exe
    PATH_TO_OVFTOOL=C:\Program Files\VMware\VMware OVF Tool\ovftool.exe
    



    You can leave the default so long as you've ensured that vCLI, VIX and  OVF Tool are installed in the locations defined by the configuration, if  not you'll need to edit the paths.

    The two variables that you'll want to change are LOCAL_TEMP_DIR which defines a scratch space for some small temporarily files that'll be created during the backup and BACKUP_DIR which is the directory local to the system running this script in which  the backups will be stored. You can also change the verbosity of the  logging from "debug" to "info" 

    The last file that'll need to be edited is the vmlist which is basically just a file containing the display name of VMs to be backed up from a specific ESX or ESXi host.

    Once the two files have been updated, then you can proceed in running the backup.


    Usage:

    [vi-admin@scofield ghettoVCBg3]$ ./ghettoVCBg3.pl
    Required command option 'conf' not specified.
    Required command option 'vmlist' not specified.
    
    Synopsis: ./ghettoVCBg3.pl OPTIONS
    
    
    Command-specific options:
       --conf (required)
          Configuration file
       --vmlist (required)
          A file containing a list of virtual machine(s) to be backed up on host
    
    Common VI options:
       --config (variable VI_CONFIG)
          Location of the VI Perl configuration file
       --credstore (variable VI_CREDSTORE)
          Name of the credential store file defaults to <HOME>/.vmware/credstore/vicredentials.xml on Linux and <APPDATA>/VMware/credstore/vicredentials.xml on Windows
       --encoding (variable VI_ENCODING, default 'utf8')
          Encoding: utf8, cp936 (Simplified Chinese), iso-8859-1 (German), shiftjis (Japanese)
       --help
          Display usage information for the script
       --passthroughauth (variable VI_PASSTHROUGHAUTH)
          Attempt to use pass-through authentication
       --passthroughauthpackage (variable VI_PASSTHROUGHAUTHPACKAGE, default 'Negotiate')
          Pass-through authentication negotiation package
       --password (variable VI_PASSWORD)
          Password
       --portnumber (variable VI_PORTNUMBER)
          Port used to connect to server
       --protocol (variable VI_PROTOCOL, default 'https')
          Protocol used to connect to server
       --savesessionfile (variable VI_SAVESESSIONFILE)
          File to save session ID/cookie to utilize
       --server (variable VI_SERVER, default 'localhost')
          VI server to connect to. Required if url is not present
       --servicepath (variable VI_SERVICEPATH, default '/sdk/webService')
          Service path used to connect to server
       --sessionfile (variable VI_SESSIONFILE)
          File containing session ID/cookie to utilize
       --url (variable VI_URL)
          VI SDK URL to connect to. Required if server is not present
       --username (variable VI_USERNAME)
          Username
       --verbose (variable VI_VERBOSE)
          Display additional debugging information
       --version
          Display version information for the script
    



     


    Sample Execution

    • Backup VMs on Linux system
    • Backup VMs on Windows system

     

    Backup VMs on Linux system


    In this example, I'll be using vMA 4.0 to backup up a single VM running  on ESXi 4.0 free license version and exporting out to OVF format.

    [vi-admin@scofield ghettoVCBg3]$ ./ghettoVCBg3.pl --server esxi4-1.primp-industries.com --username root --conf g3-linux.conf --vmlist vmlist
    Enter password:
            04-25-2010 20:34:38 -- : LOG_LEVEL = debug
            04-25-2010 20:34:38 -- : LOCAL_TEMP_DIR = /tmp
            04-25-2010 20:34:38 -- : BACKUP_DIR = /tmp/backups
            04-25-2010 20:34:38 -- : PATH_TO_VIFS = /usr/bin/vifs
            04-25-2010 20:34:38 -- : PATH_TO_VMRUN = /usr/bin/vmrun
            04-25-2010 20:34:38 -- : PATH_TO_OVFTOOL = /usr/bin/ovftool
            04-25-2010 20:34:38 -- :
            04-25-2010 20:34:38 -- : 0. Starting backup of ImportantDesktop
            04-25-2010 20:34:38 -- : 1. Copying .vmx to local system
            04-25-2010 20:34:40 -- : 2. Updating .vmx to Shadow VM Name
            04-25-2010 20:34:40 -- : 3. Upload modified .vmx to host
            04-25-2010 20:34:42 -- : 4. Snapshot source VM
            04-25-2010 20:34:45 -- : 5. Register Shadow VM
            04-25-2010 20:34:46 -- : 6. Start export of VM to OVF
            04-25-2010 20:34:52 -- : Opening VI source: vi://root@esxi4-1.primp-industries.com/ha-datacenter
    Opening OVF target: /tmp/backups/ImportantDesktop/ImportantDesktop-20100425_203446
    Target: /tmp/backups/ImportantDesktop/ImportantDesktop-20100425_203446/SHADOW__ImportantDesktop/SHADOW__ImportantDesktop.ovf
    Disk Transfer Completed
    Completed successfully
    
            04-25-2010 20:34:52 -- : 7. Unregister Shadow VM
            04-25-2010 20:34:53 -- : 8. Leaving Shadow VM files since ESXi Free License does not allow API r/w calls
            04-25-2010 20:34:53 -- : 9. Remove snapshot
            04-25-2010 20:34:55 -- : 0. Starting backup of vMA
            04-25-2010 20:34:55 -- : 1. Copying .vmx to local system
            04-25-2010 20:34:58 -- : 2. Updating .vmx to Shadow VM Name
            04-25-2010 20:34:58 -- : 3. Upload modified .vmx to host
            04-25-2010 20:35:01 -- : 4. Snapshot source VM
            04-25-2010 20:35:06 -- : 5. Register Shadow VM
            04-25-2010 20:35:07 -- : 6. Start export of VM to OVF
            04-25-2010 20:35:13 -- : Opening VI source: vi://root@esxi4-1.primp-industries.com/ha-datacenter
    Opening OVF target: /tmp/backups/vMA/vMA-20100425_203507
    Target: /tmp/backups/vMA/vMA-20100425_203507/SHADOW__vMA/SHADOW__vMA.ovf
    Disk Transfer Completed
    Completed successfully
    
            04-25-2010 20:35:13 -- : 7. Unregister Shadow VM
            04-25-2010 20:35:15 -- : 8. Leaving Shadow VM files since ESXi Free License does not allow API r/w calls
            04-25-2010 20:35:15 -- : 9. Remove snapshot
            04-25-2010 20:35:17 -- : 10. Cleaning up
            04-25-2010 20:35:17 -- : 11. Backup Completed!
    



    Let's take a look at where the backups are stored:

    [vi-admin@scofield ghettoVCBg3]$ tree /tmp/backups/
    /tmp/backups/
    |-- ImportantDesktop
    |   |-- ImportantDesktop-20100425_202629
    |   |   `-- SHADOW__ImportantDesktop
    |   |       |-- SHADOW__ImportantDesktop-disk1.vmdk
    |   |       |-- SHADOW__ImportantDesktop.mf
    |   |       `-- SHADOW__ImportantDesktop.ovf
    |   |-- ImportantDesktop-20100425_202852
    |   |   `-- SHADOW__ImportantDesktop
    |   |       |-- SHADOW__ImportantDesktop-disk1.vmdk
    |   |       |-- SHADOW__ImportantDesktop.mf
    |   |       `-- SHADOW__ImportantDesktop.ovf
    |   |-- ImportantDesktop-20100425_203057
    |   |   `-- SHADOW__ImportantDesktop
    |   |       |-- SHADOW__ImportantDesktop-disk1.vmdk
    |   |       |-- SHADOW__ImportantDesktop.mf
    |   |       `-- SHADOW__ImportantDesktop.ovf
    |   `-- ImportantDesktop-20100425_203446
    |       `-- SHADOW__ImportantDesktop
    |           |-- SHADOW__ImportantDesktop-disk1.vmdk
    |           |-- SHADOW__ImportantDesktop.mf
    |           `-- SHADOW__ImportantDesktop.ovf
    `-- vMA
        |-- vMA-20100425_202648
        |   `-- SHADOW__vMA
        |       |-- SHADOW__vMA-disk1.vmdk
        |       |-- SHADOW__vMA.mf
        |       `-- SHADOW__vMA.ovf
        |-- vMA-20100425_202911
        |   `-- SHADOW__vMA
        |       |-- SHADOW__vMA-disk1.vmdk
        |       |-- SHADOW__vMA.mf
        |       `-- SHADOW__vMA.ovf
        |-- vMA-20100425_203116
        |   `-- SHADOW__vMA
        |       |-- SHADOW__vMA-disk1.vmdk
        |       |-- SHADOW__vMA.mf
        |       `-- SHADOW__vMA.ovf
        `-- vMA-20100425_203507
            `-- SHADOW__vMA
                |-- SHADOW__vMA-disk1.vmdk
                |-- SHADOW__vMA.mf
                `-- SHADOW__vMA.ovf
    
    18 directories, 24 files
    



    Backup VMs on Windows system

    In this example, I'll be using Windows XP to backup a single VM running  on ESXi 4.0 free license version and exporting out to OVF format.

    C:\Documents and Settings\Administrator\Desktop\ghettoVCBg3>ghettoVCBg3.pl --ser
    ver esxi4-1.primp-industries.com --username root --conf g3-windows.conf --vmlist
     vmlist
    Enter password:
            04-25-2010 20:40:03 -- : 0. Starting backup of ImportantDesktop
            04-25-2010 20:40:03 -- : 1. Copying .vmx to local system
            04-25-2010 20:40:07 -- : 2. Updating .vmx to Shadow VM Name
            04-25-2010 20:40:07 -- : 3. Upload modified .vmx to host
            04-25-2010 20:40:11 -- : 4. Snapshot source VM
            04-25-2010 20:40:14 -- : 5. Register Shadow VM
            04-25-2010 20:40:16 -- : 6. Start export of VM to OVF
            04-25-2010 20:40:25 -- : Opening VI source: vi://root@esxi4-1.primp-indu
    stries.com/ha-datacenter
    Opening OVF target: C:\backups\ImportantDesktop\ImportantDesktop-20100425_204016
    
    Target: C:\backups\ImportantDesktop\ImportantDesktop-20100425_204016\SHADOW__Imp
    ortantDesktop\SHADOW__ImportantDesktop.ovf
    Disk Transfer Completed
    Completed successfully
    
            04-25-2010 20:40:25 -- : 7. Unregister Shadow VM
            04-25-2010 20:40:26 -- : 8. Leaving Shadow VM files since ESXi Free Lice
    nse does not allow API r/w calls
            04-25-2010 20:40:26 -- : 9. Remove snapshot
            04-25-2010 20:40:28 -- : 0. Starting backup of vMA
            04-25-2010 20:40:28 -- : 1. Copying .vmx to local system
            04-25-2010 20:40:32 -- : 2. Updating .vmx to Shadow VM Name
            04-25-2010 20:40:32 -- : 3. Upload modified .vmx to host
            04-25-2010 20:40:36 -- : 4. Snapshot source VM
            04-25-2010 20:40:42 -- : 5. Register Shadow VM
            04-25-2010 20:40:44 -- : 6. Start export of VM to OVF
            04-25-2010 20:40:52 -- : Opening VI source: vi://root@esxi4-1.primp-indu
    stries.com/ha-datacenter
    Opening OVF target: C:\backups\vMA\vMA-20100425_204044
    Target: C:\backups\vMA\vMA-20100425_204044\SHADOW__vMA\SHADOW__vMA.ovf
    Disk Transfer Completed
    Completed successfully
    
            04-25-2010 20:40:52 -- : 7. Unregister Shadow VM
            04-25-2010 20:40:54 -- : 8. Leaving Shadow VM files since ESXi Free Lice
    nse does not allow API r/w calls
            04-25-2010 20:40:54 -- : 9. Remove snapshot
            04-25-2010 20:40:58 -- : 10. Cleaning up
            04-25-2010 20:40:58 -- : 11. Backup Completed!
    



    Here you can see the two directories created by the script, one used  for scratch space and the other containing the actual VM backups

    backup_dirs.png

    Here are the backups in OVF format

    backups.png

     


     

    FAQ


    1. !!!! DISCLAIMER - PLEASE READ FAQ!!!!  == THIS IS JUST FOR FUN AND WILL NOT BE SUPPORTED WHETHER IT WORKS OR  NOT FOR YOU  == !!!! DISCLAIMER - PLEASE READ FAQ!!!!

    2. Why am I not supporting this? This is really a hack, just to show  what you can do with the various vSphere API/Toolkits. Properly handling  errors/other features is pretty difficult as the scope and ability of  the script is very limited. What I wanted to show is how easily it can  be to create some very neat backup solutions to support any of the  various use cases out there and how rich the VMware API/SDK Toolkits  are. If you're looking for a 100% scripted backup solution, please refer  to the popular ghettoVCB.sh - Free alternative for backing up VM's for ESX(i) 3.5 and 4.x+ - Updated 09/28/2010

    3. If you're using ESXi and the free license, you'll notice the "shadow"  VM files are left in the same directory as the original VMs. This is  due to the limitation of the vSphere API when using the free license in  which the "shadow" files can not be deleted. If you're using a license  version of ESXi, then these temporarily files will be removed

    4. How does the backup work? Basically what I do is make a copy of the VM's .vmx configuration file and create a "shadow" VM that is registered on the host. A snapshot is taken of the VM to be  backed up and then the shadow VM is registered on the host, prior to the  snapshot, the .vmx configuration points to the VM's disk. This then  allows OVF Tool to query a valid VM which is really just a  pointer to the VM's disk, thus allowing an export of the disks. Once the  export has been completed, the snapshot is then committed back into the  VM and the shadow VM is unregistered. As noted from FAQ #2, if you're  using the free license version of ESXi, the files can not be removed due  to API's limitation under "free" mode. These files should not harm the  existing VM and are place holders. If you're using a licensed version of  ESX or ESXi, then these files are properly discarded upon successful  backup.