ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Table of Contents:

    • Description
    • Features
    • Requirements
    • ESX/ESXi Version Support Table
    • Setup
    • Addings ESX/ESXi host(s) to VIMA/vMA
    • Configurations
    • Usage
    • Sample Execution 
      • Backup VMs located in a list
      • Backup VMs using individual backup policies
      • Dryrun Mode
    • Experimental Features
    • FAQ
    • Useful Links
    • Change Log

Description:

This tool is the follow up release of the ghettoVCB

backup utility which allows users to perform backups of virtual machines residing on ESX(i) 3.5+/4.x+

servers using methodology similar to VMware's VCB

tool. By incorporating highly constructive feedback from the VMware  community and utilizing the existing VI API, ghettoVCB’s framework was  completely rewritten to be harder, better, faster, stronger

.

The primary motivation for ghettoVCBg2 was to provide ESXi users with  access to the utility without relying on unlocking and utilizing the  unsupported console. To satisfy this requirement, the rebuilt framework  takes advantage of the

VI Perl Toolkit/vSphere SDK for Perl which is present in the

VIMA/vMA virtual appliance provided by VMware. As a result, ghettoVCBg2 provides  a more proper backup solution that administrators can utilize in their  virtual infrastructure.

As it stands, it is a requirement that ghettoVCB be invoked directly on  each of the ESX servers hosting virtual machines in need of a backup. By  taking advantage of VIMA/vMA, the entire backup process becomes  centralized in turn eliminating the abovementioned requirement.  Additionally, by leveraging the vi-fastpass library, unattended  authentication between VIMA/vMA and target ESX servers is made possible.  This alleviates the need to continually pass in credentials for each  ESX server.

By centralizing the backup process, individual virtual machine backup  lists pertaining to their respective ESX(i) servers are no longer  required. ghettoVCBg2 has the ability to identify virtual machines and  their respective hosts, potentially eliminating any past confusion  associated with maintaining multiple lists.

In its current configuration, the script will allow up to 3 unique  backups of the Virtual Machine before it will overwrite the previous  backups; this however, can be modified to fit procedures if need be.  Please be diligent in running the script in a test or staging  environment before using it on production live Virtual Machines; this  script functions well within our environment but there is a chance that  it may not fit well into other environments.

Features

  • Support for logging (normal & verbose)
  • Single VM backup list across multiple ESX/ESXi host(s)
  • Credential-less backups (so long as host(s) are being managed by VIMA/vMA)
  • Online back up of VM(s)
  • Only valid VMDKs presented to the VM will be backed up
  • Support for multiple VMDK backup per VM
  • Preserve original powerState of VM(s)
  • Ability to shutdown guestOS, initiate backup process and power on VM afterwards with the option of hardpower timeout
  • Ensure that snapshot removal process completes prior to continuing the backup process
  • VM(s) that initially contain snapshots and Physical RDMs (raw device mapppings) will not be backed up
  • Support for VM(s) with Virtual RDMs
  • Ability to specify the number of backup rotations per VM
  • Output back up VMDKs in ZEROEDTHICK (default behavior), EAGERZEROEDTHICK, 2GB SPARSE or THIN format
  • Ouput backup VMDKs using either BUSLOGIC or LSILOGIC adapter type
  • Fully support VMDKs stored across multiple datastores
  • VM snapshot memory and quiesce options
  • Individual VM backup policy (supported on ESX(i) 3.5u2+ & ESX(i) 4.x+)
  • Email logs (Experimental Support) NEW!
  • Ability to include/exclude specific VMDK(s) per VM (requires individual VM backup policy setup) NEW!
  • Independent disk aware (will ignore VMDK) NEW!
  • Additional debugging information including dry run execution NEW!


Requirements:

VMs running on ESX(i) 3.5u2+ or ESX(i) 4.x+

VMware VIMA 1.0 or VMware vMA 4.x


ESX/ESXi Version Support Table

VersionSupportAlternative
ESX 3.5u2Requires at least Foundation/Essentials licenseghettoVCB\
ESX 3.5u3Requires at least Foundation/Essentials licenseghettoVCB\
ESX 3.5u4Requires at least Foundation/Essentials licenseghettoVCB\
ESXi 3.5u2Works for freeghettoVCB\
ESXi 3.5u3Works for freeghettoVCB\
ESXi 3.5u4Requires at least Foundation/Essentials licenseghettoVCB\
ESX 4.0Requires at least Foundation/Essentials licenseghettoVCB\
ESXi 4.0Requires at least Foundation/Essentials licenseghettoVCB\
ESX 4.1Requires at least Foundation/Essentials licenseghettoVCB\
ESXi 4.1Requires at least Foundation/Essentials licenseghettoVCB\



Setup:

1) Download ghettoVCBg2.tar.gz

to your VIMA 1.0/vMA 4.x system

2) Extract the contents of the tarball

[vi-admin@scofield ~]$ tar -zxvf ghettoVCBg2.tar.gz
ghettoVCBg2/
ghettoVCBg2/ghettoVCBg2-vm_backup_configuration_template
ghettoVCBg2/ghettoVCBg2.pl

3) The script should be in a directory called

ghettoVCBg2


Addings ESX/ESXi host(s) to VIMA/vMA

Prior to starting, ensure that the VMware VIMA/vMA host is managing the appropriate ESX(i) 3.5u2/4.x+ host(s)

1. Add ESXi hosts to VIMA/vMA managment using vifp

A password prompt for the vi-admin account and root password to the pertinent ESXi host(s) will be presented.

[vi-admin@vima-primp-industries ~]$ sudo vifp addserver olga.resnet.ucsb.edu
root@olga.resnet.ucsb.edu's password:
[vi-admin@vima-primp-industries ~]$

2. Verify that the appropriate hosts are being managed by VIMA/vMA

[vi-admin@vima-primp-industries ~]$ sudo vifp listservers
olga.resnet.ucsb.edu
himalaya.primp-industries.com


Configurations

The following variables (similar to that of ghettoVCB) need to be defined within the script prior to execution.

Defining the datastore in which the backups are stored:

my $VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups";

Defining the name of folder that will contain backups (if folder does not exist, it will automatically be created):

my $VM_BACKUP_DIRECTORY = "WILLIAM_BACKUPS";

Defining the backup rotation per VM:

my $VM_BACKUP_ROTATION_COUNT = "3";

Defining the backup disk format (zeroedthick, eagerzeroedthick, thin, and 2gbsparse are available):

my $DISK_BACKUP_FORMAT = "zeroedthick";

Defining the adapter type for backed up VMDK (buslogic, and lsilogic are available):

my $ADAPTER_FORMAT = "buslogic";

Defining whether the VM is powered down or not prior to backup (1 = enable, 0 = disable):

Note: VM(s) that are powered off will not require snapshoting

my $POWER_VM_DOWN_BEFORE_BACKUP = "0";

Defining whether virtual machine memory is snapped and if quiescing is enabled (1 = enable, 0 = disable):

Note: By default both are disabled

my $VM_SNAPSHOT_MEMORY = "0";
my $VM_SNAPSHOT_QUIESCE = "0";

Defining the detail of logging (verbose or normal):

my $LOG_LEVEL = "normal";

*Defining VMDK(s) to backup from a particular VM either a list of vmdks or "all"

VM_VMDK_FILES="myvmdk.vmdk"

Experimental Support Configs:

Note: Port 25 by default

my $SEND_MAIL = "no";
my $EMAIL_HOST = "emailserver";
my $EMAIL_DOMAIN = "localhost.localdomain";
my $EMAIL_TO = 'William Lam <william@primp-industries.com.com>';
my $EMAIL_FROM = 'ghettoVCBg2 <ghettoVCBg2@primp-industries.com.com>';

Ensure that you do not edit past this section:

########################## DO NOT MODIFY PAST THIS LINE ##########################


Usage:

[vi-admin@scofield ~]$ ./ghettoVCBg2.pl
Required command option 'vmlist' not specified.

Synopsis: ./ghettoVCBg2.pl OPTIONS


Command-specific options:
   --config_dir
      Name of directory containing VM(s) backup configurations
   --dryrun (default '0')
      Set to 1 to enable dryrun mode (default 0)
   --output (default '/tmp/ghettoVCBg2.log')
      Full path to output log (default /tmp/ghettoVCBg2.log)
   --vmlist (required)
      A file containing a list of virtual machine(s) to be backed up on host

Common VI options:
   --config (variable VI_CONFIG)
      Location of the VI Perl configuration file
   --credstore (variable VI_CREDSTORE)
      Name of the credential store file defaults to <HOME>/.vmware/credstore/vicredentials.xml on Linux and <APPDATA>/VMware/credstore/vicredentials.xml on Windows
   --encoding (variable VI_ENCODING, default 'utf8')
      Encoding: utf8, cp936 (Simplified Chinese), iso-8859-1 (German), shiftjis (Japanese)
   --help
      Display usage information for the script
   --passthroughauth (variable VI_PASSTHROUGHAUTH)
      Attempt to use pass-through authentication
   --passthroughauthpackage (variable VI_PASSTHROUGHAUTHPACKAGE, default 'Negotiate')
      Pass-through authentication negotiation package
   --password (variable VI_PASSWORD)
      Password
   --portnumber (variable VI_PORTNUMBER)
      Port used to connect to server
   --protocol (variable VI_PROTOCOL, default 'https')
      Protocol used to connect to server
   --savesessionfile (variable VI_SAVESESSIONFILE)
      File to save session ID/cookie to utilize
   --server (variable VI_SERVER, default 'localhost')
      VI server to connect to. Required if url is not present
   --servicepath (variable VI_SERVICEPATH, default '/sdk/webService')
      Service path used to connect to server
   --sessionfile (variable VI_SESSIONFILE)
      File containing session ID/cookie to utilize
   --url (variable VI_URL)
      VI SDK URL to connect to. Required if server is not present
   --username (variable VI_USERNAME)
      Username
   --verbose (variable VI_VERBOSE)
      Display additional debugging information
   --version
      Display version information for the script

Note: There are some additional command line arguments that can be provided upon runtime: --debug, --output and --vmlist. --vmlist is the only parameter required. The rest will have defaults if not explicitly defined.

--dryrun

  • Dry run information will be written to the log and no backup will take place

--output

  • Name of the output log, if the same log is used, it will  automatically append (ensure you provide full path e.g.  /tmp/ghettoVCBg2.log)

--config_dir

  • This directory will contain the individual VM backup policy files.  If a virtual machine has an associated policy file, the global  configuration will be ignored for that backup instance.

The input to this script is a file that contains the display name of the  virtual machine(s) separated by a newline. When creating this file on a  non-Linux/UNIX system, you may introduce a ^M character which may cause  the script to misbehave. To ensure that this does not occur, please  create the file on the ESX/ESXi host.

Provided is a sample of what the file should contain:

[vi-admin@scofield ~]$ cat virtual_machine_backup_list
VIMA
vMA-2


Sample Execution

  • Backup VMs located in a list
  • Backup VMs based on individual VM backup policies
  • Debug Mode
  • Dry run Mode

Backup VMs located in a list

  • Log verbosity: info (default)
  • Log output: /tmp/ghettoVCBg2.log (default)

DISK_FORMAT2gbsparse
ADAPTER_FORMATbuslogic
LOG_LEVELinfo
VM_SNAPSHOT_MEMORY0
VM_SNAPSHOT_QUIESCE0
VMS_TO_BACKUP(VIMA,vMA-2)


[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --vmlist virtual_machine_backup_list

[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
        11-13-2009 23:34:53 --  info: ============================== ghettoVCBg2 LOG START ==============================
        11-13-2009 23:34:53 --  info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
        11-13-2009 23:34:53 --  info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
        11-13-2009 23:34:53 --  info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
        11-13-2009 23:34:53 --  info: CONFIG - DISK_BACKUP_FORMAT = 2gbsparse
        11-13-2009 23:34:53 --  info: CONFIG - ADAPTER_FORMAT = buslogic
        11-13-2009 23:34:53 --  info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
        11-13-2009 23:34:53 --  info: CONFIG - VM_SNAPSHOT_MEMORY = NO
        11-13-2009 23:34:53 --  info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
        11-13-2009 23:34:53 --  info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-13
        11-13-2009 23:34:53 --  info: CONFIG - VM_VMDK_FILES = all

        11-13-2009 23:34:58 --  info: Initiate backup for VIMA found on himalaya.primp-industries.com
        11-13-2009 23:35:18 --  info: VIMA has 1 VMDK(s)
        11-13-2009 23:37:37 --  info: Backup completed for VIMA!

        11-13-2009 23:37:37 --  info: Initiate backup for vMA-2 found on himalaya.primp-industries.com
        11-13-2009 23:37:47 --  info: vMA-2 has 1 VMDK(s)
        11-13-2009 23:40:07 --  info: Backup completed for vMA-2!

        11-13-2009 23:40:10 --  info: ============================== ghettoVCBg2 LOG END ==============================

Backup VMs based on individual VM backup policies and log output to /tmp/ghettoVCB.log

  • Log verbosity: info (default)
  • Log output: /tmp/ghettoVCB.log (default)

VIMA

VM_BACKUP_DATASTOREdlgCore-NFS-bigboi.VM-Backups
VM_BACKUP_DIRECTORYWILLIAM_BACKUPS
DISK_FORMATzeroedthick
ADAPTER_FORMATbuslogic
LOG_LEVELdebug
VM_SNAPSHOT_MEMORY1
VM_SNAPSHOT_QUIESCE1
VM_VMDK_FILESVIMA_1.vmdk


vMA-2

VM_BACKUP_DATASTOREdlgCore-NFS-bigboi.VM-Backups
VM_BACKUP_DIRECTORYvGHETTO_BACKUPS
DISK_FORMATthin
ADAPTER_FORMATlsilogic
LOG_LEVELinfo
VM_SNAPSHOT_MEMORY0
VM_SNAPSHOT_QUIESCE0
VM_VMDK_FILESall


1. Create folder to hold individual VM backup policies (can be named anything):

[vi-admin@scofield ~]$ mkdir backup_config

2. Create individual VM backup policies for each VM and ensure each file  is named exactly as the display name of the VM being backed up (use  provided template to create duplicates):

[vi-admin@scofield backup_config]$ cp ghettoVCBg2-vm_backup_configuration_template VIMA
[vi-admin@scofield backup_config]$ cp ghettoVCBg2-vm_backup_configuration_template vMA-2

4. Using a preferred editor (nano or vi), edit each VM backup policy file

Listing of VM backup policy within backup configuration directory

[vi-admin@scofield backup_config]$ ls
ghettoVCBg2-vm_backup_configuration_template  VIMA  vMA-2

5. Pass in the value, backup_config, into the --config_dir parameter

(e.g.)
--config_dir backupConfigs

Example of two distinct VM backup policies will be applied to VIMA and vMA-2

Backup policy for "VIMA" (backup only 1 specific VMDKs)

[vi-admin@scofield backup_config]$ cat VIMA
VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups"
VM_BACKUP_DIRECTORY = "WILLIAM_BACKUPS"
VM_BACKUP_ROTATION_COUNT = "3"
DISK_BACKUP_FORMAT = "zeroedthick"
ADAPTER_FORMAT = "buslogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
VM_SNAPSHOT_MEMORY = "1"
VM_SNAPSHOT_QUIESCE = "1"
LOG_LEVEL = "debug"
VM_VMDK_FILES = "VIMA_1.vmdk"

Backup policy for VM "vMA-2" (backup all VMDKs found)

VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups"
VM_BACKUP_DIRECTORY = "vGHETTO_BACKUPS"
VM_BACKUP_ROTATION_COUNT = "3"
DISK_BACKUP_FORMAT = "thin"
ADAPTER_FORMAT = "lsilogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
VM_SNAPSHOT_MEMORY = "0"
VM_SNAPSHOT_QUIESCE = "0"
LOG_LEVEL = "info"
VM_VMDK_FILES = "all"

The following is an output of the script when utilizing the new --config_dir parameter with debug mode enabled.

Note: --vmlist is still a required  parameter with the exception that backup policy files must exist for  each VM in the list else the VM will be ignored in the backup process.

Execution of backup

[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --config_dir backup_config --vmlist virtual_machine_backup_list

[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
        11-14-2009 00:18:41 --  info: ============================== ghettoVCBg2 LOG START ==============================
        11-14-2009 00:18:45 --  info: CONFIG - USING CONFIGURATION FILE = VIMA
        11-14-2009 00:18:45 --  info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
        11-14-2009 00:18:45 --  info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
        11-14-2009 00:18:45 --  info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
        11-14-2009 00:18:45 --  info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
        11-14-2009 00:18:45 --  info: CONFIG - ADAPTER_FORMAT = buslogic
        11-14-2009 00:18:45 --  info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
        11-14-2009 00:18:45 --  info: CONFIG - VM_SNAPSHOT_MEMORY = YES
        11-14-2009 00:18:45 --  info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
        11-14-2009 00:18:45 --  info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
        11-14-2009 00:18:45 --  info: CONFIG - VM_VMDK_FILES = VIMA_1.vmdk

        11-14-2009 00:18:45 --  info: Initiate backup for VIMA found on himalaya.primp-industries.com
        11-14-2009 00:18:55 --  debug: VIMA original powerState: poweredOn
        11-14-2009 00:18:55 --  debug: Creating Snapshot "ghettoVCBg2-snapshot-2009-11-14" for VIMA
        11-14-2009 00:19:45 --  info: VIMA has 2 VMDK(s)
        11-14-2009 00:19:45 --  debug: findVMDKFile: Found VMDK File: VIMA_1.vmdk
        11-14-2009 00:19:45 --  debug: backupVMDK: Backing up "[http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk" to "[http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14/VIMA_1.vmdk"
        11-14-2009 00:19:45 --  debug: backupVMDK: Signal copyThread to start
        11-14-2009 00:19:45 --  debug: backupVMDK: Backup progress: Elapsed time 0 min
        11-14-2009 00:19:45 --  debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
        11-14-2009 00:19:45 --  debug: CopyThread: Start backing up VMDK(s) ...
        11-14-2009 00:19:52 --  debug: copyTask: send copySuccess message ...
        11-14-2009 00:19:52 --  debug: copyTask: waiting for next job and sleep ...
        11-14-2009 00:20:46 --  debug: backupVMDK: Successfully completed backup for [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk Elapsed time: 1 min
        11-14-2009 00:20:49 --  debug: Removing Snapshot "ghettoVCBg2-snapshot-2009-11-14" for VIMA
        11-14-2009 00:20:51 --  debug: checkVMBackupRotation: Starting ...
        11-14-2009 00:20:57 --  debug: Purging [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-13--3 due to rotation max
        11-14-2009 00:21:05 --  info: Backup completed for VIMA!

        11-14-2009 00:21:05 --  debug: reConfigureBackupParams: VM - vMA-2
        11-14-2009 00:21:05 --  info: CONFIG - USING CONFIGURATION FILE = vMA-2
        11-14-2009 00:21:05 --  info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
        11-14-2009 00:21:05 --  info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
        11-14-2009 00:21:05 --  info: CONFIG - VM_BACKUP_DIRECTORY = vGHETTO_BACKUPS
        11-14-2009 00:21:05 --  info: CONFIG - DISK_BACKUP_FORMAT = thin
        11-14-2009 00:21:05 --  info: CONFIG - ADAPTER_FORMAT = lsilogic
        11-14-2009 00:21:05 --  info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
        11-14-2009 00:21:05 --  info: CONFIG - VM_SNAPSHOT_MEMORY = NO
        11-14-2009 00:21:05 --  info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
        11-14-2009 00:21:05 --  info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
        11-14-2009 00:21:05 --  info: CONFIG - VM_VMDK_FILES = all

        11-14-2009 00:21:05 --  info: Initiate backup for vMA-2 found on himalaya.primp-industries.com
        11-14-2009 00:21:15 --  info: vMA-2 has 1 VMDK(s)
        11-14-2009 00:23:22 --  info: Backup completed for vMA-2!

        11-14-2009 00:23:25 --  info: ============================== ghettoVCBg2 LOG END ==============================

Dry run Mode (no backup will take place)

  • Log verbosity: drymode
  • Log output: /tmp/ghettoVCB.log (default)

[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --config_dir backup_config --vmlist virtual_machine_backup_list --dryrun 1

[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
        11-14-2009 00:26:29 --  info: ============================== ghettoVCBg2 LOG START ==============================
        11-14-2009 00:26:34 --  info: CONFIG - USING CONFIGURATION FILE = VIMA
        11-14-2009 00:26:34 --  info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
        11-14-2009 00:26:34 --  info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
        11-14-2009 00:26:34 --  info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
        11-14-2009 00:26:34 --  info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
        11-14-2009 00:26:34 --  info: CONFIG - ADAPTER_FORMAT = buslogic
        11-14-2009 00:26:34 --  info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
        11-14-2009 00:26:34 --  info: CONFIG - VM_SNAPSHOT_MEMORY = YES
        11-14-2009 00:26:34 --  info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
        11-14-2009 00:26:34 --  info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
        11-14-2009 00:26:34 --  info: CONFIG - VM_VMDK_FILES = VIMA_1.vmdk

        11-14-2009 00:26:34 --  info: ---------- DRYRUN DEBUG INFO VIMA ----------
        11-14-2009 00:26:34 --  info: DEBUG - Host Build: VMware ESX 4.0.0 build-164009
        11-14-2009 00:26:34 --  info: DEBUG - Host: himalaya.primp-industries.com
        11-14-2009 00:26:34 --  info: DEBUG - Virtual Machine: VIMA
        11-14-2009 00:26:34 --  info: DEBUG - VM ConfigPath: [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA.vmx
        11-14-2009 00:26:34 --  info: DEBUG - VMX File: VIMA.vmx
        11-14-2009 00:26:34 --  info: DEBUG - BackupConfigPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14/VIMA.vmx
        11-14-2009 00:26:34 --  info: DEBUG - BackupPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14
        11-14-2009 00:26:34 --  info: DEBUG - VM Datastore: himalaya-local-SAS.Savvio
        11-14-2009 00:26:34 --  info: DEBUG - VMDK(s):
        11-14-2009 00:26:34 --  info: DEBUG - [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA.vmdk
        11-14-2009 00:26:34 --  info: DEBUG - [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk
        11-14-2009 00:26:34 --  info: ---------- DRYRUN DEBUG INFO VIMA ----------

        11-14-2009 00:26:34 --  debug: reConfigureBackupParams: VM - vMA-2
        11-14-2009 00:26:34 --  info: CONFIG - USING CONFIGURATION FILE = vMA-2
        11-14-2009 00:26:34 --  info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
        11-14-2009 00:26:34 --  info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
        11-14-2009 00:26:34 --  info: CONFIG - VM_BACKUP_DIRECTORY = vGHETTO_BACKUPS
        11-14-2009 00:26:34 --  info: CONFIG - DISK_BACKUP_FORMAT = thin
        11-14-2009 00:26:34 --  info: CONFIG - ADAPTER_FORMAT = lsilogic
        11-14-2009 00:26:34 --  info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
        11-14-2009 00:26:34 --  info: CONFIG - VM_SNAPSHOT_MEMORY = NO
        11-14-2009 00:26:34 --  info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
        11-14-2009 00:26:34 --  info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
        11-14-2009 00:26:34 --  info: CONFIG - VM_VMDK_FILES = all

        11-14-2009 00:26:34 --  info: ---------- DRYRUN DEBUG INFO vMA-2 ----------
        11-14-2009 00:26:34 --  info: DEBUG - Host Build: VMware ESX 4.0.0 build-164009
        11-14-2009 00:26:34 --  info: DEBUG - Host: himalaya.primp-industries.com
        11-14-2009 00:26:34 --  info: DEBUG - Virtual Machine: vMA-2
        11-14-2009 00:26:34 --  info: DEBUG - VM ConfigPath: [http://himalaya-local-SAS.Constellation|http://himalaya-local-SAS.Constellation] vMA-2/vMA-2.vmx
        11-14-2009 00:26:34 --  info: DEBUG - VMX File: vMA-2.vmx
        11-14-2009 00:26:34 --  info: DEBUG - BackupConfigPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] vGHETTO_BACKUPS/vMA-2/vMA-2-2009-11-14/vMA-2.vmx
        11-14-2009 00:26:34 --  info: DEBUG - BackupPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] vGHETTO_BACKUPS/vMA-2/vMA-2-2009-11-14
        11-14-2009 00:26:34 --  info: DEBUG - VM Datastore: himalaya-local-SAS.Constellation
        11-14-2009 00:26:34 --  info: DEBUG - VMDK(s):
        11-14-2009 00:26:34 --  info: DEBUG - [http://himalaya-local-SAS.Constellation|http://himalaya-local-SAS.Constellation] vMA-2/vMA-2.vmdk
        11-14-2009 00:26:34 --  info: ---------- DRYRUN DEBUG INFO vMA-2 ----------

        11-14-2009 00:26:37 --  info: ============================== ghettoVCBg2 LOG END ==============================


FAQ

0Q: I'm having some issues or would like to provide feedback

0A: Please join the ghettoVCBg2 Group to post your comments/questions

1Q: I'm using ESXi and free licensed version ( I did not pay for anyting ), will this script still work?

1A: No, you need to have a licensed version of ESXi to use this script.

2Q: I have a VM listed in my backup file but no backup of that VM  took place. The log stated that the VM could not be found. How could  this be? I know that it’s running on host X…

2A: Ensure the ESX/ESXi host is being managed by VMware VIMA/vMA (sudo vifp listservers to verify)

3Q: Can I schedule backups to take place hourly, daily, monthly, yearly?

3A: Yes, do a search online for crontab.

4Q: I want to schedule my backup on Windows, how do I do this?

4A: Do a search for

plink. Make sure you have paired SSH keys setup between your Windows system and VIMA/vMA.

5Q: Do I have to use VMware VIMA/vMA to initiate the backup?

5A: Yes, vi-fastpass is being used and only VIMA/vMA provides  this capability. However, you can kickoff the process from another host  (*nix/win)

4Q: I only have a single ESXi host. I want to take backups and  store them somewhere else. The problem is: I don't have NFS, iSCSI nor  FC SAN. What can I do?

4A: You can use local storage to store your backups assuming that  you have enough space on the destination datastore.  Afterwards, you  can use scp (WinSCP/FastSCP) or vifs (vCLI) to transfer the backups from the ESXi host to your local desktop.

5Q: I’m pissed; the backup is taking too long. My datastore is of type X?

5A: YMMV, take a look at your storage configuration and make sure it is optimized.

6Q: I noticed that the backup rotation is occurring after a  backup. I don't have enough local storage space, can the process be  changed?

6A: This is primarily done to ensure that you have at least one  good backup in case the new backup fails. If you would like to modify  the script, you're more than welcome to do so.

7Q: What is the best storage configuration for datastore type X?

7A: Search the VMTN forums; there are various configurations for the different type of storage/etc.

8Q: I want to setup NFS Server to run my backups. Which is the best and should it be virtual or physical?7

8A: Please refer to answer 7A. From experience, we’ve seen  physical instances of NFS servers to be faster than their virtual  counterparts.  As always, YMMV.

9Q: When I try to use the script I get an error stating: "restricted version". What the heck is this?

9A: Please take a look at the ESX/ESXi Version Support Table in this document. The reason is explained here.

10Q: I'm getting error X when using the script or I'm not getting  any errors, the backup didn’t even take place. Oh what could be wrong?

10A: First, you'll want to check the log (default  /tmp/ghettoVCBg2) and see if there are any obvious errors. If not,  please provide the entire output of the log, a short description of your  environment and the type of VM(s) you're trying to backup—I will try to  provide some assistance.

11Q: I have VMs that have snapshots. I want to back these things up but the script doesn’t let me do it. How do I fix that?

11A: VM snapshots are not meant to be kept for long durations.  When backing up a VM that contains a snapshot, you should ensure all  snapshots have been committed prior to running a backup. No exceptions  will be made…ever.

12Q: I would like to restore from backup, what is the best method?

12A: The restore process will be unique for each environment and  should be determined by your backup/recovery plans. At a high level you  have the option of mounting the backup datastore and registering the VM  in question or copy the VM from the backup datastore to the ESX/ESXi  host. The latter is recommended so that you're not running a VM living  on the backup datastore or inadvertently modifying your backup VM(s).

13Q: When using the experimental feature: Individual VM backup policy, if I don't have a policy will it use the default configurations defined in the script?

13A: No, this use case will not be supported, ever. If you decide  to use the per VM policy, then the global configuration in the script  is ignored. If a VM policy config is not found, the VM will not be  backed up even if it's in the –-vmlist.

14Q: In vMA 4.0, vCenter target will be supported, can I just add vCenter target and assume this will work?

14A: At the moment, this is not a supported feature. You will  need to add each ESX/ESXi host for the vi-fastpass library to be able to  locate the host(s).

15Q: When I try to run the script I get: "-bash: ./ghettoVCBg2.pl: Permission denied", what is wrong?

15A: You need to change the permission on the script to be executable, chmod +x ghettoVCBg2.pl

16Q: Where can I download the latest version of the script?

16A: The latest version is available on this page at the bottom. To get the version of the script run

./ghettoVCBg2.pl --version

17Q: I'm running into an issue when I backup my VMs using NFS on  Linux, I'm able to either get 1 successful backup and all subsequent  backups fail OR I'm having issues where snapshots are not being removed  upon completition and future backups fail.

17A: This problem/issue has been reported by quite a few users in  the community and only affects a very small percentage of users overall  under a specific configuration. The problem as I understand it today  which is still an on-going investigation by myself and the community is  as follows:

The Setup

Setup: NFS on Linux where the underlying volume is EXT3 filesystem

and backups are failing primarily around larger sized VMs ( > 150GB).

The Observed Problem

The issue is during the final phase of a VM being backed-up in the  rotation function where a request is sent out to the NFS server to  either A) rotate/rename a directory B) delete an old backup. There is  either a disconnect or delay that severely affects the backup process  that may lead to a whole slew of "unexpected" behavior from having only 1  backup complete successful and all subsquent backups failing/etc.

First off, there is nothing wrong with the script from what we've found  it and has to do with NFS server and specifically the volume that the  backup datastore is hosted on. Thanks to community user

ralfoertner who had a simliar problem using

ghettoVCBg2 using Debian with EXT3 base filesystem for his NFS volume. He decided  to do some additional troubleshooting to further isolate the problem and  actually found by changing the underlying filesystem from EXT3 to XFS  that the issue went away and resolve all issues he had ran into  regarding an NFS on Linux backup solution.

The Theory

Now, we're still 100% sure why the above solution has resolved the  problem or what the actual problem is but I'll explain the current  theory:

EXT3 is a journaled filesystem, you can do a search online for more  information. There's a few advantages with EXT3 from it's predcessor  EXT2, but one difference is in the approach it takes with journaling as  explained here in greater detail:

http://www.ibm.com/developerworks/library/l-fs7.html

Basically EXT2 does only "metadata journling" whereas EXT3 does both  data and metadata journaling, this would ensure that all changes to the  filesystem where it's data or metadata is consistent in the event of an  unexpected system crash. With EXT3, there are 3 journaling options: (

data=writeback mode,data=ordered mode,data=journal mode ) and by default it uses

data=ordered mode to ensure the consistency as described from above.

One easy method to check what mount options are configured, on a Linux system you can check /proc/mounts:

[vi-admin@scofield ~]$ cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw,data=ordered 0 0
/dev /dev tmpfs rw 0 0
/proc /proc proc rw 0 0
/sys /sys sysfs rw 0 0
/proc/bus/usb /proc/bus/usb usbfs rw 0 0
devpts /dev/pts devpts rw 0 0
/dev/VolGroup00/var /var/log ext3 rw,data=ordered 0 0
/dev/sda1 /boot ext3 rw,data=ordered 0 0
tmpfs /dev/shm tmpfs rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0

So now what does this have to do with XFS and EXT3? With the defaults of  EXT3, it may not always give you the best performance due to the extra  overhead of the way it does its journaling, if you were to change the  options to

data=writeback mode, you'll notice that this is simliar journaling method found in

XFS,JFS and ResierFS as described here:

http://www.ibm.com/developerworks/library/l-fs8.html

The Conclusion

I believe that there are some additional overheads that maybe caused by  using NFS on EXT3, which may explain the issue that users are seeing and  that moving from EXT3 to XFS has "seemed" to solve the issues for those  who have implemented the change. Though, before you go out and make  this change, you need to understand your environment and what this  change would exactly mean, please refer to differences in EXT2/XFS and  EXT3, especially around a power interruption or system crash.

In conclusion, I'm not recommending one way or the other, it really  depends on your environment. Though this has shown to resolve the issue  as users have implemented to get around this problem, but it's  definitely worth while to really understand the problem in your  environment and figure out why and decide on the best solution. I would  also recommend users perform their own tests to further isolate the  problem in their environment and try local storage, using iSCSI to see  if they can pin-point the problem.


Useful Links:

If you have found this script or other scripts to be useful and would like to contribute back, please click

here to donate! Thank You


Change Log

07/20/10 - build 4.3

Enhancements:

  • Support for vSphere 4.1 (ESX and ESXi)

Bug Fixes:

  • Due to changes in the VIFP Library introduced with vMA 4.1, the  script has been updated to work on both vMA 4.0 and vMA 4.1. Only  upgrade to vMA 4.1 if you're managing ESX(i) 4.1 hosts


03/27/10 - build 4.2

Enhancements:

  • Added a check to see if ESX(i) host is in a connected state
  • Added a check to see if ESXi host is using the free license, if so, skip host
  • Added FAQ #0

03/27/10 - build 4.1

Enhancements:

  • Updated FAQ #17 regarding "NFS" issue


02/24/09 - build 4.1

Bug Fixes:

  • There was a typo in handling the case of a VM's VMDK(s) being  distributed across multiple datastores in which the backup was not  creating the directory to reflect this layout, thanks to briandatamatters for finding this bug.

05/26/09 - build 2.5.1

Enhancements:

  • N/A

Bug Fixes:

  • Implemented shorterm solution fix for bug in VMware vCLI's vifs
  • Updated logic to allow virtual RDM to be backed up


04/25/09 - build 2.5

Enhancements:

  • Added experimental feature Individual VM backup policy
  • Added VM snapshot memory & quiesce options

Bug Fixes:

  • N/A


02/21/09 - build 2.0

Initial release

Attachments
Comments
yp1

There is something weird about running the script on vSphere.

I have 2 environments with Enterprise license, and it works on one,but not on another.

The only difference is the first one (that works) utilizes NFS datastores, the second one is on SAN.

There is no errors in the logs. It starts backing up a VMDK, then quits in exactly 1 minute (for each VMDK cycle).

Could it be because of the blank spaces in the names of the VM folder and/or VMDKs?

Any ideas?

Here is an excerpt from a typical run attempt (the VM and LUN names are masked):

02-13-2010 07:01:33 -- debug: backupVMDK: Successfully completed backup for xxxxxx - Satellite prod_1/xxxxxxxxx - Satellite prod.vmdk Elapsed time: 1 min

02-13-2010 07:01:33 -- debug: backupVMDK: Backing up "[xxxxxx] xxxxxx - Satellite prod_1/xxxxxxx - Satellite prod_1.vmdk" to "[xxxxxx] BACKUPS/xxxxxxx/xxxxxxxx-2010-02-13/xxxxxxxx - Satellite prod_1.vmdk"

02-13-2010 07:01:33 -- debug: backupVMDK: Signal copyThread to start

02-13-2010 07:01:33 -- debug: backupVMDK: Backup progress: Elapsed time 0 min

02-13-2010 07:01:33 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy

02-13-2010 07:01:33 -- debug: CopyThread: Start backing up VMDK(s) ...

02-13-2010 07:01:33 -- debug: copyTask: send copySuccess message ...

02-13-2010 07:01:33 -- debug: copyTask: waiting for next job and sleep ...

02-13-2010 07:02:34 -- debug: backupVMDK: Successfully completed backup for xxxxx - Satellite prod_1/xxxxx - Satellite prod_1.vmdk Elapsed time: 1 min

02-13-2010 07:02:34 -- debug: backupVMDK: Backing up "[xxxxxx] xxxxxxx - Satellite prod_1/xxxxxx - Satellite prod_2.vmdk" to "[hddprd0c_wms_1] BACKUPS/xxxxxxx/xxxxxxx-2010-02-13/xxxxxx - Satellite prod_2.vmdk"

02-13-2010 07:02:34 -- debug: backupVMDK: Signal copyThread to start

02-13-2010 07:02:34 -- debug: backupVMDK: Backup progress: Elapsed time 0 min

02-13-2010 07:02:34 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy

02-13-2010 07:02:34 -- debug: CopyThread: Start backing up VMDK(s) ...

02-13-2010 07:02:34 -- debug: copyTask: send copySuccess message ...

02-13-2010 07:02:34 -- debug: copyTask: waiting for next job and sleep ...

02-13-2010 07:03:34 -- debug: backupVMDK: Successfully completed backup for xxxxxxx - Satellite prod_1/xxxxxx - Satellite prod_2.vmdk Elapsed time: 1 min

Hi,

I'm also trying to backup a VM (called "glossary") to a datastore (called VM-Backups) using the ghettoVCBg2.pl version 4.1 script. The script seems to run fine, but no vmdk's appear in the datastore.

The VM-Backups datastore is an nfs mounted filesystem from a freenas box. The nfs filesystem type is ufs. The VMWare box is ESX4.0i

Logfile gives:

03-21-2010 16:34:58 -- info: ============================== ghettoVCBg2 LOG START ==============================

03-21-2010 16:34:58 -- debug: Main: Login by vi-fastpass to: vmware.reusens.com

03-21-2010 16:34:58 -- debug: copyTask: Task START

03-21-2010 16:34:58 -- debug: copyTask: waiting for next job and sleep ...

03-21-2010 16:35:01 -- debug: reConfigureBackupParams: VM - glossary

03-21-2010 16:35:01 -- info: CONFIG - USING CONFIGURATION FILE = glossary

03-21-2010 16:35:01 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log

03-21-2010 16:35:01 -- info: CONFIG - VM_BACKUP_DATASTORE = VM-Backups

03-21-2010 16:35:01 -- info: CONFIG - VM_BACKUP_DIRECTORY = VMWARE_BACKUPS

03-21-2010 16:35:01 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick

03-21-2010 16:35:01 -- info: CONFIG - ADAPTER_FORMAT = buslogic

03-21-2010 16:35:01 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = YES

03-21-2010 16:35:01 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO

03-21-2010 16:35:01 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO

03-21-2010 16:35:01 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2010-03-21

03-21-2010 16:35:01 -- info: CONFIG - VM_VMDK_FILES = all

03-21-2010 16:35:01 -- info: Initiate backup for glossary found on vmware.reusens.com

03-21-2010 16:35:06 -- debug: glossary original powerState: poweredOff

03-21-2010 16:35:06 -- info: glossary has 1 VMDK(s)

03-21-2010 16:35:06 -- debug: backupVMDK: Backing up "[datastore2] turnkey-lamp/turnkey-lamp.vmdk" to "[VM-Backups] VMWARE_BACKUPS/glossary/glo

ssary-2010-03-21/turnkey-lamp.vmdk"

03-21-2010 16:35:06 -- debug: backupVMDK: Signal copyThread to start

03-21-2010 16:35:06 -- debug: backupVMDK: Backup progress: Elapsed time 0 min

03-21-2010 16:35:06 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy

03-21-2010 16:35:06 -- debug: CopyThread: Start backing up VMDK(s) ...

03-21-2010 16:35:09 -- debug: copyTask: send copySuccess message ...

03-21-2010 16:35:09 -- debug: copyTask: waiting for next job and sleep ...

03-21-2010 16:36:06 -- debug: backupVMDK: Successfully completed backup for turnkey-lamp/turnkey-lamp.vmdk Elapsed time: 1 min

03-21-2010 16:36:08 -- debug: checkVMBackupRotation: Starting ...

03-21-2010 16:36:08 -- info: Backup completed for glossary!

03-21-2010 16:36:08 -- debug: Main: Disconnect from: vmware.reusens.com

03-21-2010 16:36:08 -- debug: Main: Calling final clean up

03-21-2010 16:36:08 -- debug: cleanUP: Thread clean up starting ...

03-21-2010 16:36:08 -- debug: cleanUp: Send exit to copyThread

03-21-2010 16:36:08 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit

03-21-2010 16:36:08 -- debug: copyTask: die ...

03-21-2010 16:36:08 -- debug: cleanUp: Join passed

03-21-2010 16:36:08 -- info: ============================== ghettoVCBg2 LOG END ==============================

yp1

That is likely due to the license. I assume you run a free license (which provides read-only API).

In my case I run it with Enterprise license...

I have a "Product: ESXi 4 Single Server Licensed for 1 physical CPUs (1-6 cores per CPU)" which was free".

Needless to say it's a big disapointment that after spending hours and hours to get the whole ESX4 story to work, a showstopper like this occurs.

I just wanted to do backups on an external USB drive, but ESX4i does not support it, so I installed a separate box with freenas which allows USB drives, and now this happens.

Question 9Q should be put first with "READ THIS FIRST" so that people don't waste their time.

yp1

http://communities.vmware.com/docs/DOC-8760

This one does not require API...

OK, I'll give that a try...

Sorry to hear that you had to go through all that, though I do expect users to read all documentation before starting. Yes FAQ 9 probably should be further up on the list, BUT the license table is at the very top of the document. The FAQ list was really created for the most commonly asked questions/comments that I've seen, hence the licensing question generally does not come up as often.

As mentioned, you can take a look at the 100% free scripted solution which requires the use of the Service Console from the classic ESX OR unsupported Busybox console for ESXi as an alternative.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

We have been using this script for a while and it appears to be having an issue with a few of our VM's. We have 3 fully licensed ESX servers and use iSCSI storage. Our backup target is an Openfiler NAS. Initially the script will run fine but after a few runs it has trouble locating some of our VM's. here are the error messages :

03-21-2010 01:06:45 -- error: getFinalList: ERROR - Unable to locate VM: node_3

03-21-2010 01:06:45 -- error: getFinalList: ERROR - Unable to locate VM: node_1

03-21-2010 01:06:45 -- error: getFinalList: ERROR - Unable to locate VM: node_2

nothing has changed in the location of the vm's or anything since the last successful run here:

03-20-2010 01:33:17 -- info: Initiate backup for node_3 found on ZZZZ.ZZZZ.com

03-20-2010 01:33:31 -- debug: node_3 original powerState: poweredOn

03-20-2010 01:33:31 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2010-03-20" for node_3

03-20-2010 01:33:41 -- info: node_3 has 1 VMDK(s)

03-20-2010 01:33:41 -- debug: backupVMDK: Backing up "[scsi1] node_3/node_3.vmdk" to "[vmbackup] backups/node_3/node_3-2010-03-20/node_3.vmdk"

03-20-2010 01:33:41 -- debug: backupVMDK: Signal copyThread to start

03-20-2010 01:33:41 -- debug: backupVMDK: Backup progress: Elapsed time 0 min

03-20-2010 01:33:41 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy

03-20-2010 01:33:41 -- debug: CopyThread: Start backing up VMDK(s) ...

03-20-2010 01:35:58 -- debug: copyTask: send copySuccess message ...

03-20-2010 01:35:58 -- debug: copyTask: waiting for next job and sleep ...

03-20-2010 01:36:42 -- debug: backupVMDK: Successfully completed backup for node_3/node_3.vmdk Elapsed time: 3 min

03-20-2010 01:36:45 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2010-03-20" for node_3

03-20-2010 01:36:48 -- debug: checkVMBackupRotation: Starting ...

03-20-2010 01:36:56 -- debug: Purging backups/node_3/node_3-2010-03-19--1 due to rotation max

03-20-2010 01:36:56 -- info: Backup completed for node_3!

any thoughts? if you need any mroe info just ask, we monitor this thread actively.

~ Brian

You're saying that these VMs has always worked with regards to the backup going to this NAS openfiler? How large are each of these VMs? Can you consistently re-produce the issue, have you tried to backup individual VMs and see if you still run into the same problem.?

I would also recommend taking a look at the ESXi logs to see if there's any issues during the period of the backup also looking at the logs of the NAS server if you're saturating the link OR if there's other issues going on.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

yp1

Hi William,

Is there a problem with backing up large (200GB or so) VMDKs?

The problem I described in on of my last comments where backup starts and ends after 1 min without producing any errors happens on a VM with 2x200GB VMDKs. Thoug the 3rd one is only 25GB.

I am just trying to understand why would it fail...

There shouldn't be any problems but I've not tested a VM backup with that capacity. I would also take a look at your logs on the host to see if there's any correlation. Also what is your destination storage type? local? iSCSI/FC SAN? NFS?

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

William,

For the past few weeks I've struggled with random VMs in my backup list not being "found". I'm backing up to an NFS export that resides on a VM within one of my ESXi hosts. I switched to XFS yesterday and for the first time every VM backed up with no issue. Any thoughts as to why the switch from EXT3 to XFS would produce such a result?

Thanks,

Ian

So it was actually a thanks to user ralfoertner in this forum that found a solution from going to ext3 to XFS filesystem when using NFS, this information so far seems to have resolved the problem and I've actually brought this over to the ghettoVCB forum per this post:

http://communities.vmware.com/docs/DOC-8760#comments-14679

There are inherently differences between ext3 and XFS as mentioned in few replies in ghettoVCB forum due to the way updates are done when a request goes out to the filesystem. Again, this is the only thing I've concluded, again there are others that are using NFS perfectly fine, so hard to say if it's the hardware or software. It's definitely not the script. There's also been another correlation that I've been able to draw from the various reports and that is "larger" VMs > 100-150GB may also experience this problem and VMs that are < 100 may not. Though that isn't always the case but it's another interesting fact that I've found.

I plan on doing a write up, though I've only had a small amount of feedback from this fix, so it would be nice to get more feedback or if users notice other things.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

The script is working fine for me (thanks for that!) except one thing: the VM_BACKUP_ROTATION_COUNT

If I give it a value from 1-9 it's working okay. But when I give it a number starting from 10 or higher it will not work.

Latest test:

5 backup folders

run script with value VM_BACKUP_ROTATION_COUNT="10"

2 folders left

Any idea?

Thanks, Gerald

Do you take more than one backup per day? Or is it one backup per day and after certain number, you want to start rotating?

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

Normally I do 1 backup per day. For testing I set it to 9 and it creates all the backups on the same day. When I change it to 20 it removes backups as I sad before.

Gerald

Hello William,

i am having this error when i am trying to backup a Vm in a Esxi 4.0 licensed server. I have checked all post and the error seems to reference a unlicensed server, this is not my case, i have bought this server with a Esxi single server license and i have tested this in Configuration --> Lincensed features: Product: ESXi 4 Single Server Licensed for 2 physical CPUs (1-6 cores per CPU)

License Key: My number license Key

Expires: Never

This is the error log:

03-31-2010 12:24:42 -- debug: Main: Login by vi-fastpass to: 134.134.20.4

03-31-2010 12:24:44 -- debug: reConfigureBackupParams: VM - viosr

03-31-2010 12:24:44 -- info: CONFIG - USING CONFIGURATION FILE = viosr

03-31-2010 12:24:44 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log

03-31-2010 12:24:44 -- info: CONFIG - VM_BACKUP_DATASTORE = Backup

03-31-2010 12:24:44 -- info: CONFIG - VM_BACKUP_DIRECTORY = vmware

03-31-2010 12:24:44 -- info: CONFIG - DISK_BACKUP_FORMAT = thin

03-31-2010 12:24:44 -- info: CONFIG - ADAPTER_FORMAT = lsilogic

03-31-2010 12:24:44 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO

03-31-2010 12:24:44 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO

03-31-2010 12:24:44 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO

03-31-2010 12:24:44 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2010-03-31

03-31-2010 12:24:44 -- info: CONFIG - VM_VMDK_FILES = all

03-31-2010 12:24:44 -- info: Initiate backup for viosr found on 134.134.20.4

03-31-2010 12:24:55 -- debug: Sntenviosr original powerState: poweredOn

03-31-2010 12:24:55 -- info: viosr has 1 VMDK(s)

03-31-2010 12:24:55 -- debug: backupVMDK: Backing up "[datastore3] viosr/viosr.vmdk" to "[Backup] vmware/viosr/viosr-2010-03-31/viosr.vmdk"

03-31-2010 12:24:55 -- debug: backupVMDK: Signal copyThread to start

03-31-2010 12:24:55 -- debug: backupVMDK: Backup progress: Elapsed time 0 min

03-31-2010 12:24:55 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy

03-31-2010 12:24:55 -- debug: CopyThread: Start backing up VMDK(s) ...

03-31-2010 12:25:00 -- debug: copyTask: send copySuccess message ...

03-31-2010 12:25:00 -- debug: copyTask: waiting for next job and sleep ...

03-31-2010 12:25:55 -- debug: backupVMDK: Successfully completed backup for viosr/viosr.vmdk Elapsed time: 1 min

03-31-2010 12:25:55 -- warn: WARNING: Snapshot "ghettoVCBg2-snapshot-2010-03-31" not found

03-31-2010 12:25:57 -- debug: checkVMBackupRotation: Starting ...

03-31-2010 12:25:57 -- debug: Unable to purge vmware/viosr/GET https:134.134.20.4foldervmwareviosr?dsName=Backup unsuccessful : 404 Not Found due to rotation max

03-31-2010 12:25:57 -- info: Backup completed for viosr!

03-31-2010 12:25:57 -- debug: Main: Disconnect from: 134.134.20.4

03-31-2010 12:25:57 -- debug: Main: Calling final clean up

03-31-2010 12:25:57 -- debug: cleanUP: Thread clean up starting ...

03-31-2010 12:25:57 -- debug: cleanUp: Send exit to copyThread

03-31-2010 12:25:57 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit

03-31-2010 12:25:57 -- debug: copyTask: die ...

03-31-2010 12:25:58 -- debug: cleanUp: Join passed

03-31-2010 12:25:58 -- info: ============================== ghettoVCBg2 LOG END =====

Any idea??, is this license enough to run the script? in the comaptible versions table seems it is valid..

Thanks

Let me do some more testing, there might be an issue when the backups are all taken on the same day and it's the way it tries to do a comparison when it hits double digits like "10" and compares that with "1", it's not numeric comparison as you would expect, so you would get 1,10,2,20,3,4,etc.

I believe it should work if you have just 1 backup per day and it'll use the date to do the comparison and get the right order for the rename.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

Could you please try to run this script: http://pastebin.com/NWKDFh89 on your vMA host and specify the host in question:

If it's licensed properly, you should not see any output else you may see something like this:

[vi-admin@scofield ~]$ ./checkHostVersion.pl --server esxi4-1.primp-industries.com --username root
This script requires your ESXi be licensed, the free version will not allow you to perform any write operations!

Thanks

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

Thanks, William. It works great. I am using it to replaced Xtravirt's vi-backup script since it doesn't support vSphere. While using it, one feature that I would love to have is the ability to backup a particular vmdk file, or better yet to backup different vmdk files to different location. For example, a Windows VM has two drives, and only D drive is required to be backed up.

Great work, thanks again.

Actually this is a feature that already exists in the script, it looks like there was a small hiccup in the documentation and I sent an older revision in.

It's been updated now to reflect this.

Thanks

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

Thanks, will try it out.

Hi William,

meanwhile I got the right licences and tried again.

The backup works but is extremely slow:

1) for a vm that was swithed off: OK, fast enough

2) for a 50 Gb Windows2003:

backupVMDK: Successfully completed backup for windows2003/windows2003.vmdk Elapsed time: 192 min !

3) for a 200Gb Centos:

backupVMDK: Successfully completed backup for centos/centos_1.vmdk Elapsed time: 1140 min !!

What could be the cause of this?

Thanks in advance,

Johnny

Please take a look at FAQ #5 and also make sure you give the documentation a full read including the FAQs.

Thanks

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

Thanks William,

even my license was correct, it was not registered yet at vmware.com and the original license key did not allow to run the script. After registration and proper introduction of the new license key the scipt works properly.

Thanks again

Jesús

It works great, exactly like what you documented.

I did run into an error that stopped the backup process though. I suspected one ESX had hardware issues, and I put it into maintenance mode, then rebooted it. It hung at a pink screen after reboot. This script failed with a message: "Error connecting to server at 'https://esx21:433/sdk': Perhaps host is not a Virtual Center or ESX server. A thread exited while 2 threads were running." I just thought that if it could catch this error, skip this ESX and continue the search on the rest of ESX, it would require less human intervention, and this script would do its best to backup most of the VM in the list.

Thank you!

Hi William,

In my backup directory I only see the following files:

vm.vmdk

vm.vmx

vm-flat.vmdk

Where are the nvram, vmsd, vmxf and log files?

Thanks,

Gerald

That is the expected behavior, those are the minimal files that are required for backup. If you need the remainder files for whatever reason, you're more than welcome to edit the script to include them.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

William,

Great script! I've been testing it to use in our environment until we can implement a more cross-platform system and I had one question I wanted to bring up to you. We are getting similar error messages to Byron_Zhao, but not because we are having hardware problems.

Our issue is that when DPM has one of the servers powered down when the script is run it is causing the script to error out. Is there a way to avoid having it fail just because one of the ESX servers is unavailable (purposely shutdown, to conserve power)?

Thank you for your great script!

Josh

I've updated the script with your suggestion, it will also check the licensing type on ESXi host as well.

Let me know if this solves your DPM problem.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

William,

I am still getting the same behavior. The last line of the log is:

04-14-2010 07:11:44 -- debug: Main: Login by vi-fastpass to: fmdns-249-33-182.navsisa.navy.mil

And I get this message on the screen where it is running the script:

Error connecting to server at 'https://fmdns-249-33-182.navsisa.navy.mil:443/sdk ': Perhaps host is not a Virtual Center or ESX server

A thread exited while 2 threads were running.

The listed server is the server that is currently in power save mode. If I can provide you with any other information, let me know.

Thanks for the script and for the update.

Josh

Unfortunately I don't have a host I can test DPM with. I'm guessing when a host is in DPM, you can't even communicate to it which is probably why it's giving the connection failed.

Could you point your browser to the following and see if something returns:

https://fmdns-249-33-182.navsisa.navy.mil/mob

What it sounds like is I need to put some validation around when it tries to login through vi-fastpass and if it fails, then just ignore this host since we can't reach it. I'll try to add this update sometime this evening and get that out

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

Actually I just updated the script with the fix, it was only few lines. Give the new version a try and let me know

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

That seems to have fixed it. It ran to completion and I got the log file email successfully.

Great work! Thanks!

Josh

Great to hear!

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

Hi William,

took a good look at the FAQ and related documentation. So I implemented the "best practice" with extra NIC's so that my Freenas NFS-export has its own separate ip address in a different subnet, own NIC. On the ESX4i box I added as wel a NIC and connected this via a "straight cable". However the result is the same: extremely slow. A backup of 200G does not finish with a whole night.

Then I search further and came across the thread

"Poor ESXi 4 NFS Datastore Performance with Various NAS Systems" (http://communities.vmware.com/thread/263165).

So It looks there is an NFS ussue with ESX4i.. Maybe it might be a good idea to flag this in the FAQ..

Greetings,

Johnny

I William,

already in the past i have helped.

My Version Product;

vSphere Management Assistant

Version: 4.0.0 ( 4.0.0-161993)

$ ./ghettoVCBg2.pl --version

VI Perl Toolkit version: 4.0

Script 'ghettoVCBg2.pl' version: 4.2

VMware ESX 4.0.0 build-208167

I install ghettoVCBg2 on my

vSphere Management Assistant

Version: 4.0.0 ( 4.0.0-161993)

When i run this script;

vi-admin@itrmvma ~$ ./ghettoVCBg2.pl --config_dir backup_config --vmlist virtual_machine_backup_list

Where in dir backup_config there's this file;

vi-admin@itrmvma backup_config$ more IT20001288

VM_BACKUP_DATASTORE = "LS_VMWT2"

VM_BACKUP_DIRECTORY = "IT20001288"

VM_BACKUP_ROTATION_COUNT = "3"

DISK_BACKUP_FORMAT = "zeroedthick"

ADAPTER_FORMAT = "lsilogic"

POWER_VM_DOWN_BEFORE_BACKUP = "0"

VM_SNAPSHOT_MEMORY = "1"

VM_SNAPSHOT_QUIESCE = "1"

LOG_LEVEL = "debug"

VM_VMDK_FILES = "all"

and on virtual_machine_backup_list there's name my VM.

The script finished with Successfully, but i see on datastore of backup only file vmx.

This is my log;

05-26-2010 16:38:00 -- info: ============================== ghettoVCBg2 LOG START ==============================

05-26-2010 16:38:09 -- info: CONFIG - USING CONFIGURATION FILE = IT20001288

05-26-2010 16:38:09 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log

05-26-2010 16:38:09 -- info: CONFIG - VM_BACKUP_DATASTORE = LS_VMWT2

05-26-2010 16:38:09 -- info: CONFIG - VM_BACKUP_DIRECTORY = IT20001288

05-26-2010 16:38:09 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick

05-26-2010 16:38:09 -- info: CONFIG - ADAPTER_FORMAT = lsilogic

05-26-2010 16:38:09 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO

05-26-2010 16:38:09 -- info: CONFIG - VM_SNAPSHOT_MEMORY = YES

05-26-2010 16:38:09 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = YES

05-26-2010 16:38:09 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2010-05-26

05-26-2010 16:38:09 -- info: CONFIG - VM_VMDK_FILES = all

05-26-2010 16:38:09 -- info: Initiate backup for IT20001288 found on 172.23.42.73

05-26-2010 16:38:21 -- debug: IT20001288 original powerState: poweredOn

05-26-2010 16:38:21 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2010-05-26" for IT20001288

05-26-2010 16:39:21 -- info: IT20001288 has 1 VMDK(s)

05-26-2010 16:39:21 -- debug: backupVMDK: Backing up "TESTVS4 IT20001288/IT20001288.vmdk" to "LS_VMWT2 IT20001288/IT20001288/IT20001288-2010-05-26/IT20001288.vmdk"

05-26-2010 16:39:21 -- debug: backupVMDK: Signal copyThread to start

05-26-2010 16:39:21 -- debug: backupVMDK: Backup progress: Elapsed time 0 min

05-26-2010 16:39:21 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy

05-26-2010 16:39:21 -- debug: CopyThread: Start backing up VMDK(s) ...

05-26-2010 16:39:29 -- debug: copyTask: send copySuccess message ...

05-26-2010 16:39:29 -- debug: copyTask: waiting for next job and sleep ...

05-26-2010 16:40:22 -- debug: backupVMDK: Successfully completed backup for TESTVS4 IT20001288/IT20001288.vmdk Elapsed time: 1 min

05-26-2010 16:40:24 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2010-05-26" for IT20001288

05-26-2010 16:40:27 -- debug: checkVMBackupRotation: Starting ...

05-26-2010 16:40:31 -- info: Backup completed for IT20001288!

05-26-2010 16:40:31 -- debug: Main: Disconnect from: 172.23.42.73

05-26-2010 16:40:31 -- debug: Main: Calling final clean up

05-26-2010 16:40:31 -- debug: cleanUP: Thread clean up starting ...

05-26-2010 16:40:31 -- debug: cleanUp: Send exit to copyThread

05-26-2010 16:40:31 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit

05-26-2010 16:40:31 -- debug: copyTask: die ...

05-26-2010 16:40:31 -- debug: cleanUp: Join passed

05-26-2010 16:40:31 -- info: ============================== ghettoVCBg2 LOG END ==============================

Can you help please?

by

ii00907

I noticed from your debug output, that it does not contain some of the new debugging information such as verifying a VMDK prior to backup. Can you confirm that you are using the latest version of this script? If not, plesae download and try again with the debug flag. Could also paste the output to pastebin.com and just provide the link, it makes it easier to read.

Thanks

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

William,

i have download ghettoVCBg2 today.

I have run ;

./ghettoVCBg2.pl --version

And this is output

$ ./ghettoVCBg2.pl --version

VI Perl Toolkit version: 4.0

Script 'ghettoVCBg2.pl' version: 4.2

A thread exited while 2 threads were running.

Have you tried to re-run the script?

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

I Williamo,

i test script to another host, and is ok.

Imagines that my esx development has a problem.

Last question;

You can specify the differences between;

VM_SNAPSHOT_MEMORY = 0 or 1

VM_SNAPSHOT_QUIESCE = 0 or 1

What do you suggest?

Very thanks

VM_SNAPSHOT_MEMORY is really for my testing purposes, there's no real value in setting this to 1, so I would leave it as a default 0.

I would definitely recommend setting VM_SNAPSHOT_QUIESCE to 1 as it'll ensure the guest is quieted down prior to snapshot to ensure that all I/O has been flushed to disk. To ensure this is as optimal as possible, make sure you have VMware Tools installed on your guest

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Twitter: @lamw

vGhetto Script Repository

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

William,

thanks for your help.

By

So where is everyone downloading ghettoVCBg2.tar.gz from?

At the bottom of this document Smiley Happy

Search for the word "attachment" if you still can't find it.

=========================================================================

William Lam

VMware vExpert 2009

VMware scripts and resources at:

Twitter: @lamw

vGhetto Script Repository

Getting Started with the vMA (tips/tricks)

Getting Started with the vSphere SDK for Perl

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

We am running ghettoVCB2 through vMA 4.0 managing ESX 3.5u4. The NFS share connected to the ESX chassis is hosted on a Windows 2003 Storage Server. When the ghettoVCB2 script runs it is able to backup the virtual machines without a problem, however overtime (could be the 2nd backup or later) file permissions on the Windows Servers become corrupted somehow; causing the script not to be able to enforce the retention (rotation). This also plays into an issue where our Administrative accounts are unable to manage the files (i.e.: delete, modify, or copy the files via Backup Exec to a tape).

We are able to change the permissions (security and owner) on the folder on the 2003 Storage Server to gain access and control however the next time the script runs the permissions break. The permissions that become inherited on the folder are:

ANONYMOUS LOGON, NULL SID, Everyone

This also happens if we open the data store and create a new folder or upload a file via vSphere client; however we are able to delete, modify, and copy the files no problem. The corruption only appears to happen at some point when ghettoVCB2 runs. Also we have tried ghettoVCB and have the same issues on the NFS Share.

I am at the point where I am going to create an NFS share on a Linux system to (hopefully) correct the permission issue; however wanted to know if anyone else has experienced this problem and if so if they had a resolution.

Thanks for the time.

I have the problem you mention with EXT3 and NFS datastores.

In my case I successfully solved it exporting the NFS with ASYNC option:

/mnt/sig/nas/vmachines esx-host(async,no_subtree_check,no_root_squash,rw)

Also the backup time for 6 vmachines go from 2 hours to 40 minutes.

I don't know wich is the best option, use EXT3 with ASYNC or use XFS?.

Please, any suggestions will be welcome.

I was doing some research about de nfs/ext3 issue. Supose we need to backup several VM.

In my opinion there are 2 options:

1) Use ext3 or ext4 with data=writeback, or XFS.

Here the problem could be:

if server crash when or after doing a VM backup, all VM backups recently done, could be corrupted because we don't have data journaling in the file system and data could be in cache and the moment of the crash.

2) Use ext3 or preferably ext4 and export the NFS in async mode.

Here the problem could be:

if server crash when or after doing a vm backup, only the VM that was being backed up at that moment could be incomplete if data was not flushed to disk at the moment of the crash. In the case of the NFS server crash after signaling ESX server that data was write, guettoVCB will log that the VM was completely backed up, but this may be wrong.

In my case, and only for the purpose of backing up VM, I prefer option 2, because in the worst case I have only one VM backup corrupted, in case 1 I could have all the list of VM backups corrupted.

I'm wrong about possible problems?

Feedback will be welcome.

Some links:

http://lwn.net/Articles/329008/

http://nfs.sourceforge.net/

Sorry for the late reply, I've been studying for an exam these last few weeks and just been swamped with other things.

From the replies I've gotten back, XFS resolved the issue and was preferred over the issues that came with using ext3. I'm not aware of any additional performance penalties that maybe incurred from switching, YMMV

=========================================================================

William Lam

VMware vExpert 2009,2010

VMware scripts and resources at:

Twitter: @lamw

vGhetto Script Repository

Getting Started with the vMA (tips/tricks)

Getting Started with the vSphere SDK for Perl

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

Sorry for the late reply, I've been studying for an exam these last few weeks and just been swamped with other things.

I concur with findings, there are some downsides with XFS and that is also mentioned in the documentation if I'm not mistaken. I know from users verifying the switch, that XFS does in fact resolve the problem with the EXT3/NFS issue but definitely there are some down sides. It will be up to the individual users to choose that best solution that fits their environment and understand the risks that are involved.

=========================================================================

William Lam

VMware vExpert 2009,2010

VMware scripts and resources at:

Twitter: @lamw

vGhetto Script Repository

Getting Started with the vMA (tips/tricks)

Getting Started with the vSphere SDK for Perl

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

Version history
Revision #:
1 of 1
Last update:
‎04-22-2009 09:36 PM
Updated by: