Up to Documents in vSphere Management Assistant (vMA)

ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

VERSION 9 Published

Created on: Apr 22, 2009 9:36 PM by lamw - Last Modified: Nov 18, 2009 7:47 PM by lamw

Description:

This tool is the follow up release of the ghettoVCB backup utility which allows users to perform backups of virtual machines residing on ESX(i) 3.5+/4.0+ servers using methodology similar to VMware's VCB tool. By incorporating highly constructive feedback from the VMware community and utilizing the existing VI API, ghettoVCB’s framework was completely rewritten to be harder, better, faster, stronger.

The primary motivation for ghettoVCBg2 was to provide ESXi users with access to the utility without relying on unlocking and utilizing the unsupported console. To satisfy this requirement, the rebuilt framework takes advantage of the VI Perl Toolkit/vSphere SDK for Perl which is present in the VIMA/vMA virtual appliance provided by VMware. As a result, ghettoVCBg2 provides a more proper backup solution that administrators can utilize in their virtual infrastructure.

As it stands, it is a requirement that ghettoVCB be invoked directly on each of the ESX servers hosting virtual machines in need of a backup. By taking advantage of VIMA/vMA, the entire backup process becomes centralized in turn eliminating the abovementioned requirement. Additionally, by leveraging the vi-fastpass library, unattended authentication between VIMA/vMA and target ESX servers is made possible. This alleviates the need to continually pass in credentials for each ESX server.

By centralizing the backup process, individual virtual machine backup lists pertaining to their respective ESX(i) servers are no longer required. ghettoVCBg2 has the ability to identify virtual machines and their respective hosts, potentially eliminating any past confusion associated with maintaining multiple lists.

In its current configuration, the script will allow up to 3 unique backups of the Virtual Machine before it will overwrite the previous backups; this however, can be modified to fit procedures if need be. Please be diligent in running the script in a test or staging environment before using it on production live Virtual Machines; this script functions well within our environment but there is a chance that it may not fit well into other environments.

Features

Support for logging (normal & verbose)
Single VM backup list across multiple ESX/ESXi host(s)
Credential-less backups (so long as host(s) are being managed by VIMA/vMA)
Online back up of VM(s)
Only valid VMDKs presented to the VM will be backed up
Support for multiple VMDK backup per VM
Preserve original powerState of VM(s)
Ability to shutdown guestOS, initiate backup process and power on VM afterwards with the option of hardpower timeout
Ensure that snapshot removal process completes prior to continuing the backup process
VM(s) that initially contain snapshots and Physical RDMs (raw device mapppings) will not be backed up
Support for VM(s) with Virtual RDMs
Ability to specify the number of backup rotations per VM
Output back up VMDKs in ZEROEDTHICK (default behavior), EAGERZEROEDTHICK, 2GB SPARSE or THIN format
Ouput backup VMDKs using either BUSLOGIC or LSILOGIC adapter type
Fully support VMDKs stored across multiple datastores
VM snapshot memory and quiesce options
Individual VM backup policy (supported on ESX(i) 3.5u2+ & ESX(i) 4.x+)
Email logs (Experimental Support) NEW!
Ability to include/exclude specific VMDK(s) per VM (requires individual VM backup policy setup) NEW!
Independent disk aware (will ignore VMDK) NEW!
Additional debugging information including dry run execution NEW!

Requirements:

VMs running on ESX(i) 3.5u2+ or ESX(i) 4.x+
VMware VIMA 1.0 or VMware vMA 4.0

ESX/ESXi Version Support Table

Version	Support	Alternative
ESX 3.5u2	Requires at least Foundation/Essentials license	ghettoVCB
ESX 3.5u3	Requires at least Foundation/Essentials license	ghettoVCB
ESX 3.5u4	Requires at least Foundation/Essentials license	ghettoVCB
ESXi 3.5u2	Works for free	ghettoVCB
ESXi 3.5u3	Works for free	ghettoVCB
ESXi 3.5u4	Requires at least Foundation/Essentials license	ghettoVCB
ESX 4.0	Requires at least Foundation/Essentials license	ghettoVCB
ESXi 4.0	Requires at least Foundation/Essentials license	ghettoVCB

Setup:

1) Download ghettoVCBg2.pl to your VIMA 1.0/vMA 4.0 system

2) Change the permissions on the script to ensure it can be executed:

[vi-admin@scofield ~]$ chmod +x ghettoVCBg2.pl

Addings ESX/ESXi host(s) to VIMA/vMA

Prior to starting, ensure that the VMware VIMA/vMA host is managing the appropriate ESX(i) 3.5u2/4.x+ host(s)

1. Add ESXi hosts to VIMA/vMA managment using vifp

A password prompt for the vi-admin account and root password to the pertinent ESXi host(s) will be presented.

[vi-admin@vima-primp-industries ~]$ sudo vifp addserver olga.resnet.ucsb.edu
root@olga.resnet.ucsb.edu's password:
[vi-admin@vima-primp-industries ~]$

2. Verify that the appropriate hosts are being managed by VIMA/vMA

[vi-admin@vima-primp-industries ~]$ sudo vifp listservers
olga.resnet.ucsb.edu
himalaya.primp-industries.com

Configurations

The following variables (similar to that of ghettoVCB) need to be defined within the script prior to execution.

Defining the datastore in which the backups are stored:

my $VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups";

Defining the name of folder that will contain backups (if folder does not exist, it will automatically be created):

my $VM_BACKUP_DIRECTORY = "WILLIAM_BACKUPS";

Defining the backup rotation per VM:

my $VM_BACKUP_ROTATION_COUNT = "3";

Defining the backup disk format (zeroedthick, eagerzeroedthick, thin, and 2gbsparse are available):

my $DISK_BACKUP_FORMAT = "zeroedthick";

Defining the adapter type for backed up VMDK (buslogic, and lsilogic are available):

my $ADAPTER_FORMAT = "buslogic";

Defining whether the VM is powered down or not prior to backup (1 = enable, 0 = disable):

Note: VM(s) that are powered off will not require snapshoting

my $POWER_VM_DOWN_BEFORE_BACKUP = "0";

Defining whether virtual machine memory is snapped and if quiescing is enabled (1 = enable, 0 = disable):

Note: By default both are disabled

my $VM_SNAPSHOT_MEMORY = "0";
my $VM_SNAPSHOT_QUIESCE = "0";

Defining the detail of logging (verbose or normal):

my $LOG_LEVEL = "normal";

*Defining VMDK(s) to backup from a particular VM either a list of vmdks or "all"

VM_VMDK_FILES="myvmdk.vmdk"

Experimental Support Configs:

Note: Port 25 by default

my $SEND_MAIL = "no";
my $EMAIL_HOST = "emailserver";
my $EMAIL_DOMAIN = "localhost.localdomain";
my $EMAIL_TO = 'William Lam <william@primp-industries.com.com>';
my $EMAIL_FROM = 'ghettoVCBg2 <ghettoVCBg2@primp-industries.com.com>';

Ensure that you do not edit past this section:

########################## DO NOT MODIFY PAST THIS LINE ##########################

Usage:

[vi-admin@scofield ~]$ ./ghettoVCBg2.pl
Required command option 'vmlist' not specified.

Synopsis: ./ghettoVCBg2.pl OPTIONS


Command-specific options:
   --config_dir
      Name of directory containing VM(s) backup configurations
   --dryrun (default '0')
      Set to 1 to enable dryrun mode (default 0)
   --output (default '/tmp/ghettoVCBg2.log')
      Full path to output log (default /tmp/ghettoVCBg2.log)
   --vmlist (required)
      A file containing a list of virtual machine(s) to be backed up on host

Common VI options:
   --config (variable VI_CONFIG)
      Location of the VI Perl configuration file
   --credstore (variable VI_CREDSTORE)
      Name of the credential store file defaults to <HOME>/.vmware/credstore/vicredentials.xml on Linux and <APPDATA>/VMware/credstore/vicredentials.xml on Windows
   --encoding (variable VI_ENCODING, default 'utf8')
      Encoding: utf8, cp936 (Simplified Chinese), iso-8859-1 (German), shiftjis (Japanese)
   --help
      Display usage information for the script
   --passthroughauth (variable VI_PASSTHROUGHAUTH)
      Attempt to use pass-through authentication
   --passthroughauthpackage (variable VI_PASSTHROUGHAUTHPACKAGE, default 'Negotiate')
      Pass-through authentication negotiation package
   --password (variable VI_PASSWORD)
      Password
   --portnumber (variable VI_PORTNUMBER)
      Port used to connect to server
   --protocol (variable VI_PROTOCOL, default 'https')
      Protocol used to connect to server
   --savesessionfile (variable VI_SAVESESSIONFILE)
      File to save session ID/cookie to utilize
   --server (variable VI_SERVER, default 'localhost')
      VI server to connect to. Required if url is not present
   --servicepath (variable VI_SERVICEPATH, default '/sdk/webService')
      Service path used to connect to server
   --sessionfile (variable VI_SESSIONFILE)
      File containing session ID/cookie to utilize
   --url (variable VI_URL)
      VI SDK URL to connect to. Required if server is not present
   --username (variable VI_USERNAME)
      Username
   --verbose (variable VI_VERBOSE)
      Display additional debugging information
   --version
      Display version information for the script

Note: There are some additional command line arguments that can be provided upon runtime: --debug, --output and --vmlist. --vmlist is the only parameter required. The rest will have defaults if not explicitly defined.

--dryrun

Dry run information will be written to the log and no backup will take place

--output

Name of the output log, if the same log is used, it will automatically append (ensure you provide full path e.g. /tmp/ghettoVCBg2.log)

--config_dir

This directory will contain the individual VM backup policy files. If a virtual machine has an associated policy file, the global configuration will be ignored for that backup instance.

The input to this script is a file that contains the display name of the virtual machine(s) separated by a newline. When creating this file on a non-Linux/UNIX system, you may introduce a ^M character which may cause the script to misbehave. To ensure that this does not occur, please create the file on the ESX/ESXi host.

Provided is a sample of what the file should contain:

[vi-admin@scofield ~]$ cat virtual_machine_backup_list
VIMA
vMA-2

Sample Execution

Backup VMs located in a list
Backup VMs based on individual VM backup policies
Debug Mode
Dry run Mode

Backup VMs located in a list

Log verbosity: info (default)
Log output: /tmp/ghettoVCBg2.log (default)

DISK_FORMAT	2gbsparse
ADAPTER_FORMAT	buslogic
LOG_LEVEL	info
VM_SNAPSHOT_MEMORY	0
VM_SNAPSHOT_QUIESCE	0
VMS_TO_BACKUP	(VIMA,vMA-2)

[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --vmlist virtual_machine_backup_list

[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
        11-13-2009 23:34:53 --  info: ============================== ghettoVCBg2 LOG START ==============================
        11-13-2009 23:34:53 --  info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
        11-13-2009 23:34:53 --  info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
        11-13-2009 23:34:53 --  info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
        11-13-2009 23:34:53 --  info: CONFIG - DISK_BACKUP_FORMAT = 2gbsparse
        11-13-2009 23:34:53 --  info: CONFIG - ADAPTER_FORMAT = buslogic
        11-13-2009 23:34:53 --  info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
        11-13-2009 23:34:53 --  info: CONFIG - VM_SNAPSHOT_MEMORY = NO
        11-13-2009 23:34:53 --  info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
        11-13-2009 23:34:53 --  info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-13
        11-13-2009 23:34:53 --  info: CONFIG - VM_VMDK_FILES = all

        11-13-2009 23:34:58 --  info: Initiate backup for VIMA found on himalaya.primp-industries.com
        11-13-2009 23:35:18 --  info: VIMA has 1 VMDK(s)
        11-13-2009 23:37:37 --  info: Backup completed for VIMA!

        11-13-2009 23:37:37 --  info: Initiate backup for vMA-2 found on himalaya.primp-industries.com
        11-13-2009 23:37:47 --  info: vMA-2 has 1 VMDK(s)
        11-13-2009 23:40:07 --  info: Backup completed for vMA-2!

        11-13-2009 23:40:10 --  info: ============================== ghettoVCBg2 LOG END ==============================

Backup VMs based on individual VM backup policies and log output to /tmp/ghettoVCB.log

Log verbosity: info (default)
Log output: /tmp/ghettoVCB.log (default)

VIMA

VM_BACKUP_DATASTORE	dlgCore-NFS-bigboi.VM-Backups
VM_BACKUP_DIRECTORY	WILLIAM_BACKUPS
DISK_FORMAT	zeroedthick
ADAPTER_FORMAT	buslogic
LOG_LEVEL	debug
VM_SNAPSHOT_MEMORY	1
VM_SNAPSHOT_QUIESCE	1
VM_VMDK_FILES	VIMA_1.vmdk

vMA-2

VM_BACKUP_DATASTORE	dlgCore-NFS-bigboi.VM-Backups
VM_BACKUP_DIRECTORY	vGHETTO_BACKUPS
DISK_FORMAT	thin
ADAPTER_FORMAT	lsilogic
LOG_LEVEL	info
VM_SNAPSHOT_MEMORY	0
VM_SNAPSHOT_QUIESCE	0
VM_VMDK_FILES	all

1. Create folder to hold individual VM backup policies (can be named anything):

[vi-admin@scofield ~]$ mkdir backup_config

2. Create individual VM backup policies for each VM and ensure each file is named exactly as the display name of the VM being backed up (use provided template to create duplicates):

[vi-admin@scofield backup_config]$ cp ghettoVCBg2-vm_backup_configuration_template VIMA
[vi-admin@scofield backup_config]$ cp ghettoVCBg2-vm_backup_configuration_template vMA-2

4. Using a preferred editor (nano or vi), edit each VM backup policy file

Listing of VM backup policy within backup configuration directory

[vi-admin@scofield backup_config]$ ls
ghettoVCBg2-vm_backup_configuration_template  VIMA  vMA-2

5. Pass in the value, backup_config, into the --config_dir parameter

(e.g.)
--config_dir backupConfigs

Example of two distinct VM backup policies will be applied to VIMA and vMA-2

Backup policy for "VIMA" (backup only 1 specific VMDKs)

[vi-admin@scofield backup_config]$ cat VIMA
VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups"
VM_BACKUP_DIRECTORY = "WILLIAM_BACKUPS"
VM_BACKUP_ROTATION_COUNT = "3"
DISK_BACKUP_FORMAT = "zeroedthick"
ADAPTER_FORMAT = "buslogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
VM_SNAPSHOT_MEMORY = "1"
VM_SNAPSHOT_QUIESCE = "1"
LOG_LEVEL = "debug"
VM_VMDK_FILES = "VIMA_1.vmdk"

Backup policy for VM "vMA-2" (backup all VMDKs found)

VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups"
VM_BACKUP_DIRECTORY = "vGHETTO_BACKUPS"
VM_BACKUP_ROTATION_COUNT = "3"
DISK_BACKUP_FORMAT = "thin"
ADAPTER_FORMAT = "lsilogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
VM_SNAPSHOT_MEMORY = "0"
VM_SNAPSHOT_QUIESCE = "0"
LOG_LEVEL = "info"
VM_VMDK_FILES = "all"

The following is an output of the script when utilizing the new --config_dir parameter with debug mode enabled.

Note: --vmlist is still a required parameter with the exception that backup policy files must exist for each VM in the list else the VM will be ignored in the backup process.

Execution of backup

[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --config_dir backup_config --vmlist virtual_machine_backup_list

[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
        11-14-2009 00:18:41 --  info: ============================== ghettoVCBg2 LOG START ==============================
        11-14-2009 00:18:45 --  info: CONFIG - USING CONFIGURATION FILE = VIMA
        11-14-2009 00:18:45 --  info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
        11-14-2009 00:18:45 --  info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
        11-14-2009 00:18:45 --  info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
        11-14-2009 00:18:45 --  info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
        11-14-2009 00:18:45 --  info: CONFIG - ADAPTER_FORMAT = buslogic
        11-14-2009 00:18:45 --  info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
        11-14-2009 00:18:45 --  info: CONFIG - VM_SNAPSHOT_MEMORY = YES
        11-14-2009 00:18:45 --  info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
        11-14-2009 00:18:45 --  info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
        11-14-2009 00:18:45 --  info: CONFIG - VM_VMDK_FILES = VIMA_1.vmdk

        11-14-2009 00:18:45 --  info: Initiate backup for VIMA found on himalaya.primp-industries.com
        11-14-2009 00:18:55 --  debug: VIMA original powerState: poweredOn
        11-14-2009 00:18:55 --  debug: Creating Snapshot "ghettoVCBg2-snapshot-2009-11-14" for VIMA
        11-14-2009 00:19:45 --  info: VIMA has 2 VMDK(s)
        11-14-2009 00:19:45 --  debug: findVMDKFile: Found VMDK File: VIMA_1.vmdk
        11-14-2009 00:19:45 --  debug: backupVMDK: Backing up "[http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk" to "[http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14/VIMA_1.vmdk"
        11-14-2009 00:19:45 --  debug: backupVMDK: Signal copyThread to start
        11-14-2009 00:19:45 --  debug: backupVMDK: Backup progress: Elapsed time 0 min
        11-14-2009 00:19:45 --  debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
        11-14-2009 00:19:45 --  debug: CopyThread: Start backing up VMDK(s) ...
        11-14-2009 00:19:52 --  debug: copyTask: send copySuccess message ...
        11-14-2009 00:19:52 --  debug: copyTask: waiting for next job and sleep ...
        11-14-2009 00:20:46 --  debug: backupVMDK: Successfully completed backup for [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk Elapsed time: 1 min
        11-14-2009 00:20:49 --  debug: Removing Snapshot "ghettoVCBg2-snapshot-2009-11-14" for VIMA
        11-14-2009 00:20:51 --  debug: checkVMBackupRotation: Starting ...
        11-14-2009 00:20:57 --  debug: Purging [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-13--3 due to rotation max
        11-14-2009 00:21:05 --  info: Backup completed for VIMA!

        11-14-2009 00:21:05 --  debug: reConfigureBackupParams: VM - vMA-2
        11-14-2009 00:21:05 --  info: CONFIG - USING CONFIGURATION FILE = vMA-2
        11-14-2009 00:21:05 --  info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
        11-14-2009 00:21:05 --  info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
        11-14-2009 00:21:05 --  info: CONFIG - VM_BACKUP_DIRECTORY = vGHETTO_BACKUPS
        11-14-2009 00:21:05 --  info: CONFIG - DISK_BACKUP_FORMAT = thin
        11-14-2009 00:21:05 --  info: CONFIG - ADAPTER_FORMAT = lsilogic
        11-14-2009 00:21:05 --  info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
        11-14-2009 00:21:05 --  info: CONFIG - VM_SNAPSHOT_MEMORY = NO
        11-14-2009 00:21:05 --  info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
        11-14-2009 00:21:05 --  info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
        11-14-2009 00:21:05 --  info: CONFIG - VM_VMDK_FILES = all

        11-14-2009 00:21:05 --  info: Initiate backup for vMA-2 found on himalaya.primp-industries.com
        11-14-2009 00:21:15 --  info: vMA-2 has 1 VMDK(s)
        11-14-2009 00:23:22 --  info: Backup completed for vMA-2!

        11-14-2009 00:23:25 --  info: ============================== ghettoVCBg2 LOG END ==============================

Dry run Mode (no backup will take place)

Log verbosity: drymode
Log output: /tmp/ghettoVCB.log (default)

[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --config_dir backup_config --vmlist virtual_machine_backup_list --dryrun 1

[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
        11-14-2009 00:26:29 --  info: ============================== ghettoVCBg2 LOG START ==============================
        11-14-2009 00:26:34 --  info: CONFIG - USING CONFIGURATION FILE = VIMA
        11-14-2009 00:26:34 --  info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
        11-14-2009 00:26:34 --  info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
        11-14-2009 00:26:34 --  info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
        11-14-2009 00:26:34 --  info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
        11-14-2009 00:26:34 --  info: CONFIG - ADAPTER_FORMAT = buslogic
        11-14-2009 00:26:34 --  info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
        11-14-2009 00:26:34 --  info: CONFIG - VM_SNAPSHOT_MEMORY = YES
        11-14-2009 00:26:34 --  info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
        11-14-2009 00:26:34 --  info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
        11-14-2009 00:26:34 --  info: CONFIG - VM_VMDK_FILES = VIMA_1.vmdk

        11-14-2009 00:26:34 --  info: ---------- DRYRUN DEBUG INFO VIMA ----------
        11-14-2009 00:26:34 --  info: DEBUG - Host Build: VMware ESX 4.0.0 build-164009
        11-14-2009 00:26:34 --  info: DEBUG - Host: himalaya.primp-industries.com
        11-14-2009 00:26:34 --  info: DEBUG - Virtual Machine: VIMA
        11-14-2009 00:26:34 --  info: DEBUG - VM ConfigPath: [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA.vmx
        11-14-2009 00:26:34 --  info: DEBUG - VMX File: VIMA.vmx
        11-14-2009 00:26:34 --  info: DEBUG - BackupConfigPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14/VIMA.vmx
        11-14-2009 00:26:34 --  info: DEBUG - BackupPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14
        11-14-2009 00:26:34 --  info: DEBUG - VM Datastore: himalaya-local-SAS.Savvio
        11-14-2009 00:26:34 --  info: DEBUG - VMDK(s):
        11-14-2009 00:26:34 --  info: DEBUG - [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA.vmdk
        11-14-2009 00:26:34 --  info: DEBUG - [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk
        11-14-2009 00:26:34 --  info: ---------- DRYRUN DEBUG INFO VIMA ----------

        11-14-2009 00:26:34 --  debug: reConfigureBackupParams: VM - vMA-2
        11-14-2009 00:26:34 --  info: CONFIG - USING CONFIGURATION FILE = vMA-2
        11-14-2009 00:26:34 --  info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
        11-14-2009 00:26:34 --  info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
        11-14-2009 00:26:34 --  info: CONFIG - VM_BACKUP_DIRECTORY = vGHETTO_BACKUPS
        11-14-2009 00:26:34 --  info: CONFIG - DISK_BACKUP_FORMAT = thin
        11-14-2009 00:26:34 --  info: CONFIG - ADAPTER_FORMAT = lsilogic
        11-14-2009 00:26:34 --  info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
        11-14-2009 00:26:34 --  info: CONFIG - VM_SNAPSHOT_MEMORY = NO
        11-14-2009 00:26:34 --  info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
        11-14-2009 00:26:34 --  info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
        11-14-2009 00:26:34 --  info: CONFIG - VM_VMDK_FILES = all

        11-14-2009 00:26:34 --  info: ---------- DRYRUN DEBUG INFO vMA-2 ----------
        11-14-2009 00:26:34 --  info: DEBUG - Host Build: VMware ESX 4.0.0 build-164009
        11-14-2009 00:26:34 --  info: DEBUG - Host: himalaya.primp-industries.com
        11-14-2009 00:26:34 --  info: DEBUG - Virtual Machine: vMA-2
        11-14-2009 00:26:34 --  info: DEBUG - VM ConfigPath: [http://himalaya-local-SAS.Constellation|http://himalaya-local-SAS.Constellation] vMA-2/vMA-2.vmx
        11-14-2009 00:26:34 --  info: DEBUG - VMX File: vMA-2.vmx
        11-14-2009 00:26:34 --  info: DEBUG - BackupConfigPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] vGHETTO_BACKUPS/vMA-2/vMA-2-2009-11-14/vMA-2.vmx
        11-14-2009 00:26:34 --  info: DEBUG - BackupPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] vGHETTO_BACKUPS/vMA-2/vMA-2-2009-11-14
        11-14-2009 00:26:34 --  info: DEBUG - VM Datastore: himalaya-local-SAS.Constellation
        11-14-2009 00:26:34 --  info: DEBUG - VMDK(s):
        11-14-2009 00:26:34 --  info: DEBUG - [http://himalaya-local-SAS.Constellation|http://himalaya-local-SAS.Constellation] vMA-2/vMA-2.vmdk
        11-14-2009 00:26:34 --  info: ---------- DRYRUN DEBUG INFO vMA-2 ----------

        11-14-2009 00:26:37 --  info: ============================== ghettoVCBg2 LOG END ==============================

FAQ

1Q: I have a VM listed in my backup file but no backup of that VM took place. The log stated that the VM could not be found. How could this be? I know that it’s running on host X…
1A Ensure the ESX/ESXi host is being managed by VMware VIMA/vMA ( sudo vifp listservers to verify)

2Q: I'm getting error X when using the script or I'm not getting any errors, the backup didn’t even take place. Oh what could be wrong?
2A: Please provide the entire output of a manual execution (debug & dry run mode) or if you're logging the output withing \{code\} tags, a short description of your environment and the type of VM(s) you're trying to backup—I will try to provide some assistance.

3Q: Can I schedule backups to take place hourly, daily, monthly, yearly?
3A: Yes, do a search online for crontab.

4Q: I want to schedule my backup on Windows, how do I do this?
4A: Do a search for plink. Make sure you have paired SSH keys setup between your Windows system and VIMA/vMA.

5Q: Do I have to use VMware VIMA/vMA to initiate the backup?
5A: Yes, vi-fastpass is being used and only VIMA/vMA provides this capability. However, you can kickoff the process from another host (*nix/win)

6Q: I only have a single ESXi host. I want to take backups and store them somewhere else. The problem is: I don't have NFS, iSCSI nor FC SAN. What can I do?
6A: You can use local storage to store your backups assuming that you have enough space on the destination datastore. Afterwards, you can use scp (WinSCP/FastSCP) or vifs (vCLI) to transfer the backups from the ESXi host to your local desktop.

7Q: I’m pissed; the backup is taking too long. My datastore is of type X?
7A: YMMV, take a look at your storage configuration and make sure it is optimized.

8Q: I noticed that the backup rotation is occurring after a backup. I don't have enough local storage space, can the process be changed?
8A: This is primarily done to ensure that you have at least one good backup in case the new backup fails. If you would like to modify the script, you're more than welcome to do so.

9Q: What is the best storage configuration for datastore type X?
9A: Search the VMTN forums; there are various configurations for the different type of storage/etc.

10Q: I want to setup NFS Server to run my backups. Which is the best and should it be virtual or physical?
10A: Please refer to answer 7A. From experience, we’ve seen physical instances of NFS servers to be faster than their virtual counterparts. As always, YMMV.

11Q: When I try to use the script I get an error stating: "restricted version". What the heck is this?
11A: Please take a look at the ESX/ESXi Version Support Table in this document. The reason is explained here.

12Q: I'm getting error X when using the script or I'm not getting any errors, the backup didn’t even take place. Oh what could be wrong?
12A: First, you'll want to check the log (default /tmp/ghettoVCBg2) and see if there are any obvious errors. If not, please provide the entire output of the log, a short description of your environment and the type of VM(s) you're trying to backup—I will try to provide some assistance.

13Q: I have VMs that have snapshots. I want to back these things up but the script doesn’t let me do it. How do I fix that?
13A: VM snapshots are not meant to be kept for long durations. When backing up a VM that contains a snapshot, you should ensure all snapshots have been committed prior to running a backup. No exceptions will be made…ever.

14Q: I would like to restore from backup, what is the best method?
14A: The restore process will be unique for each environment and should be determined by your backup/recovery plans. At a high level you have the option of mounting the backup datastore and registering the VM in question or copy the VM from the backup datastore to the ESX/ESXi host. The latter is recommended so that you're not running a VM living on the backup datastore or inadvertently modifying your backup VM(s).

15Q: When using the experimental feature: Individual VM backup policy, if I don't have a policy will it use the default configurations defined in the script?
15A: No, this use case will not be supported, ever. If you decide to use the per VM policy, then the global configuration in the script is ignored. If a VM policy config is not found, the VM will not be backed up even if it's in the –-vmlist.

16Q: In vMA 4.0, vCenter target will be supported, can I just add vCenter target and assume this will work?
16A: At the moment, this is not a supported feature. You will need to add each ESX/ESXi host for the vi-fastpass library to be able to locate the host(s).

17Q: When I try to run the script I get: "-bash: ./ghettoVCBg2.pl: Permission denied", what is wrong?
17A: You need to change the permission on the script to be executable, chmod +x ghettoVCBg2.pl

18Q: Where can I download the latest version of the script?
18A: The latest version is available on this page at the bottom. To get the version of the script run ./ghettoVCBg2.pl --version

Change Log

11/18/09 - New variable "EMAIL_DOMAIN" was added for email log functionality as email domain must be defined or left as default "localhost.localdomain". Thanks to adrian888 for finding this bug

my $EMAIL_DOMAIN = "localhost.localdomain";

11/17/09 - The following enhancements and fixes have been implemented in this release of ghettoVCBg2. Special thanks goes out to Gerhard Ostermann for assisting with some of the logic in the ghettoVCBg2 script and the rest of the ghettoVCBg2 BETA testers. Thanks for everyones time and comments to make this script better!

Enhancements:

Email log support
Include/exclude specific VMDK(s)
Additional logging + dry run mode

Fixes:

Independent disk aware
Large VMDK backups

Attachments:

ghettoVCBg2-vm_backup_configuration_template (315 bytes)
ghettoVCBg2.pl (35.1 K)

Tags: vcb, backup, vm_backup, esx3.5, vima, vima1.0, vma, vma4.0, vghettovcb, esx3.5, esx4, esxi_3.5, esxi_4, esxi_4

Average User Rating

(3 ratings)

Comments (176)

May 21, 2009 10:07 PM smyle says:

I have a VM with a virtual RDM. When I run the script, it immediately says 'WARN - RDM was found for (vmname), backup will not take place.' The comments in the code seem to indicate that this is normal, though the 'features list' above says they're supported. Any advice?

May 22, 2009 12:27 AM lamw

says: in response to: smyle

Hi smyle,

I'm sorry, I think when I originally started writing the script I was implementing the features in ghettoVCB which did not support RDM period...I'm looking at the code right now and it is in fact blocking RDM's all together, let me jot that down as a note and I'll get back to you.

Thanks

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
http://twitter.com/lamw

If you find this information useful, please award points for "correct" or "helpful".

May 22, 2009 11:38 AM lamw

says:

Everyone,

There is a known bug at the moment with vMA 4.0 and vSphere ESX(i) 4.0 that will not allow you to backup more than 1 VM, the logs will state that VM X can not be found after the first VM on the list is completed. I initially saw this with vMA 4.0 GA with vSphere ESX 4.0 RC but was hoping this might be due to the miss-match of the build. Something was dramatically alter/changed during RC release of vMA 4.0 and the GA release of vMA 4.0 that's causing this problem. I'll be investigating the root cause, but with VIMA 1.0 & ESX(i) 3.5u2+ will work and has been tested/verified.

Please let me know if you run into any other issues.

Thanks

May 22, 2009 2:22 PM lamw

says: in response to: lamw

The bug has been identified and the issue actually lies with the vSphere CLI's version of vifs, I'm still awaiting feedback from VMware to confirm the issue.

Some details can be found here: http://engr.ucsb.edu/~duonglt/vmware/vifs_bug.html

Once the issue has been identified, I'll push out a temporarily fix to remediate the issue.

May 26, 2009 8:21 PM lamw

says:

Just updated the script to fix both the virtual RDM issue & a bug with VMware's vifs, let me know if anyone runs into any other issues

May 28, 2009 5:34 PM yp1 says:

Hi,

It does not work for me with ESXi 4. It exits in 10-15 seconds. Looks like it fails to take a snapshot. I tried different types of the snapshots - same result. This is the latest version (same problem with the previous one).

Here is a typical log.

05-27-2009 18:03:53 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
05-27-2009 18:03:53 -- CONFIG - VM_BACKUP_DATASTORE = backup
05-27-2009 18:03:53 -- CONFIG - VM_BACKUP_DIRECTORY = backups
05-27-2009 18:03:53 -- CONFIG - DISK_BACKUP_FORMAT = eagerzeroedthick
05-27-2009 18:03:53 -- CONFIG - ADAPTER_FORMAT = lsilogic
05-27-2009 18:03:53 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
05-27-2009 18:03:53 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-27-2009 18:03:53 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-27-2009 18:03:53 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-05-27

05-27-2009 18:03:58 -- Initiate backup for "winxp" found on aaa.bbb.ccc.ddd!
05-27-2009 18:04:11 -- winxp original powerState: poweredOff
05-27-2009 18:04:11 -- winxp has 1 VMDK(s)
05-27-2009 18:04:11 -- Backing up "nexenta_nfs winxp/winxp.vmdk" to "backup backups/winxp/winxp-2009-05-27/winxp.vmdk"
05-27-2009 18:04:15 -- Backup completed for nexenta_nfs winxp/winxp.vmdk
05-27-2009 18:04:20 -- Backup completed for "winxp"!

May 28, 2009 6:12 PM lamw

says: in response to: yp1

Are you using the free version of ESXi unlicensed? Licensed ESXi 4.0 or eval ESXi 4.0? I assume you're using vMA 4.0?

Can you please re-run the script and set the loglevel to "verbose" and attach the entire log as an attachment. Also run the script passing in the commandline param: --debug 1 and provide the entire log output as an attachment.

May 28, 2009 7:31 PM yp1 says: in response to: lamw

Sorry, I could not figure out how to attach a file, so have to add the log inline.
It is Free ESXi 4 with a registered license (not eval).

Verbose mode run, followed by debug:

============================== 05-28-2009 18:16:52 -- ghettoVCBg2 LOG START ==============================
05-28-2009 18:16:52 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
05-28-2009 18:16:52 -- CONFIG - VM_BACKUP_DATASTORE = backup
05-28-2009 18:16:52 -- CONFIG - VM_BACKUP_DIRECTORY = backups
05-28-2009 18:16:52 -- CONFIG - DISK_BACKUP_FORMAT = thin
05-28-2009 18:16:52 -- CONFIG - ADAPTER_FORMAT = lsilogic
05-28-2009 18:16:52 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
05-28-2009 18:16:52 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-28-2009 18:16:52 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-28-2009 18:16:52 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-05-28

05-28-2009 18:17:49 -- Initiate backup for "winxp" found on aaa.bbb.ccc.ddd!
05-28-2009 18:18:28 -- winxp original powerState: poweredOff
05-28-2009 18:18:28 -- winxp has 1 VMDK(s)
05-28-2009 18:18:28 -- Backing up "nexenta_nfs winxp/winxp.vmdk" to "backup backups/winxp/winxp-2009-05-28/winxp.vmdk"
05-28-2009 18:18:33 -- Backup completed for nexenta_nfs winxp/winxp.vmdk
05-28-2009 18:18:39 -- Backup completed for "winxp"!

============================== 05-28-2009 18:18:39 -- ghettoVCBg2 LOG END ==============================

============================== 05-28-2009 18:27:08 -- ghettoVCBg2 LOG START ==============================
05-28-2009 18:27:08 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
05-28-2009 18:27:08 -- CONFIG - VM_BACKUP_DATASTORE = backup
05-28-2009 18:27:08 -- CONFIG - VM_BACKUP_DIRECTORY = backups
05-28-2009 18:27:08 -- CONFIG - DISK_BACKUP_FORMAT = thin
05-28-2009 18:27:08 -- CONFIG - ADAPTER_FORMAT = lsilogic
05-28-2009 18:27:08 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
05-28-2009 18:27:08 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-28-2009 18:27:08 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-28-2009 18:27:08 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-05-28

05-28-2009 18:27:12 --

DEBUG INFO winxp

05-28-2009 18:27:12 -- DEBUG - Host Build: VMware ESXi 4.0.0 build-164009
05-28-2009 18:27:12 -- DEBUG - Host: aaa.bbb.ccc.ddd
05-28-2009 18:27:12 -- DEBUG - Virtual Machine: winxp
05-28-2009 18:27:12 -- DEBUG - VM ConfigPath: nexenta_nfs winxp/winxp.vmx
05-28-2009 18:27:12 -- DEBUG - VMX File: winxp.vmx
05-28-2009 18:27:12 -- DEBUG - BackupConfigPath: backup backups/winxp/winxp-2009-05-28/winxp.vmx
05-28-2009 18:27:12 -- DEBUG - BackupPath: backup backups/winxp/winxp-2009-05-28
05-28-2009 18:27:12 -- DEBUG - VM Datastore: nexenta_nfs
05-28-2009 18:27:12 -- DEBUG - VMDK(s):
05-28-2009 18:27:12 -- DEBUG - nexenta_nfs winxp/winxp.vmdk
05-28-2009 18:27:12 --

DEBUG INFO winxp

============================== 05-28-2009 18:27:12 -- ghettoVCBg2 LOG END ==============================

May 28, 2009 7:35 PM lamw

says: in response to: yp1

Is your license either Foundation, Standard or Enterprise?

May 28, 2009 7:45 PM yp1 says: in response to: lamw

It is a Free license (I just downloaded and requested the license)

Product: ESXi 4 Single Server Licensed for 1 physical CPUs (1-6 cores per CPU)

May 28, 2009 7:48 PM lamw

says: in response to: yp1

Please take a look at the FAQ section, specifically 9A for your answer.

Jun 17, 2009 3:25 AM kpc says: in response to: lamw

Hi William

Good work on the scripts, not had a chance to try out this new script yet.

Love the centralized feature, will save on having to setup the NFS share on each ESXi host.

just wondering if you plan on adding in a feature where we don't have to keep lists of VM's at all, basically assuming that the user will always want to backup every VM they have. I know you now only have to keep the single list but I find I have so many VM's which come and go I often forget to add them to the backup list.

Cheers

Jun 17, 2009 12:28 PM angelone says:

Hi William,
thanks for this great script, this is exactly what I looked for.
But I have a small problem: The cloning doesn't work

I'm on ESXi 3.5u3 with vMA 4.0

When I'm on the ESXi via ssh and do the cloning with vmkfstools -i this works.
Your script displays "Unable to clone virtual disk : destName"

This is what it says:
~~vi-admin@service01 ~~~$ ./ghettoVCBg2.pl --config_dir backupConfigs --vmlist vmlist --debug 0
Unable to clone virtual disk : destName

This is the logfile:
============================== 06-17-2009 12:15:10 -- ghettoVCBg2 LOG START ==============================

06-17-2009 12:15:13 -- CONFIG - USING CONFIGURATION FILE = pcbae04
06-17-2009 12:15:13 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
06-17-2009 12:15:13 -- CONFIG - VM_BACKUP_DATASTORE = nas_7_backup
06-17-2009 12:15:13 -- CONFIG - VM_BACKUP_DIRECTORY = sicherung
06-17-2009 12:15:13 -- CONFIG - DISK_BACKUP_FORMAT = thin
06-17-2009 12:15:13 -- CONFIG - ADAPTER_FORMAT = buslogic;
06-17-2009 12:15:13 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = YES
06-17-2009 12:15:13 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-17-2009 12:15:13 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-17-2009 12:15:13 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-06-17
06-17-2009 12:15:13 -- Initiate backup for "pcbae04" found on 5.20.33.2!
06-17-2009 12:15:28 -- pcbae04 original powerState: poweredOff
06-17-2009 12:15:28 -- pcbae04 has 1 VMDK(s)
06-17-2009 12:15:28 -- Backing up "datastore1 pcbae04/pcbae04.vmdk" to "nas_7_backup sicherung/pcbae04/pcbae04-2009-06-17/pcbae04.vmdk"
06-17-2009 12:15:32 -- Backup completed for datastore1 pcbae04/pcbae04.vmdk
06-17-2009 12:15:38 -- Purging "nas_7_backup sicherung/pcbae04/pcbae04-2009-06-17--2" due to rotation max
06-17-2009 12:15:46 -- Backup completed for "pcbae04"!

============================== 06-17-2009 12:15:47 -- ghettoVCBg2 LOG END ==============================

And this is the backupconfig for pcbae04:
=====================================
VM_BACKUP_DATASTORE = "nas_7_backup"
VM_BACKUP_DIRECTORY = "sicherung"
VM_BACKUP_ROTATION_COUNT = "2"
DISK_BACKUP_FORMAT = "thin"
ADAPTER_FORMAT = "buslogic";
POWER_VM_DOWN_BEFORE_BACKUP = "1"
LOG_LEVEL = "verbose"
VM_SNAPSHOT_MEMORY = "0"
VM_SNAPSHOT_QUIESCE = "0"
=====================================

The only file copied to the NAS is the .vmx file.
It seems the whole backup process works, even snapshots are generated.
Just the clone command itself fails.

Do you have an idea what happens here?

thanks in advance,
Martin

Jun 17, 2009 7:30 PM lamw

says: in response to: kpc

Thanks for the comments.

There will be no plans on changing the script to automatically assume all VM(s) will be backed up. If you need this feature you can modify the script accordlying, it's not that hard

Jun 17, 2009 7:40 PM lamw

says: in response to: angelone

I know I ran into some oddities with VIMA 1.0's version of vmkfstools.pl but since you're using vMA 4.0, there should not be any issues. I've tested vMA with ESX 3.5u3 but not with ESXi and it should not matter in either case.

Looking at the logs, everything looks okay to me and I can't see anything that could be off. Can you provide the debugging output by running the following:

./ghettoVCBg2.pl --config_dir backupConfigs --vmlist vmlist --debug 1

Also can you modify the script and see what the actual command for trying to execute the backup is, please find line 623 which should look like the following:

$vmkfstools_cpy = `$vmkfstools_cmd --server "$host" --username "$host_username" --password "$host_password" -        i "$_" -a $ADAPTER_FORMAT -d $DISK_BACKUP_FORMAT "$vmdk_backup_destination" 2>&1`;

and replace it with:

$vmkfstools_cpy = `$vmkfstools_cmd --server "$host" --username "$host_username" --password "$host_password" -        i "$_" -a $ADAPTER_FORMAT -d $DISK_BACKUP_FORMAT "$vmdk_backup_destination"`;
print "Copy command: ", $vmkfstools_cpy,"\n";

Re-run the script as the following:

./ghettoVCBg2.pl --config_dir backupConfigs --vmlist vmlist

FYI - You don't need to specify "--debug 0" as it defaults to 0.

Jun 18, 2009 12:58 AM angelone says: in response to: lamw

Mea culpa!
Got it...

After an hour of debugging and digging in my great Perl knowledge I found the problem in the backupconfig:

ADAPTER_FORMAT = "buslogic";

That small semicolon at the end of the line leads to the ADAPTER_FORMAT variable being empty
So the vmkfstools command fails because -d needs a parameter.

Everything works now, thanks for the good work.

regards,
Martin

Jun 18, 2009 1:45 AM kpc says: in response to: lamw

Tried the script, great work. If anyone else is backing up to NFS datastores you can mount the same store in the VMA appliance using the following:

sudo service portmap start
sudo service nfs start
sudo chkconfig portmap on
sudo chkconfig nfs on

sudo vi /etc/fstab (Add your nfs entry here e.g.)

10.192.4.99:/vmware /backup_vmware nfs defaults 0 0

sudo mount -a

Hope this helps

Jun 19, 2009 1:39 PM roadfox says:

Thanks for this great script and documentation.

Some small errors:

In "vm-backup-config-template" there is a semicolon at the of of ADAPTER_FORMAT which breaks the script when using config_dir

And some text in the log output is wrong QUIESCE where it should be MEMORY:

diff -f ghettoVCBg2.pl.bak ghettoVCBg2.pl
c195
print LOG timeStamp('MDYHMS'), "CONFIG - VM_SNAPSHOT_MEMORY = $quitext\n";
.
c390
print LOG timeStamp('MDYHMS'), "CONFIG - VM_SNAPSHOT_MEMORY = $quitext\n";
.

Jun 20, 2009 8:30 AM lamw

says: in response to: angelone

Thanks, a new vm-backup-config-template has been uploaded.

Jun 20, 2009 8:32 AM lamw

says: in response to: kpc

Is there any reason you would mount the backup datastore to vMA? In general, I would recommend locking down the NFS datastore to only ESX(i) hosts that way no accidental mounting from another host is allowed.

Jun 20, 2009 8:32 AM lamw

says: in response to: roadfox

Thanks, a new vm-backup-config-template has been uploaded.

Jun 25, 2009 8:06 AM IT_Architect says: in response to: lamw

THANK YOU for this script. It's a real life saver. However, GhettoVCBg2 will be the deciding factor of whether or not people upgrade from ESXi 3.5u3. You could remedy that by adding thin provisioning support and a logging fix to GhettoVCB4i.

There are many people just getting in to virtualization who are not going to buy a foundation license. They will grow with the first virtualization vendor that has a solution that makes businesses sense for their current need, which includes a backup. Those who buy a foundation license, automatically receive VMware's VCB. For one or two servers, I would rather use GhettoVCB4i and a Windows machine to schedule it. People will need custom oem.tgz files with ESXi to address their needs for the foreseeable future anyway.

Please consider adding thin provisioning support and a logging fix to GhettoVCB4i, with the thin provisioning being the most important.

Thanks!

Jun 26, 2009 6:48 AM Al3xandr3 says: in response to: IT_Architect

Hello lamw,

thanks a lot for your work but i got problems to run it.

Just after the snapshot, i got an error to copy virtual disk. It say: "general fault cause by file. One of the parameters supplied is invalid."

I use ESXi 3.5U3 with vMA4.0.

I tried with 2 differents VMs, with the local datastore and one NFS store and it still...

in /var/log/messages:
Jun 26 13:46:52 Hostd: ~~2009-06-26 13:46:52.014 'VdisksvcPlugin' 114696 warning~~ Invalid desination disk format 'preallocated'
Jun 26 13:46:52 Hostd: ~~2009-06-26 13:46:52.050 'BaseLibs' 114696 info~~ DISKLIB-LIB : Opened "/vmfs/volumes/4a4470d9-522186bf-68b6-0016355d8d4b/w2k3oem/w2k3oem.vmdk" (flags 0x6). A821FA4
Jun 26 13:46:52 Hostd: ~~2009-06-26 13:46:52.050 'BaseLibs' 114696 info~~ DISKLIB-LIB : Invalid type (0)
Jun 26 13:46:52 Hostd: ~~2009-06-26 13:46:52.050 'BaseLibs' 114696 info~~ DISKLIB-LIB : Failed to clone : One of the parameters supplied is invalid (1).
Jun 26 13:46:52 Hostd: ~~2009-06-26 13:46:52.050 'VdisksvcPlugin' 114696 warning~~ Failed to clone disk : One of the parameters supplied is invalid (1).

Ok, i found the solution. It was an error from DISK_BACKUP_FORMAT definition

sorry

Jun 28, 2009 3:42 PM Shawn-H says:

Hi lamw,

Thanks for the script. I think it will be exactly what we are looking for but I am having some issues getting it to work. I am running it from VMA 4.0 and connecting to 2 ESXi 3.5 hosts. I have attached the log with a trial run and a debug run. Looks like it is having troubles doing a snapshot. Would really appreciate if you could take a look and give me some ideas. Thanks.

============================== 06-27-2009 22:09:00 -- ghettoVCBg2 LOG START ==============================
06-27-2009 22:09:00 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
06-27-2009 22:09:00 -- CONFIG - VM_BACKUP_DATASTORE = /vmfs/volumes/SANDisk2
06-27-2009 22:09:00 -- CONFIG - VM_BACKUP_DIRECTORY = Backups
06-27-2009 22:09:00 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
06-27-2009 22:09:00 -- CONFIG - ADAPTER_FORMAT = buslogic
06-27-2009 22:09:00 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
06-27-2009 22:09:00 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-27-2009 22:09:00 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-27-2009 22:09:00 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-06-27

06-27-2009 22:09:02 -- Initiate backup for "AntivirusServer" found on esx2.brwmsc!
06-27-2009 22:09:06 -- AntivirusServer original powerState: poweredOn
06-27-2009 22:09:06 -- AntivirusServer has 1 VMDK(s)
06-27-2009 22:09:06 -- Backing up "SANDisk2 Antivirus Server/Antivirus Server.vmdk" to "[/vmfs/volumes/SANDisk2] Backups/AntivirusServer/AntivirusServer-2009-06-27/Antivirus Server.vmdk"
06-27-2009 22:09:07 -- Backup completed for SANDisk2 Antivirus Server/Antivirus Server.vmdk
06-27-2009 22:09:07 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-06-27" not found
06-27-2009 22:09:08 -- Unable to purge "[/vmfs/volumes/SANDisk2] Backups/AntivirusServer/GET https:esx2.brwmscfolderBackupsAntivirusServer?dsName=vmfsvolumesSANDisk2 unsuccessful : 404 Not Found" due to rotation max
06-27-2009 22:09:08 -- Backup completed for "AntivirusServer"!

============================== 06-27-2009 22:09:08 -- ghettoVCBg2 LOG END ==============================

============================== 06-28-2009 14:30:24 -- ghettoVCBg2 LOG START ==============================
06-28-2009 14:30:24 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
06-28-2009 14:30:24 -- CONFIG - VM_BACKUP_DATASTORE = /vmfs/volumes/SANDisk2
06-28-2009 14:30:24 -- CONFIG - VM_BACKUP_DIRECTORY = Backups
06-28-2009 14:30:24 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
06-28-2009 14:30:24 -- CONFIG - ADAPTER_FORMAT = buslogic
06-28-2009 14:30:24 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
06-28-2009 14:30:24 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-28-2009 14:30:24 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-28-2009 14:30:24 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-06-28

06-28-2009 14:30:25 --

DEBUG INFO AntivirusServer

06-28-2009 14:30:25 -- DEBUG - Host Build: VMware ESX Server 3i 3.5.0 build-163429
06-28-2009 14:30:25 -- DEBUG - Host: esx2.brwmsc
06-28-2009 14:30:25 -- DEBUG - Virtual Machine: AntivirusServer
06-28-2009 14:30:25 -- DEBUG - VM ConfigPath: SANDisk2 Antivirus Server/Antivirus Server.vmx
06-28-2009 14:30:25 -- DEBUG - VMX File: Antivirus Server.vmx
06-28-2009 14:30:25 -- DEBUG - BackupConfigPath: [/vmfs/volumes/SANDisk2] Backups/AntivirusServer/AntivirusServer-2009-06-28/Antivirus Server.vmx
06-28-2009 14:30:25 -- DEBUG - BackupPath: [/vmfs/volumes/SANDisk2] Backups/AntivirusServer/AntivirusServer-2009-06-28
06-28-2009 14:30:25 -- DEBUG - VM Datastore: SANDisk2
06-28-2009 14:30:25 -- DEBUG - VMDK(s):
06-28-2009 14:30:25 -- DEBUG - SANDisk2 Antivirus Server/Antivirus Server.vmdk
06-28-2009 14:30:25 --

DEBUG INFO AntivirusServer

============================== 06-28-2009 14:30:26 -- ghettoVCBg2 LOG END ==============================

Jun 29, 2009 6:07 AM Koldijk says: in response to: lamw

Hello thanks for this script, I am still using your older script! Trying your new script gives me some errors

~~vi-admin@bbikol-vima ~~~$ ./ghettoVCBg2.pl --config_dir backupConfigs --vmlist vm_backup_list
Use of uninitialized value in pattern match (m//) at /usr/lib/perl5/5.8.8/VMware/VIExt.pm line 124.
Use of uninitialized value in pattern match (m//) at /usr/lib/perl5/5.8.8/VMware/VIExt.pm line 124.
Use of uninitialized value in pattern match (m//) at /usr/lib/perl5/5.8.8/VMware/VIExt.pm line 124.
Use of uninitialized value in pattern match (m//) at /usr/lib/perl5/5.8.8/VMware/VIExt.pm line 124.
Use of uninitialized value in concatenation (.) or string at /usr/bin/vifs.pl line 294.
Unable to copy NFS BBIKOL-ZW-DC01/BBIKOL-ZW-DC01.vmx to :
destinationName

backupConfigs: 1 file named BBIKOL-ZW-DC01

##################################

VM BACKUP CONFIGURATIONS ###

##################################
VM_BACKUP_DATASTORE = "vmbackup"
VM_BACKUP_DIRECTORY = "esx"
VM_BACKUP_ROTATION_COUNT ="7"
DISK_BACKUP_FORMAT = "zeroedthick"
ADAPTER_FORMAT = "lsilogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
LOG_LEVEL = "normal"
VM_SNAPSHOT_MEMORY = "0"
VM_SNAPSHOT_QUIESCE = "0"

Ideas?

Jun 30, 2009 7:56 AM freez267 says: in response to: Koldijk

I have some problem to backup a VM in my esxi 4.0 server. I always receive this error:

/vmfs/volumes/48d3c7c5-9c6dddaa-59ff-001ec9b5cb21 # sh ghettoVCB4i.sh test
Taking backup snapshot for ANTHEAPRO ...
DiskLib_Check() failed for source disk The file specified is not a virtual disk (15).
Destination disk format: VMFS zeroedthick
Cloning disk '/vmfs/volumes/datastore1/antheapro.GruppoAnthea.local/antheapro.GruppoAnthea.local.vmdk'...
Clone: 100% done.
Removing snapshot from ANTHEAPRO ...
sh: /vmfs/volumes/VMBACKUP/ESX15/ANTHEAPRO/ANTHEAPRO-2009-06-27: bad number
ghettoVCB4i.sh: line 349: syntax error: /vmfs/volumes/VMBACKUP/ESX15/ANTHEAPRO/ANTHEAPRO-2009-06-27+1
/vmfs/volumes/48d3c7c5-9c6dddaa-59ff-001ec9b5cb21 #

Jun 30, 2009 9:48 PM lamw

says: in response to: Shawn-H

I can't see anything out of the ordinary, just make sure you don't have any "semi-colon" in your configuration file and that you're abiding by the "ESX/ESXi Version Support Table" on this page.

Jun 30, 2009 9:50 PM lamw

says: in response to: Koldijk

Looks like it's complaining about a param that's not correct or unable to parse, what version of ESX(i) and VIMA/vMA are you using? I would suggest keeping the two consistent between the old VI 3.5 and vSphere 4.0

Also make sure you don't have any "semi-colon" in your configuration file

Jun 30, 2009 9:51 PM lamw

says: in response to: freez267

Not sure why you're getting this issue, are you using the latest ghettoVCB4i.sh script? Is it just failing for this single VM?

Jul 1, 2009 1:47 AM freez267 says: in response to: lamw

Yes I'm using the last ghettoVCB4i.sh script.
The script is used to backup 20 vms and this is the only one that has some problems.

Jul 3, 2009 12:02 AM angelone says:

Hi William,
I still have problems here.

I have 2 ESXi 3.5u3 with one VMA4 each.

And on both systems the script can't find the snapshot it created prior to copying the disks.

This is what I get every night on both systems:

============================== 07-03-2009 02:00:01 -- ghettoVCBg2 LOG START ==============================

07-03-2009 02:00:04 -- CONFIG - USING CONFIGURATION FILE = stobae03
07-03-2009 02:00:04 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
07-03-2009 02:00:04 -- CONFIG - VM_BACKUP_DATASTORE = nas_9_backup
07-03-2009 02:00:04 -- CONFIG - VM_BACKUP_DIRECTORY = sicherung
07-03-2009 02:00:04 -- CONFIG - DISK_BACKUP_FORMAT = thin
07-03-2009 02:00:04 -- CONFIG - ADAPTER_FORMAT = lsilogic
07-03-2009 02:00:04 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-03-2009 02:00:04 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-03-2009 02:00:04 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-03-2009 02:00:04 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-03
07-03-2009 02:00:04 -- Initiate backup for "stobae03" found on 5.20.33.17!
07-03-2009 02:00:18 -- stobae03 original powerState: poweredOn
07-03-2009 02:00:20 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-03" for stobae03
07-03-2009 02:00:20 -- stobae03 has 2 VMDK(s)
07-03-2009 02:00:20 -- Backing up "datastore1 stobae03/stobae03.vmdk" to "nas_9_backup sicherung/stobae03/stobae03-2009-07-03/stobae03.vmdk"
07-03-2009 02:05:01 -- Backup completed for datastore1 stobae03/stobae03.vmdk
07-03-2009 02:05:01 -- Backing up "datastore1 stobae03/stobae03_1.vmdk" to "nas_9_backup sicherung/stobae03/stobae03-2009-07-03/stobae03_1.vmdk"
07-03-2009 04:52:35 -- Backup completed for datastore1 stobae03/stobae03_1.vmdk
07-03-2009 04:52:35 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-03" not found
07-03-2009 04:52:42 -- Backup completed for "stobae03"!

============================== 07-03-2009 04:52:42 -- ghettoVCBg2 LOG END ==============================

The snapshot is still there and works.
I then have to shut down the machine and then delete all snapshots in the virtualcenter snapshotmanager.
"removeallsnapshots" somehow doesn't work from console.

Another interesting point is, that after errors like this one, all other VMs in my vmlist won't be backed up.
I have 5 vms in my vmlist.
The first is backed up without problems, the second might have an error like the one above and the other three will not be found later on.

If the first backup has an error the following four VMs won't be found.

The problem lies imho somewhere in the VMA or the script, I can put the last one of my VMs on top of the vmlist and it will be found and backed up.

The hostd.log on the ESXi says the following:
http://www.myangelone.de/files/hostd.log-esxi.txt (the ESXi is 2h behind in time, sry)

And has anyone seen this error before?
This pops up on my ESXi every 20min all day long.
Regardless if it's used or not.
Even if no vm is running and no Infrastructure client connected.
I get this on both ESXi, too.

~~2009-07-03 05:40:32.721 'App' 147466 error~~ Failed to send response to the client: Broken pipe
~~2009-07-03 06:00:33.552 'Vmomi' 65541 info~~ Activation N5Vmomi10ActivationE:0x18e03e60 : Invoke done waitForUpdates on ~~vmodl.query.PropertyCollector:ha-property-collector~~
~~2009-07-03 06:00:33.552 'Vmomi' 65541 info~~ Throw vmodl.fault.RequestCanceled
~~2009-07-03 06:00:33.552 'Vmomi' 65541 info~~ Result:
(vmodl.fault.RequestCanceled) {
dynamicType = <unset>,
msg = ""
}
~~2009-07-03 06:00:33.557 'App' 65541 error~~ Failed to send response to the client: Broken pipe
~~2009-07-03 06:20:34.383 'Vmomi' 131081 info~~ Activation N5Vmomi10ActivationE:0x18702a40 : Invoke done waitForUpdates on ~~vmodl.query.PropertyCollector:ha-property-collector~~
~~2009-07-03 06:20:34.384 'Vmomi' 131081 info~~ Throw vmodl.fault.RequestCanceled
~~2009-07-03 06:20:34.384 'Vmomi' 131081 info~~ Result:
(vmodl.fault.RequestCanceled) {
dynamicType = <unset>,
msg = ""
}
~~2009-07-03 06:20:34.385 'App' 131081 error~~ Failed to send response to the client: Broken pipe

Any help is appreciated, maybe I should switch to the old script.

Is there a way to modify the script, so that it deletes all snapshots after the copy process?
Maybe the problem is the named snapshot.

Jul 3, 2009 6:32 PM grantdavies says: in response to: angelone

Great job on the script, I've got it backing up 3 Virtual Machine guests to a Windows XP 64 bit target.

One of the VMs works perfectly as expected, but two of them fail to close the Snapshots and move on to the next virtual machine.

I've checked the names and the scripts (and the logs) and the script tries to close snapshots with exactly the same name as it opens, but for some reason fails. Any ideas?

I think this has to do with the NFS mount (OpenFiler 2.3) that both ZEROEDTHICK and EAGERZEROEDTHICKas the disk type result in 'General fault caused by file. One of the parameters supplied is invalid' in the VI client.

When using disk type 'THIN' the backup completes, but the snapshot if left open and the log says it failed to find the snap - even though it's exactly the same name of the snap it created!

Jul 9, 2009 5:05 PM coobic says: in response to: angelone

Hi, angelone and William.
We have the same problem with 'angelone' with backup more than 2 vm`s (ESXi 4, vMA 4).
We try to separate backups for Unix and Windows machines. Backup for Unix (Linux and FreeBSD) machines working perfectlty (3 vm`s), but for Windows machines backup work only for one, that located on top of backup list. Log:
============================== 07-10-2009 08:57:50 -- ghettoVCBg2 LOG START ==============================
07-10-2009 08:57:50 -- CONFIG - BACKUP_LOG_OUTPUT = /var/log/ghettoVCBg2.log
07-10-2009 08:57:50 -- CONFIG - VM_BACKUP_DATASTORE = Backup
07-10-2009 08:57:50 -- CONFIG - VM_BACKUP_DIRECTORY = BACKUPS
07-10-2009 08:57:50 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
07-10-2009 08:57:50 -- CONFIG - ADAPTER_FORMAT = lsilogic
07-10-2009 08:57:50 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-10-2009 08:57:50 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-10-2009 08:57:50 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-10-2009 08:57:50 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-10

07-10-2009 08:57:54 -- Initiate backup for "maildat.xxx" found on c3000-box1-vmware2.xxx!
07-10-2009 08:58:03 -- maildat.xxx original powerState: poweredOn
07-10-2009 08:58:05 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-10" for maildat.xxx
07-10-2009 08:58:05 -- maildat.xxx has 1 VMDK(s)
07-10-2009 08:58:05 -- Backing up "Array10-1 maildat.xxx/maildat.xxx.vmdk" to "Backup BACKUPS/maildat.xxx/maildat.xxx-2009-07-10/maildat.xxx.vmdk"
07-10-2009 09:13:56 -- Backup completed for Array10-1 maildat.xxx/maildat.xxx.vmdk
07-10-2009 09:13:59 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-07-10" for maildat.xxx
07-10-2009 09:14:30 -- Backup completed for "maildat.xxx"!
07-10-2009 09:14:33 -- ERROR - Unable to locate VM: FC.xxx
07-10-2009 09:14:33 -- ERROR - Unable to locate VM: R2D2.xxx
07-10-2009 09:14:33 -- ERROR - Unable to locate VM: ts.xxx
07-10-2009 09:14:33 -- ERROR - Unable to locate VM: n2000-bms.xxx

Sometimes backup completed successfully, but error for 'top' machine persist:
07-09-2009 12:31:29 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-09" not found
And we can delete snapshot with Snapshot Manager 'by hand'.

I can e-mail verbose logging for this situations.

Thanks for future help.

Jul 10, 2009 3:04 AM angelone says: in response to: coobic

now, does anyone know why removing all snapshots with the GUI manager works and "removeallsnapshots" from CLI tells me that there are no snapshots?

Jul 10, 2009 6:55 AM lamw

says:

Sorry for the delay of response but I've been extremely busy over the last month and I currently do not have the cycles to look into the current issues. I'm aware there are some interesting problems being faced that may all be related. I will provide some more details when I get a chance to take a look at the issue hopefully in a few weeks after my VCP4 exam. Thanks for your patience

Jul 15, 2009 3:34 PM coobic says: in response to: angelone

http://kb.vmware.com/kb/1012383

Jul 15, 2009 11:27 PM angelone says: in response to: coobic

Hm, I don't think this is the problem here since ghettoVCB doesn't set these flags.

Our problem is that the snapshots aren't found and therefore can't be deleted.
Above KB article says, that you can't delete your snapshots because the left over flag phobits it.

Jul 18, 2009 4:30 PM cookieme says: in response to: angelone

I've read through this thread and I am really interested in using this script to backup my VMs. I am still learning how to use ESXi 4, so this might be a repeat question.

Can I use this script on my ESXi 4 host if I am using the "free" version? I saw something about it not working if you have not paid for a license due to some CLI thingy missing. If this is the case, then what to do for backup?

Thanks

Jul 18, 2009 4:32 PM lamw

says: in response to: cookieme

Please re-read the document/requirements, specifically the section labeled "ESX/ESXi Version Support Table".

Jul 18, 2009 4:33 PM lamw

says: in response to: angelone

Correct, that has nothing to do with the issues being seen now. The KB relates to using vDR (VMware Data Recovery) where snapshots are not being removed and there has been a fix for that which is NOT related to this script at all, unless you're using vDR.

Jul 20, 2009 7:02 AM lamw

says:

For those that are running into issues where backups are occurring but snapshots are not being removed or other issues that are similarly related, please take a look here http://tinyurl.com/layekn

The drawing starts today and you have ~1week to enter. Thanks everyone

Jul 22, 2009 6:14 AM chris647 says:

Is there any way to prevent the script from backing up all vmdks attached to the VM?
I just want to backup the primary vmdk, not vmdks across multiple datastores.
Right now when I backup a VM, the primary vmdk processes fine, but when it hits the secondardy vmdk on another datastore, it gives an error that the file is locked or in use.

Thanks in advance!

Jul 22, 2009 6:53 AM lamw

says: in response to: chris647

No, that is not a supported feature of the script. You're more than welcome to add logic to exclude certain VMDK(s), no support will be provided for this.

The script should work if you have multiple VMDK(s) across multiple datastores. Is the script taking snapshots prior to backup? Please provide more details and take a look at FAQ #10

Jul 22, 2009 6:59 AM chris647 says: in response to: lamw

Yes I am doing snapshots prior to the backup.
The second disk is configured as an Independant Disk which is why snapshots will not run, I assume.
I was hoping by using it as an Independant disk, the backup would skip it.

Jul 22, 2009 7:03 AM lamw

says: in response to: chris647

Bingo, Independant Disk can not be snapshotted hence the error that its locked since w/o the snapshot and I assume the VM was powered on, it would be locked for write access.

Aug 3, 2009 1:59 AM Al3xandr3 says:

Hello,

I'm trying to use the script on two similar configuration: ESXi 3.5u3 free licensied and vMA 4.0

I got always same problems.

Snapshot is not removed.
VMs turned off are not turned on
The backup rotation isn't working either.

============================== 07-29-2009 22:01:01 -- ghettoVCBg2 LOG START ==============================

07-29-2009 22:01:04 -- CONFIG - USING CONFIGURATION FILE = SRVLINUX
07-29-2009 22:01:04 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
07-29-2009 22:01:04 -- CONFIG - VM_BACKUP_DATASTORE = NFS
07-29-2009 22:01:04 -- CONFIG - VM_BACKUP_DIRECTORY = DAILY
07-29-2009 22:01:04 -- CONFIG - DISK_BACKUP_FORMAT = thin
07-29-2009 22:01:04 -- CONFIG - ADAPTER_FORMAT = buslogic
07-29-2009 22:01:04 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-29-2009 22:01:04 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-29-2009 22:01:04 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-29-2009 22:01:04 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-29
07-29-2009 22:01:04 -- Initiate backup for "SRVLINUX" found on 192.168.0.110!
07-29-2009 22:01:15 -- SRVLINUX original powerState: poweredOn
07-29-2009 22:01:17 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-29" for SRVLINUX
07-29-2009 22:01:17 -- SRVLINUX has 1 VMDK(s)
07-29-2009 22:01:17 -- Backing up "datastore1 SRVLINUX/SRVLINUX_2.vmdk" to "NFS DAILY/SRVLINUX/SRVLINUX-2009-07-29/SRVLINUX_2.vmdk"
07-29-2009 22:33:21 -- Backup completed for datastore1 SRVLINUX/SRVLINUX_2.vmdk
07-29-2009 22:33:21 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-29" not found
07-29-2009 22:33:45 -- Unable to purge "NFS DAILY/SRVLINUX/SRVLINUX-2009-07-21--5" due to rotation max
07-29-2009 22:33:49 -- Backup completed for "SRVLINUX"!

============================== 07-29-2009 22:33:49 -- ghettoVCBg2 LOG END ==============================

============================== 07-29-2009 22:34:00 -- ghettoVCBg2 LOG START ==============================

07-29-2009 22:34:03 -- CONFIG - USING CONFIGURATION FILE = SRVHP
07-29-2009 22:34:03 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
07-29-2009 22:34:03 -- CONFIG - VM_BACKUP_DATASTORE = NFS
07-29-2009 22:34:03 -- CONFIG - VM_BACKUP_DIRECTORY = DAILY
07-29-2009 22:34:03 -- CONFIG - DISK_BACKUP_FORMAT = thin
07-29-2009 22:34:03 -- CONFIG - ADAPTER_FORMAT = lsilogic
07-29-2009 22:34:03 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-29-2009 22:34:03 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
07-29-2009 22:34:03 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
07-29-2009 22:34:03 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-29
07-29-2009 22:34:03 -- Initiate backup for "SRVHP" found on 192.168.0.110!
07-29-2009 22:34:14 -- SRVHP original powerState: poweredOn
07-29-2009 22:36:05 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-29" for SRVHP
07-29-2009 22:36:05 -- SRVHP has 1 VMDK(s)
07-29-2009 22:36:05 -- Backing up "datastore1 SRVHP/SRVHP.vmdk" to "NFS DAILY/SRVHP/SRVHP-2009-07-29/SRVHP.vmdk"
07-30-2009 01:39:17 -- Backup completed for datastore1 SRVHP/SRVHP.vmdk
07-30-2009 01:39:17 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-29" not found
07-30-2009 01:39:40 -- Unable to purge "NFS DAILY/SRVHP/SRVHP-2009-07-21--5" due to rotation max
07-30-2009 01:39:40 -- Backup completed for "SRVHP"!

============================== 07-30-2009 01:39:40 -- ghettoVCBg2 LOG END ==============================

============================== 07-30-2009 01:39:54 -- ghettoVCBg2 LOG START ==============================

07-30-2009 01:39:57 -- CONFIG - USING CONFIGURATION FILE = SRVAPPLI
07-30-2009 01:39:57 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
07-30-2009 01:39:57 -- CONFIG - VM_BACKUP_DATASTORE = NFS
07-30-2009 01:39:57 -- CONFIG - VM_BACKUP_DIRECTORY = DAILY
07-30-2009 01:39:57 -- CONFIG - DISK_BACKUP_FORMAT = thin
07-30-2009 01:39:57 -- CONFIG - ADAPTER_FORMAT = lsilogic
07-30-2009 01:39:57 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-30-2009 01:39:57 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
07-30-2009 01:39:57 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
07-30-2009 01:39:57 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-30
07-30-2009 01:39:57 -- Initiate backup for "SRVAPPLI" found on 192.168.0.110!
07-30-2009 01:40:08 -- SRVAPPLI original powerState: poweredOn
07-30-2009 01:41:44 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-30" for SRVAPPLI
07-30-2009 01:41:44 -- SRVAPPLI has 1 VMDK(s)
07-30-2009 01:41:44 -- Backing up "datastore1 SRVAPPLI/SRVAPPLI.vmdk" to "NFS DAILY/SRVAPPLI/SRVAPPLI-2009-07-30/SRVAPPLI.vmdk"
07-30-2009 05:10:32 -- Backup completed for datastore1 SRVAPPLI/SRVAPPLI.vmdk
07-30-2009 05:10:32 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-30" not found
07-30-2009 05:10:55 -- Unable to purge "NFS DAILY/SRVAPPLI/SRVAPPLI-2009-07-21--5" due to rotation max
07-30-2009 05:10:59 -- Backup completed for "SRVAPPLI"!

============================== 07-30-2009 05:10:59 -- ghettoVCBg2 LOG END ==============================

Aug 3, 2009 8:48 PM lamw

says: in response to: Al3xandr3

Curious, have you tried to backup the 3 VMs but instead of using the configuration file just a flat file containing the VMs that need to be backed up. I know few others have ran into similar situations and so far everyone has been using the configurations to execute their backups. Just wondering if there is a bug in the configuration backup method or if this is a general issue.

Aug 4, 2009 12:25 AM Al3xandr3 says: in response to: lamw

I made a test on saturday on a similar architecture.

I launched manually the script for each VM (so I modified each time vmlist file) without configuration file.I didn't try the backup rotation and power off VM.

All is ok for the first VM but for the two others, snapshot still not found.

It's really strange because, like i told you, i launched each backup process independently.

============================== 08-01-2009 16:19:48 -- ghettoVCBg2 LOG START ==============================
08-01-2009 16:19:48 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-01-2009 16:19:48 -- CONFIG - VM_BACKUP_DATASTORE = NFS
08-01-2009 16:19:48 -- CONFIG - VM_BACKUP_DIRECTORY = WEEKLY
08-01-2009 16:19:48 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-01-2009 16:19:48 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-01-2009 16:19:48 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-01-2009 16:19:48 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 16:19:48 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 16:19:48 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-01

08-01-2009 16:19:51 -- Initiate backup for "SRVAD" found on 192.128.10.250!
08-01-2009 16:20:01 -- SRVAD original powerState: poweredOn
08-01-2009 16:28:24 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-01" for SRVAD
08-01-2009 16:28:24 -- SRVAD has 1 VMDK(s)
08-01-2009 16:28:24 -- Backing up "datastore1 SRVAD/SRVAD.vmdk" to "NFS WEEKLY/SRVAD/SRVAD-2009-08-01/SRVAD.vmdk"
08-01-2009 16:55:08 -- Backup completed for datastore1 SRVAD/SRVAD.vmdk
08-01-2009 16:55:47 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-08-01" for SRVAD
08-01-2009 16:55:53 -- Backup completed for "SRVAD"!

============================== 08-01-2009 16:55:53 -- ghettoVCBg2 LOG END ==============================

============================== 08-01-2009 17:08:40 -- ghettoVCBg2 LOG START ==============================
08-01-2009 17:08:40 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-01-2009 17:08:40 -- CONFIG - VM_BACKUP_DATASTORE = NFS
08-01-2009 17:08:40 -- CONFIG - VM_BACKUP_DIRECTORY = WEEKLY
08-01-2009 17:08:40 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-01-2009 17:08:40 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-01-2009 17:08:40 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-01-2009 17:08:40 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 17:08:40 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 17:08:40 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-01

08-01-2009 17:08:43 -- Initiate backup for "SRVEXCHANGE" found on 192.128.10.250!
08-01-2009 17:08:52 -- SRVEXCHANGE original powerState: poweredOn
08-01-2009 17:11:37 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-01" for SRVEXCHANGE
08-01-2009 17:11:37 -- SRVEXCHANGE has 1 VMDK(s)
08-01-2009 17:11:37 -- Backing up "datastore1 SRVEXCHANGE/SRVEXCHANGE.vmdk" to "NFS WEEKLY/SRVEXCHANGE/SRVEXCHANGE-2009-08-01/SRVEXCHANGE.vmdk"
08-01-2009 19:15:12 -- Backup completed for datastore1 SRVEXCHANGE/SRVEXCHANGE.vmdk
08-01-2009 19:15:13 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-01" not found
08-01-2009 19:15:18 -- Backup completed for "SRVEXCHANGE"!

============================== 08-01-2009 19:15:18 -- ghettoVCBg2 LOG END ==============================

============================== 08-01-2009 19:20:59 -- ghettoVCBg2 LOG START ==============================
08-01-2009 19:20:59 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-01-2009 19:20:59 -- CONFIG - VM_BACKUP_DATASTORE = NFS
08-01-2009 19:20:59 -- CONFIG - VM_BACKUP_DIRECTORY = WEEKLY
08-01-2009 19:20:59 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-01-2009 19:20:59 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-01-2009 19:20:59 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-01-2009 19:20:59 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 19:20:59 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 19:20:59 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-01

08-01-2009 19:21:02 -- Initiate backup for "SRVTSE" found on 192.128.10.250!
08-01-2009 19:21:11 -- SRVTSE original powerState: poweredOn
08-01-2009 19:45:45 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-01" for SRVTSE
08-01-2009 19:45:45 -- SRVTSE has 2 VMDK(s)
08-01-2009 19:45:45 -- Backing up "datastore1 SRVTSE/SRVTSE.vmdk" to "NFS WEEKLY/SRVTSE/SRVTSE-2009-08-01/SRVTSE.vmdk"
08-01-2009 20:53:10 -- Backup completed for datastore1 SRVTSE/SRVTSE.vmdk
08-01-2009 20:53:10 -- Backing up "datastore1 SRVTSE/SRVTSE_1.vmdk" to "NFS WEEKLY/SRVTSE/SRVTSE-2009-08-01/SRVTSE_1.vmdk"
08-01-2009 22:48:32 -- Backup completed for datastore1 SRVTSE/SRVTSE_1.vmdk
08-01-2009 22:48:32 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-01" not found
08-01-2009 22:48:38 -- Backup completed for "SRVTSE"!

============================== 08-01-2009 22:48:38 -- ghettoVCBg2 LOG END ==============================

Aug 4, 2009 6:49 AM lamw

says: in response to: Al3xandr3

Can you do the same test but with all 3 VMs on 1 backup vmlist and post your results.

Aug 10, 2009 12:51 AM Al3xandr3 says: in response to: lamw

Hello,

i did the same test with 3 VMs on 1 backup vmlist. I post the result.

============================== 08-08-2009 11:28:24 -- ghettoVCBg2 LOG START ==============================
08-08-2009 11:28:24 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-08-2009 11:28:24 -- CONFIG - VM_BACKUP_DATASTORE = NFS
08-08-2009 11:28:24 -- CONFIG - VM_BACKUP_DIRECTORY = WEEKLY
08-08-2009 11:28:24 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-08-2009 11:28:24 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-08-2009 11:28:24 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-08-2009 11:28:24 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-08-2009 11:28:24 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-08-2009 11:28:24 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-08

08-08-2009 11:28:28 -- Initiate backup for "SRVAD" found on 192.128.10.250!
08-08-2009 11:28:38 -- SRVAD original powerState: poweredOn
08-08-2009 11:36:17 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-08" for SRVAD
08-08-2009 11:36:17 -- SRVAD has 1 VMDK(s)
08-08-2009 11:36:17 -- Backing up "datastore1 SRVAD/SRVAD.vmdk" to "NFS WEEKLY/SRVAD/SRVAD-2009-08-08/SRVAD.vmdk"
08-08-2009 12:04:06 -- Backup completed for datastore1 SRVAD/SRVAD.vmdk
08-08-2009 12:04:41 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-08-08" for SRVAD
08-08-2009 12:04:50 -- Backup completed for "SRVAD"!

08-08-2009 12:04:50 -- Initiate backup for "SRVTSE" found on 192.128.10.250!
08-08-2009 12:05:00 -- SRVTSE original powerState: poweredOn
08-08-2009 12:13:45 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-08" for SRVTSE
08-08-2009 12:13:45 -- SRVTSE has 2 VMDK(s)
08-08-2009 12:13:45 -- Backing up "datastore1 SRVTSE/SRVTSE.vmdk" to "NFS WEEKLY/SRVTSE/SRVTSE-2009-08-08/SRVTSE.vmdk"
08-08-2009 12:55:26 -- Backup completed for datastore1 SRVTSE/SRVTSE.vmdk
08-08-2009 12:55:26 -- Backing up "datastore1 SRVTSE/SRVTSE_1.vmdk" to "NFS WEEKLY/SRVTSE/SRVTSE-2009-08-08/SRVTSE_1.vmdk"
08-08-2009 14:39:29 -- Backup completed for datastore1 SRVTSE/SRVTSE_1.vmdk
08-08-2009 14:39:30 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-08" not found
08-08-2009 14:39:39 -- Backup completed for "SRVTSE"!

08-08-2009 14:39:39 -- ERROR - Unable to locate VM: SRVEXCHANGE

============================== 08-08-2009 14:39:39 -- ghettoVCBg2 LOG END ==============================

1st: OK.
2nd: Snapshot not removed.
3rd: VM not found.

Aug 11, 2009 4:56 PM adrian888 says: in response to: lamw

Could this be a time stamp issue?

I was backing up my domain controller, and it usually only takes 2-3 minutes with no issues. I was testing a new NFS share and it took almost an hour, and had the error that it could not find the snap shot. I usually only have snap shot issues with my larger VM's that take at least an hour or more to backup.

Aug 12, 2009 2:59 PM dwhittaker says: in response to: Al3xandr3

I'm just lending my voice to state that I have experienced the same phenomenon as Al3xandr3. I haven't yet tried to determine if the order of running individual backups affects the result or if it's a specific fault of one of the VM's. I have experienced it with multiple VMs in one list and also individually listed VMs with manual starts. With all of my VM's in a single list, in order:

(1st) Win2k TS (not the assigned names, there are no spaces): OK
(2nd) Win2k3 App: Snapshot not removed. Once I found that the snapshot not only wasn't removed but also didn't show up in the VI Client to be deleted and required manual creation of a snapshot to then show 2 snapshots and deleting them then.
(etc) The rest: Backup doesn't begin

When I find the logs I'll review and post them up, hopefully correlating the same errors.

Environment is vSphere Essentials (1 ESX, 1 vCentre VM, vMA4. My VMs ranged from 15GB to 45GB at the time and backup time was not excessive. Brand new setup, overprovisioned atm and plenty of resources and speed. No activity during the time of backups which are stored on an iSCSI NAS.

Aug 14, 2009 2:22 PM grrltech says:

Are there any plans to add compression to this version of ghettoVCB like there is in generation 1? Compression is key to our off-site backup regimen.

Aug 14, 2009 7:22 PM lamw

says: in response to: grrltech

ghettoVCB compression is currently at an experimental support, no plans for g2 yet. There are still some outstanding issues that I've yet had the cycles to investigate for ghettoVCBg2 and I'm hoping I'll have some time before VMworld else it'll have to be after.

Aug 15, 2009 7:32 AM dwhittaker says: in response to: dwhittaker

Seems I am getting the same errors in the logs as Al3xandr3.

Backup log:

============================== 08-15-2009 05:27:09 -- ghettoVCBg2 LOG START ==============================
08-15-2009 05:27:09 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-15-2009 05:27:09 -- CONFIG - VM_BACKUP_DATASTORE = Thecus_iSCSI_BkUp
08-15-2009 05:27:09 -- CONFIG - VM_BACKUP_DIRECTORY = vMA_Backups
08-15-2009 05:27:09 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-15-2009 05:27:09 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-15-2009 05:27:09 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-15-2009 05:27:09 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-15-2009 05:27:09 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-15-2009 05:27:09 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-15

08-15-2009 05:27:13 -- Initiate backup for "CapServer_New" found on myESX.domain.dc!
08-15-2009 05:27:24 -- CapServer_New original powerState: poweredOn
08-15-2009 05:28:14 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-15" for CapServer_New
08-15-2009 05:28:14 -- CapServer_New has 1 VMDK(s)
08-15-2009 05:28:14 -- Backing up "COS_Datastore CapServer_New/CapServer_New.vmdk" to "Thecus_iSCSI_BkUp vMA_Backups/CapServer_New/CapServer_New-2009-08-15/CapServer_New.vmdk"
08-15-2009 05:53:48 -- Backup completed for COS_Datastore CapServer_New/CapServer_New.vmdk
08-15-2009 05:53:50 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-08-15" for CapServer_New
08-15-2009 05:54:00 -- Backup completed for "CapServer_New"!

08-15-2009 05:54:01 -- Initiate backup for "BrisSteelADS01" found on myESX.domain.dc!
08-15-2009 05:54:11 -- BrisSteelADS01 original powerState: poweredOn
08-15-2009 05:58:10 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-15" for BrisSteelADS01
08-15-2009 05:58:10 -- BrisSteelADS01 has 2 VMDK(s)
08-15-2009 05:58:10 -- Backing up "COS_Datastore BrisSteelADS01/BrisSteelADS01.vmdk" to "Thecus_iSCSI_BkUp vMA_Backups/BrisSteelADS01/BrisSteelADS01-2009-08-15/BrisSteelADS01.vmdk"
08-15-2009 06:14:03 -- Backup completed for COS_Datastore BrisSteelADS01/BrisSteelADS01.vmdk
08-15-2009 06:14:03 -- Backing up "COS_Datastore BrisSteelADS01/BrisSteelADS01_2.vmdk" to "Thecus_iSCSI_BkUp vMA_Backups/BrisSteelADS01/BrisSteelADS01-2009-08-15/BrisSteelADS01_2.vmdk"
08-15-2009 06:31:22 -- Backup completed for COS_Datastore BrisSteelADS01/BrisSteelADS01_2.vmdk
08-15-2009 06:31:22 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-15" not found
08-15-2009 06:31:37 -- Backup completed for "BrisSteelADS01"!

08-15-2009 06:31:37 -- ERROR - Unable to locate VM: CapServer2_New

============================== 08-15-2009 06:31:37 -- ghettoVCBg2 LOG END ==============================

.
.
.
.
.
.
Debug log (after the fact, snapshot was not there prior to backup above):

============================== 08-15-2009 07:25:08 -- ghettoVCBg2 LOG START ==============================
08-15-2009 07:25:08 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-15-2009 07:25:08 -- CONFIG - VM_BACKUP_DATASTORE = Thecus_iSCSI_BkUp
08-15-2009 07:25:08 -- CONFIG - VM_BACKUP_DIRECTORY = vMA_Backups
08-15-2009 07:25:08 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-15-2009 07:25:08 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-15-2009 07:25:08 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-15-2009 07:25:08 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-15-2009 07:25:08 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-15-2009 07:25:08 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-15

08-15-2009 07:25:12 --

DEBUG INFO CapServer_New

08-15-2009 07:25:12 -- DEBUG - Host Build: VMware ESX 4.0.0 build-171294
08-15-2009 07:25:12 -- DEBUG - Host: myESX.domain.dc
08-15-2009 07:25:12 -- DEBUG - Virtual Machine: CapServer_New
08-15-2009 07:25:12 -- DEBUG - VM ConfigPath: COS_Datastore CapServer_New/CapServer_New.vmx
08-15-2009 07:25:12 -- DEBUG - VMX File: CapServer_New.vmx
08-15-2009 07:25:12 -- DEBUG - BackupConfigPath: Thecus_iSCSI_BkUp vMA_Backups/CapServer_New/CapServer_New-2009-08-15/CapServer_New.vmx
08-15-2009 07:25:12 -- DEBUG - BackupPath: Thecus_iSCSI_BkUp vMA_Backups/CapServer_New/CapServer_New-2009-08-15
08-15-2009 07:25:12 -- DEBUG - VM Datastore: COS_Datastore
08-15-2009 07:25:12 -- DEBUG - VMDK(s):
08-15-2009 07:25:12 -- DEBUG - COS_Datastore CapServer_New/CapServer_New.vmdk
08-15-2009 07:25:12 --

DEBUG INFO CapServer_New

08-15-2009 07:25:12 -- WARN - Snapshot found for BrisSteelADS01, backup will not take place

08-15-2009 07:25:12 --

DEBUG INFO CapServer2_New

08-15-2009 07:25:12 -- DEBUG - Host Build: VMware ESX 4.0.0 build-171294
08-15-2009 07:25:12 -- DEBUG - Host: myESX.domain.dc
08-15-2009 07:25:12 -- DEBUG - Virtual Machine: CapServer2_New
08-15-2009 07:25:12 -- DEBUG - VM ConfigPath: COS_Datastore CapServer2_New/CapServer2_New.vmx
08-15-2009 07:25:12 -- DEBUG - VMX File: CapServer2_New.vmx
08-15-2009 07:25:12 -- DEBUG - BackupConfigPath: Thecus_iSCSI_BkUp vMA_Backups/CapServer2_New/CapServer2_New-2009-08-15/CapServer2_New.vmx
08-15-2009 07:25:12 -- DEBUG - BackupPath: Thecus_iSCSI_BkUp vMA_Backups/CapServer2_New/CapServer2_New-2009-08-15
08-15-2009 07:25:12 -- DEBUG - VM Datastore: COS_Datastore
08-15-2009 07:25:12 -- DEBUG - VMDK(s):
08-15-2009 07:25:12 -- DEBUG - COS_Datastore CapServer2_New/CapServer2_New.vmdk
08-15-2009 07:25:12 --

DEBUG INFO CapServer2_New

============================== 08-15-2009 07:25:12 -- ghettoVCBg2 LOG END ==============================

Aug 15, 2009 10:45 AM yp1 says:

I get "Hard power off, VMware Tools is not installed" when try to use power off before backup feature.
The tools are there and VIC says "OK" for the tools status.
Has the feature ever been tested?
Another bug?
I thought to use the power off option it as a workaround since I had never succeeded backing up "hot" due to a known bug with non-deleted snapshots, but it does not seem to work properly either.

Aug 16, 2009 12:40 PM lamw

says: in response to: yp1

This feature has been tested and has worked. Please provide the following pieces of information on your environment:

1) What version of ESX(i)?
2) Are you using vCenter? What version? Are you using vMotion?
3) What version of VIMA/vMA are you running?
4) Please execute the following script to verify your VMware Tools status on your VMs: getVMToolsStatus.pl

Aug 17, 2009 12:02 AM datenfalke says:

Hello,

I would like to use this nice script but I also have the problem that the first VM gets backuped, the snapshot does not get deleted and the following VMs cannot be found.

I am using VIMA 1.0.

Here are the logfiles:

============================== 08-16-2009 15:37:44 -- ghettoVCBg2 LOG START ==============================
08-16-2009 15:37:44 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-16-2009 15:37:44 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
08-16-2009 15:37:44 -- CONFIG - VM_BACKUP_DIRECTORY = VM_BACKUPS
08-16-2009 15:37:44 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
08-16-2009 15:37:44 -- CONFIG - ADAPTER_FORMAT = buslogic
08-16-2009 15:37:44 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-16-2009 15:37:44 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
08-16-2009 15:37:44 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
08-16-2009 15:37:44 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-16

08-16-2009 15:37:50 --

DEBUG INFO vps006

08-16-2009 15:37:50 -- DEBUG - Host Build: VMware ESX Server 3i 3.5.0 build-123629
08-16-2009 15:37:50 -- DEBUG - Host: xx.xx.xx.xx
08-16-2009 15:37:50 -- DEBUG - Virtual Machine: vps006
08-16-2009 15:37:50 -- DEBUG - VM ConfigPath: datastore1 vps006/vps006.vmx
08-16-2009 15:37:50 -- DEBUG - VMX File: vps006.vmx
08-16-2009 15:37:50 -- DEBUG - BackupConfigPath: datastore1 VM_BACKUPS/vps006/vps006-2009-08-16/vps006.vmx
08-16-2009 15:37:50 -- DEBUG - BackupPath: datastore1 VM_BACKUPS/vps006/vps006-2009-08-16
08-16-2009 15:37:50 -- DEBUG - VM Datastore: datastore1
08-16-2009 15:37:50 -- DEBUG - VMDK(s):
08-16-2009 15:37:50 -- DEBUG - datastore1 vps006/vps006.vmdk
08-16-2009 15:37:50 --

DEBUG INFO vps006

08-16-2009 15:37:50 --

DEBUG INFO vps003

08-16-2009 15:37:50 -- DEBUG - Host Build: VMware ESX Server 3i 3.5.0 build-123629
08-16-2009 15:37:50 -- DEBUG - Host: xx.xx.xx.xx
08-16-2009 15:37:50 -- DEBUG - Virtual Machine: vps003
08-16-2009 15:37:50 -- DEBUG - VM ConfigPath: datastore1 vps003/vps003.vmx
08-16-2009 15:37:50 -- DEBUG - VMX File: vps003.vmx
08-16-2009 15:37:50 -- DEBUG - BackupConfigPath: datastore1 VM_BACKUPS/vps003/vps003-2009-08-16/vps003.vmx
08-16-2009 15:37:50 -- DEBUG - BackupPath: datastore1 VM_BACKUPS/vps003/vps003-2009-08-16
08-16-2009 15:37:50 -- DEBUG - VM Datastore: datastore1
08-16-2009 15:37:50 -- DEBUG - VMDK(s):
08-16-2009 15:37:50 -- DEBUG - datastore1 vps003/vps003.vmdk
08-16-2009 15:37:50 --

DEBUG INFO vps003

08-16-2009 15:37:50 --

DEBUG INFO vps001

08-16-2009 15:37:50 -- DEBUG - Host Build: VMware ESX Server 3i 3.5.0 build-123629
08-16-2009 15:37:50 -- DEBUG - Host: xx.xx.xx.xx
08-16-2009 15:37:50 -- DEBUG - Virtual Machine: vps001
08-16-2009 15:37:50 -- DEBUG - VM ConfigPath: datastore1 vps001/vps001.vmx
08-16-2009 15:37:50 -- DEBUG - VMX File: vps001.vmx
08-16-2009 15:37:50 -- DEBUG - BackupConfigPath: datastore1 VM_BACKUPS/vps001/vps001-2009-08-16/vps001.vmx
08-16-2009 15:37:50 -- DEBUG - BackupPath: datastore1 VM_BACKUPS/vps001/vps001-2009-08-16
08-16-2009 15:37:50 -- DEBUG - VM Datastore: datastore1
08-16-2009 15:37:50 -- DEBUG - VMDK(s):
08-16-2009 15:37:50 -- DEBUG - datastore1 vps001/vps001.vmdk
08-16-2009 15:37:50 --

DEBUG INFO vps001

============================== 08-16-2009 15:37:50 -- ghettoVCBg2 LOG END ==============================

============================== 08-16-2009 15:41:25 -- ghettoVCBg2 LOG START ==============================
08-16-2009 15:41:25 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-16-2009 15:41:25 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
08-16-2009 15:41:25 -- CONFIG - VM_BACKUP_DIRECTORY = VM_BACKUPS
08-16-2009 15:41:25 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
08-16-2009 15:41:25 -- CONFIG - ADAPTER_FORMAT = buslogic
08-16-2009 15:41:25 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-16-2009 15:41:25 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
08-16-2009 15:41:25 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
08-16-2009 15:41:25 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-16

08-16-2009 15:41:29 -- Initiate backup for "vps006" found on xx.xx.xx.xx!
08-16-2009 15:41:43 -- vps006 original powerState: poweredOn
08-16-2009 15:41:45 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-16" for vps006
08-16-2009 15:41:45 -- vps006 has 1 VMDK(s)
08-16-2009 15:41:45 -- Backing up "datastore1 vps006/vps006.vmdk" to "datastore1 VM_BACKUPS/vps006/vps006-2009-08-16/vps006.vmdk"
08-16-2009 16:26:19 -- Backup completed for datastore1 vps006/vps006.vmdk
08-16-2009 16:26:19 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-16" not found
08-16-2009 16:26:27 -- Backup completed for "vps006"!

08-16-2009 16:26:27 -- ERROR - Unable to locate VM: vps001

08-16-2009 16:26:27 -- ERROR - Unable to locate VM: vps003

============================== 08-16-2009 16:26:27 -- ghettoVCBg2 LOG END ==============================

Aug 20, 2009 7:41 AM adrian888 says: in response to: Al3xandr3

Is this a valid perl statement, as I have not found an example of using it anywhere else.

line 699 - $ref = $subRef if ($subCount);

I have changed it to the following, and it look good so far.

if ($subCount) {
$ref = $subRef;
}

Aug 20, 2009 9:43 AM lamw

says: in response to: adrian888

It's a valid Perl statement, test it out for yourself.

Here's a quick example:

[vi-admin@scofield]$ cat test.pl 
#!/usr/bin/perl -w

use strict;
use warnings;

my $subCount = 1;
my $subRef = "hello world";
my $ref = "not hellow world";

$ref = $subRef if($subCount);

print $ref . "\n";

Run it and you tell me what it prints

Aug 24, 2009 3:17 AM yp1 says: in response to: yp1

I figured the problem with VM not being powered down was in the Tools being out of date, so sorry for the false alarm.
However, now the VM does not being powered back up after the backup. Even though I can see the code is there.
The script just quits after the last VMDK backup by saying "Backup completed", but without the "LOG END".
What could be wrong?

Sep 3, 2009 11:03 AM vipernicus42 says:

Hey William,

First off, thanks so much for making these scripts. Between ghettoShutdown.pl and ghettoVCBg2.pl (if I ever get it working) I'll have everything I need to run ESXi without worry!

That being said... I'm having some issues with getting ghettoVCBg2.pl working

Environment Setup:

vmbox1.domain.com ESXi 3.5u3 host (free license) which has one guest
- "vmguest1.domain.com" - a winXP "workstation"
vmbox2.domain.com ESXi 3.5u3 host (free license) which has one guest
- vMA-vmbox2.domain.com - a vMA 1.0 virtual appliance

This is the same setup I was using with my questions over in the ghettoShutdown.pl thread, and that's working very well now thanks!

Anyway, it -appears- to work, but when it's all done the only thing that appears in the backup folder is a single *.vmx file that's 2.03kb and that's it. The whole process takes no more than two minutes as well, which is quite a short time for something that's 10gb to go from one datastore to another (which are on separate physical hard drives in the same box) Here's a dump of the log

============================== 09-03-2009 09:49:48 -- ghettoVCBg2 LOG START ==============================
09-03-2009 09:49:48 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
09-03-2009 09:49:48 -- CONFIG - VM_BACKUP_DATASTORE = datastore2
09-03-2009 09:49:48 -- CONFIG - VM_BACKUP_DIRECTORY = VMBackups
09-03-2009 09:49:48 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
09-03-2009 09:49:48 -- CONFIG - ADAPTER_FORMAT = buslogic
09-03-2009 09:49:48 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
09-03-2009 09:49:48 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
09-03-2009 09:49:48 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
09-03-2009 09:49:48 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-09-03

09-03-2009 09:50:26 -- Initiate backup for "vmguest1" found on vmbox1.domain.com!
09-03-2009 09:50:45 -- vmguest1 original powerState: poweredOn
09-03-2009 09:50:47 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-09-03" for vmguest1
09-03-2009 09:50:47 -- vmguest1 has 1 VMDK(s)
09-03-2009 09:50:47 -- Backing up "datastore1 vmguest1/vmguest1.vmdk" to "datastore2 VMBackups/vmguest1/vmguest1-2009-09-03/vmguest1.vmdk"
09-03-2009 09:50:53 -- Backup completed for datastore1 vmguest1/vmguest1.vmdk
09-03-2009 09:50:56 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-09-03" for vmguest1
09-03-2009 09:51:06 -- Backup completed for "vmguest1"!

============================== 09-03-2009 09:51:06 -- ghettoVCBg2 LOG END ==============================

As you can see the log looks fine. There's plenty of space on datastore2 (like 100gb and the VM I'm trying to backup is 10GB).

Any ideas why it's not creating/copying anything to the backup folder other than a single tiny *.vmx file?

Jon

Sep 7, 2009 4:07 AM mvdheuvel says:

Hi William,

First of all, works great. Easy to configure and very fast free alternative.
I have a question, is it possible to have the status mailed to me.
Case of a backup error, I like to have a email.

Menno

Sep 7, 2009 5:59 AM martingreiner says: in response to: mvdheuvel

Hi

I will try as described but i am hanging by creating a NFS-datastore in my ESXi/vMA
If I understand correctly the NFS-datastore have to be mounted on vMA (central)
But how to do ?

regards
Martin

Sep 7, 2009 7:05 AM angelone says: in response to: mvdheuvel

Hi Menno,
you could pipe the cron output to mail.

Just append something like

| mail -s "ghettoVCBg2 <your vMA server>" someone@mydomain.com

at the end of the cron entry.

But this way you will get the mail everytime the cronjob runs, not just in case of errors.

Sep 7, 2009 7:08 AM angelone says: in response to: martingreiner

@martin

You have to mount the datastore on your ESXi to backup to it.
You can do this via the vCenter.

The only case in which you have to mount the datastore in your vMA is when you want to store the vmlist files in some central location.

Sep 7, 2009 7:36 AM martingreiner says: in response to: angelone

Following Log, whats going wrong here, NFS_datastore is mounted on esxi03 !

============================== 09-07-2009 07:28:01 -- ghettoVCBg2 LOG START ==============================

09-07-2009 07:28:05 -- CONFIG - USING CONFIGURATION FILE = vMA
09-07-2009 07:28:05 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
09-07-2009 07:28:05 -- CONFIG - VM_BACKUP_DATASTORE = NFS_datastore
09-07-2009 07:28:05 -- CONFIG - VM_BACKUP_DIRECTORY = esxi03_backups
09-07-2009 07:28:05 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
09-07-2009 07:28:05 -- CONFIG - ADAPTER_FORMAT = lsilogic
09-07-2009 07:28:05 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
09-07-2009 07:28:05 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
09-07-2009 07:28:05 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
09-07-2009 07:28:05 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-09-07
09-07-2009 07:28:05 -- Initiate backup for "vMA" found on esxi03.softline.at!
09-07-2009 07:28:14 -- vMA original powerState: poweredOn
09-07-2009 07:28:14 -- vMA has 1 VMDK(s)
09-07-2009 07:28:14 -- Backing up "datastore1 vMA/vMA.vmdk" to "NFS_datastore esxi03_backups/vMA/vMA-2009-09-07/vMA.vmdk"
09-07-2009 07:28:18 -- Backup completed for datastore1 vMA/vMA.vmdk
09-07-2009 07:28:18 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-09-07" not found
09-07-2009 07:28:21 -- Unable to purge "NFS_datastore esxi03_backups/vMA/GET https:esxi03.softline.atfolderesxi03_backupsvMA?dsName=NFS_datastore unsuccessful : 404 Not Found" due to rotation max
09-07-2009 07:28:21 -- Backup completed for "vMA"!

============================== 09-07-2009 07:28:21 -- ghettoVCBg2 LOG END ==============================

Sep 17, 2009 9:29 AM ermasadmin says:

William,
Phenomenal work and effort on this project, and thanks for all your support for the community! I have just executed a few initial trials, and like Shawn and martingreiner I am seeing a "404 Not Found due to rotation max" error. I have vMA and ESXi 3.5 build 184236 along with ghettoVCBg2.pl version 2.5.1. I was wondering if you had an idea what might be causing this error?
(debug output below):

============================== 09-17-2009 08:21:03 -- ghettoVCBg2 LOG START ==============================
09-17-2009 08:21:03 -- CONFIG - BACKUP_LOG_OUTPUT = ghettoVCBg2.log
09-17-2009 08:21:03 -- CONFIG - VM_BACKUP_DATASTORE = DSS1-bkupvm-eth4
09-17-2009 08:21:03 -- CONFIG - VM_BACKUP_DIRECTORY = ghettoVCB
09-17-2009 08:21:03 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
09-17-2009 08:21:03 -- CONFIG - ADAPTER_FORMAT = buslogic
09-17-2009 08:21:03 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
09-17-2009 08:21:03 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
09-17-2009 08:21:03 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
09-17-2009 08:21:03 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-09-17

09-17-2009 08:21:08 -- Initiate backup for "actitime" found on 192.168.50.7!
09-17-2009 08:21:24 -- actitime original powerState: poweredOn
09-17-2009 08:21:24 -- actitime has 1 VMDK(s)
09-17-2009 08:21:24 -- Backing up "DSS1-bond0 actitime/actitime.vmdk" to "DSS1-bkupvm-eth4 ghettoVCB/actitime/actitime-2009-09-17/actitime.vmdk"
09-17-2009 08:21:30 -- Backup completed for DSS1-bond0 actitime/actitime.vmdk
09-17-2009 08:21:30 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-09-17" not found
09-17-2009 08:21:34 -- Unable to purge "DSS1-bkupvm-eth4 ghettoVCB/actitime/GET https:192.168.50.7folderghettoVCBactitime?dsName=DSS1-bkupvm-eth4 unsuccessful : 404 Not Found" due to rotation max
09-17-2009 08:21:34 -- Backup completed for "actitime"!

09-17-2009 08:21:34 -- ERROR - Unable to locate VM: operator

============================== 09-17-2009 08:21:34 -- ghettoVCBg2 LOG END ==============================

Sep 21, 2009 7:51 AM adrian888 says: in response to: datenfalke

Hi All,

I have been hacking this script to find out what the problem is with the snap shots not being deleted and have changed the call from the internal function to the external script snapshotmanager.pl.

#&remove_snapshot($vm_view, $snapshot_name, $vm_name);

my $res = `/home/vi-admin/bin/snapshotmanager.pl -server vc1 --operation remove --vmname $vm_name --username ******* --password ******** --snapshotname $snapshot_name --children 0`;

print LOG timeStamp('MDYHMS'), "Remove Snapshot $snapshot_name from VM $vm_name with results: $res";

It seems like you need to close and reopen the connections to get a refreshed listing of the snap shots? I tried using delays etc, but did not have any consistent results till I tried the external call. This new call has been working the past 2 weeks on the VM's that failed previously.

Sep 21, 2009 7:56 PM lamw

says: in response to: adrian888

This has been on my to-do list, I'm wondering if the connection eventually times out while it's executing the backup ... I have few others that are running into the same issue with removing the snapshot, but I haven't had a chance to replicate the issue. Drop me your email via VMTN PM and I'll send out an email this weekend regarding the next steps and perhaps a live support chat.

Thanks for your patience

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

Sep 21, 2009 10:38 PM bmckerritnu says: in response to: ermasadmin

I had this error and traced it (via perl debugger) to the 'restricted version' error. I had the script working perfectly and all of a sudden it stopped with a message like yours. Turns out that while I was evaluating the product it allows writeable CLI. I never used the script for a while, but sometime later I grabbed free licenses and applied it to all of my ESX4i servers. I'm guessing that this is what has changed the CLI to be read-only and therefor ghetto no longer works ! Can anyone confirm this ?

<RANT>
If this is true then I will for sure be moving all my servers over to Xenserver 5.5 sure it has restrictions also but none as ridiculous as ESXi and the read only CLI. It does have Xenmotion (like vmotion) Xencenter (like vcenter) and a writeable CLI unlike the 'free' ESXi.
</RANT>

Sep 21, 2009 11:00 PM lamw

says: in response to: bmckerritnu

Take a look at this article for your answer: http://vmetc.com/2009/03/31/esxi-u4-ends-free-version-read-and-write-access-from-the-rcli/

The API was never meant for full read/write, this was a bug in VMware's code between U2 and U3 which was remediated in U4 and subsequent release in vSphere. I can understand where VMware's coming from, though I may not agree ... but you pay for you what you get. You still have the ability to use the unsupported SSH console for automating various set of tasks and licensed version of ESXi gives you more than just vCLI access and that is what makes them a leader in this space.

Good luck.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

Sep 21, 2009 11:14 PM bmckerritnu says: in response to: lamw

Thanks William,

I had read the article previously and it did not specifically mention ESX4i. I suppose I am disappointed because your excellent script worked during the evaluation and now it doesn't.

I have enabled 'tech support' mode on all 3 servers but I fear that it too one day soon will disappear.

Sep 22, 2009 6:48 AM lamw

says: in response to: bmckerritnu

The article was published well before vSphere came out, but again this was an internal bug that VMware had and the API was never intended to be free (both read/write), so once this was fixed it would be the case for ESXi 4.0 as well.

Take a look at this comparison chart for the various licensing versions: http://www.vmware.com/products/vsphere/buy/editions_comparison.html as you can see even the simplest licensed version provides much value added than just full r/w access to the API.

Regarding the unsupported console in ESXi, it's VMware's direction to go to the ESXi platform and eventually the classic ESX will no longer be around ( I wouldn't hold my breath anytime soon, but eventually). They may eventually remove the Busybox console as well but again, it'll take time but enjoy it for now.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

Oct 6, 2009 5:28 AM Kabs123 says:

Hi William,

when i run the script, i get the following output:

print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 150
print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 300
print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 300
print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 300
print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 198

ESXi 4 Enterprise, vMA 4.0, backup: nfs

Thank you in advance.

Oct 6, 2009 8:47 AM angelone says: in response to: Kabs123

hi, I suppose the script can't write to the directoy or to the logfile you specified.
Try /tmp/ghettoVCB2.log that should work.

regards,
Martin

Oct 7, 2009 9:20 PM lamw

says:

I wanted to give an update to everyone that may be using this script.

The issues that have been mentioned are being actively worked on, as you know I do support many many other scripts and there's only so many hours in each day I do assure you, I've not forgotten about you guys/gals and the issues being reported. I do ask for you patience and I'll provide an update as soon as I have one.

Stay tuned ...

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

Oct 8, 2009 11:44 PM Charon4you says:

Hi all here!

I/We have 3 ESX4 machines with a shared SAN datastore (all machines can see all SAN datastores). One machine has a "big" Local Storage and I want to backup remote VMs to the Local Storage using SAN, not over a network. Is it possible to configure ghettoVCBg2 to backup VMs this way?

Thanks a lot

Oct 14, 2009 9:21 PM lamw

says:

Hi everyone,

I'm currently in development (as time permits) on some new features for the ghettoVCBg2 script and I'm looking for some individuals that may be interested in taking part in a BETA. The requirement is pretty simple, willing to help test and provide feedback on the next version of the script (remember to test on non-production VMs)

If you're interested, please PM on VTMN with your email address and the script you would like to take part in and I'll be selecting a group users and send out email notification in the coming weeks.

Thanks

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

Oct 21, 2009 2:51 AM Kabs123 says:

When i make a backup from a big VM with 300-500 GB, the VM does not get started automatically after backup prozess.
What could be the matter of this? With smaller VMs it works.

Oct 21, 2009 6:14 AM lamw

says: in response to: Kabs123

This is a known issue with larger VMs and will be fixed in a future release.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

Oct 21, 2009 7:15 AM Kabs123 says: in response to: lamw

OK, thank you for the fast answer.
Do you already know what causes this issue? Maybe i can fix it byself.

Oct 21, 2009 9:28 AM lamw

says: in response to: Kabs123

Nope the solution is actually pretty complicated, we're pretty much there but requires further testing which will be part of the BETA. I won't go into further details, if you're already emailed me then you'll know in the coming weeks if you're one of the few few to be on the BETA.

Thanks for your patience.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

Oct 21, 2009 10:06 AM Kabs123 says: in response to: lamw

All right, i´ve sent you already an email last week for taking part in the BETA. I would help you willingly.

Oct 21, 2009 4:37 PM bbaas75 says: in response to: Kabs123

Hi,

I'm using ESXi 4.0, have a few virtual machines running, and trying to get the backup done. The thing is that I created all virtual machines with Thin disks. However, there are in all those VM's two vmdk's, one called <machinename>-flat.vmdk, and one called <machinename>.vmdk. When I do some filecopying and rename the <machinename>.vmdk to <machinename>.vmdk.old and the <machinename>-flat.vmdk to <machinename>.vmdk the backup seems to work, although I cannot find the backupfiles somewhere on my system. See the attached logs, can anybody help me out?

============================== 10-21-2009 15:44:02 -- ghettoVCBg2 LOG START ==============================
10-21-2009 15:44:02 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DIRECTORY = backup
10-21-2009 15:44:02 -- CONFIG - DISK_BACKUP_FORMAT = thin
10-21-2009 15:44:02 -- CONFIG - ADAPTER_FORMAT = lsilogic
10-21-2009 15:44:02 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
10-21-2009 15:44:02 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 15:44:02 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-10-21

10-21-2009 15:44:07 -- Initiate backup for "TS3" found on 192.168.72.113!
10-21-2009 15:44:14 -- TS3 original powerState: poweredOn
10-21-2009 15:44:14 -- TS3 has 1 VMDK(s)
10-21-2009 15:44:14 -- Backing up "datastore1 TS3/TS3.vmdk" to "datastore1 backup/TS3/TS3-2009-10-21/TS3.vmdk"
10-21-2009 15:44:18 -- Backup completed for datastore1 TS3/TS3.vmdk
10-21-2009 15:44:18 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-10-21" not found
10-21-2009 15:44:19 -- Unable to purge "datastore1 backup/TS3/GET https:192.168.72.113folderbackupTS3?dsName=datastore1 unsuccessful : 404 Not Found" due to rotation max
10-21-2009 15:44:19 -- Backup completed for "TS3"!

============================== 10-21-2009 15:44:19 -- ghettoVCBg2 LOG END ==============================

============================== 10-21-2009 16:00:29 -- ghettoVCBg2 LOG START ==============================
10-21-2009 16:00:29 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DIRECTORY = backup
10-21-2009 16:00:29 -- CONFIG - DISK_BACKUP_FORMAT = thin
10-21-2009 16:00:29 -- CONFIG - ADAPTER_FORMAT = lsilogic
10-21-2009 16:00:29 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
10-21-2009 16:00:29 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 16:00:29 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-10-21

10-21-2009 16:00:35 -- Initiate backup for "TS3" found on 192.168.72.113!
10-21-2009 16:00:42 -- TS3 original powerState: poweredOff
10-21-2009 16:00:42 -- TS3 has 1 VMDK(s)
10-21-2009 16:00:42 -- Backing up "datastore1 TS3/TS3.vmdk" to "datastore1 backup/TS3/TS3-2009-10-21/TS3.vmdk"
10-21-2009 16:00:45 -- Backup completed for datastore1 TS3/TS3.vmdk
10-21-2009 16:00:47 -- Backup completed for "TS3"!

============================== 10-21-2009 16:00:47 -- ghettoVCBg2 LOG END ==============================

Oct 21, 2009 9:13 PM lamw

says: in response to: bbaas75

I'm not exactly sure what you're trying to do? Was the backup successful? Are you saying that you executed a backup of 1 VM (TS3) and after the backup, you manually renamed the files on the ESX(i) host? From the looks of the log, you should have your backup VM stored in /vmfs/volumes/backup/TS3/TS3-2009-10-21/ and note that if you browse your datastore using the vSphere Client that the *-flat.vmdk will not display, only the descriptor.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

Oct 22, 2009 4:18 AM bbaas75 says: in response to: lamw

Hi,

First I tried to do a backup, just of the VM TS3. That resulted in the following:

============================== 10-21-2009 15:44:02 -- ghettoVCBg2 LOG START ==============================
10-21-2009 15:44:02 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DIRECTORY = backup
10-21-2009 15:44:02 -- CONFIG - DISK_BACKUP_FORMAT = thin
10-21-2009 15:44:02 -- CONFIG - ADAPTER_FORMAT = lsilogic
10-21-2009 15:44:02 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
10-21-2009 15:44:02 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 15:44:02 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-10-21

10-21-2009 15:44:07 -- Initiate backup for "TS3" found on 192.168.72.113!
10-21-2009 15:44:14 -- TS3 original powerState: poweredOn
10-21-2009 15:44:14 -- TS3 has 1 VMDK(s)
10-21-2009 15:44:14 -- Backing up "datastore1 TS3/TS3.vmdk" to "datastore1 backup/TS3/TS3-2009-10-21/TS3.vmdk"
10-21-2009 15:44:18 -- Backup completed for datastore1 TS3/TS3.vmdk
10-21-2009 15:44:18 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-10-21" not found
10-21-2009 15:44:19 -- Unable to purge "datastore1 backup/TS3/GET https:192.168.72.113folderbackupTS3?dsName=datastore1 unsuccessful : 404 Not Found" due to rotation max
10-21-2009 15:44:19 -- Backup completed for "TS3"!

============================== 10-21-2009 15:44:19 -- ghettoVCBg2 LOG END ==============================

The backup directory was still empty. So, i tried to get info via Google, and saw something about two VMDK-files. So, I thought probably I can rename the TS3-flat.vmdk to TS3.vmdk. I did (made a saveset first) and then the second run:

============================== 10-21-2009 16:00:29 -- ghettoVCBg2 LOG START ==============================
10-21-2009 16:00:29 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DIRECTORY = backup
10-21-2009 16:00:29 -- CONFIG - DISK_BACKUP_FORMAT = thin
10-21-2009 16:00:29 -- CONFIG - ADAPTER_FORMAT = lsilogic
10-21-2009 16:00:29 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
10-21-2009 16:00:29 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 16:00:29 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-10-21

10-21-2009 16:00:35 -- Initiate backup for "TS3" found on 192.168.72.113!
10-21-2009 16:00:42 -- TS3 original powerState: poweredOff
10-21-2009 16:00:42 -- TS3 has 1 VMDK(s)
10-21-2009 16:00:42 -- Backing up "datastore1 TS3/TS3.vmdk" to "datastore1 backup/TS3/TS3-2009-10-21/TS3.vmdk"
10-21-2009 16:00:45 -- Backup completed for datastore1 TS3/TS3.vmdk
10-21-2009 16:00:47 -- Backup completed for "TS3"!

============================== 10-21-2009 16:00:47 -- ghettoVCBg2 LOG END ==============================

Still the backup directory was empty, but the log seems successfull. Can you point me in the right direction?

Regards,

Bas

Oct 22, 2009 9:26 PM lamw

says:

BETA email invitations will be going out this weekend ... keep an eye out

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

Oct 22, 2009 11:48 PM Koldijk says: in response to: lamw

I have installed this wonderfull script. Keep up the good work!

I am having trouble with crontab. My crontab (vi-admin) looks like:
vi-admin@vma ~$ crontab -l
0 0 * * * /home/vi-admin/ghettoVCBg2.pl --config_dir /home/vi-admin/backupConfigs --vmlist /home/vi-admin/virtual_machne_back_list_belangrijk
0 10 * * 7 /home/vi-admin/ghettoVCBg2.pl --config_dir /home/vi-admin/backupConfigs --vmlist /home/vi-admin/virtual_machne_back_list
0 5 * * 1-5 /home/vi-admin/ghettoVCBg2.pl --config_dir backupConfigs --vmlist /home/vi-admin/virtual_machne_back_list_branderweg

But cron doesn't run it at all. Did I missed something? When I use the lines at the shell the commands make great backups.
a tail of /var/log/cron gives:
~~vi-admin@vma ~~~$ sudo tail /var/log/cron
Password:
Oct 23 00:01:01 localhost crond14745: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 01:01:01 localhost crond14987: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 02:01:01 localhost crond15229: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 03:01:01 localhost crond15471: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 04:01:01 localhost crond15713: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 05:01:01 localhost crond15955: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 06:01:01 localhost crond16224: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 07:01:01 localhost crond16466: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 08:01:01 localhost crond16708: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 08:26:49 localhost crond2074: (CRON) STARTUP (V5.0)

Why cron isn't running my backups?

Oct 27, 2009 10:25 AM BasicVirtu says: in response to: Koldijk

I am trying to backup vms in an ESX4 host that has mostly SAN based storage to an ESXi server with local storage over the network to a remote location. It's not clear to me whether I need to mount an NFS store on the ESX4 host or if the store still needs to be visible to the ESX4 host or if the vMA should take care of this as long as it is listed as being managed by the vMA. Do I still need to mount the NFS store manually on each ESX4 host so that $VM_BACKUP_DATASTORE that I'm backing up to is visible to that individual ESX4 host?

I think I'm just confused on the syntax for the datastore or how the script works with vMA, I verified that both hosts are being managed by vMA with sudo vifp listservers and I have tried several different names for the $VM_BACKUP_DATASTORE value. Everything in the logs says the backup completes but on the ESX4 host it keeps saying invalid datastore specified so it is clear it doesn't know what datastore I am trying to backup to. I have also tried specifying the ESXi server that I am trying to backup to with the --server command option but doesn't seem to make a difference.

I’ve tried several different values and no luck.

my $VM_BACKUP_DATASTORE = "/vmfs/volumes/backups";

my $VM_BACKUP_DATASTORE = "https://172.16.x.x/folder?dcPath=ha-datacenter&dsName=backups";

my $VM_BACKUP_DATASTORE = "172.16.x.x/folder?dcPath=ha-datacenter&dsName=backups";

my $VM_BACKUP_DATASTORE = "172.16.x.x";

Thanks again for all your work any help is apprieciated!

Oct 27, 2009 10:51 AM lamw

says: in response to: BasicVirtu

The NFS datastore MUST be mounted on the ESX(i)host and must be visible to the host, vMA will instruct the various operations using the vSphere API to perform the backup to the NFS datastore or to any valid datastore for that matter.

Your syntax is for VM_BACKUP_DATASTORE is incorrect, you apparently did not go through the documentation throughly. The entry should just be the name of your datastore (e.g. datastore1)

If your NFS datastore is mounted on the ESX(i) host as backups then variable VM_BACKUP_DATASTORE should be set to backups

I would recommend going over the documentation and read through it carefully as much of your questions can be answered.

Thanks

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
vGhetto Script Repository
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity
Twitter: @lamw

If you find this information useful, please award points for "correct" or "helpful".

Oct 27, 2009 10:53 AM BasicVirtu says: in response to: lamw

Thank you that helps alot!

Nov 9, 2009 4:33 AM Koldijk says: in response to: lamw

Hi William,

Every night I am backing up my vm's with the script. with 3 of them the snapshot is not found and not deleted. For your information these Vm's are greater dan 30GB..
See the log file.

11-08-2009 00:52:06 -- CONFIG - USING CONFIGURATION FILE = UPDATES
11-08-2009 00:52:06 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-08-2009 00:52:06 -- CONFIG - VM_BACKUP_DATASTORE = vmbackup
11-08-2009 00:52:06 -- CONFIG - VM_BACKUP_DIRECTORY = vmware
11-08-2009 00:52:06 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
11-08-2009 00:52:06 -- CONFIG - ADAPTER_FORMAT = lsilogic
11-08-2009 00:52:06 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-08-2009 00:52:06 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-08-2009 00:52:06 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-08-2009 00:52:06 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-08
11-08-2009 00:52:06 -- Initiate backup for "UPDATES" found on 10.3.1.25!
11-08-2009 00:52:16 -- UPDATES original powerState: poweredOn
11-08-2009 00:52:18 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-11-08" for UPDATES
11-08-2009 00:52:18 -- UPDATES has 1 VMDK(s)
11-08-2009 00:52:18 -- Backing up "NFS updates/UPDATES_ESXi1.vmdk" to "vmbackup vmware/UPDATES/UPDATES-2009-11-08/UPDATES_ESXi1.vmdk"
11-08-2009 02:03:40 -- Backup completed for NFS updates/UPDATES_ESXi1.vmdk
11-08-2009 02:03:40 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-11-08" not found
11-08-2009 02:03:55 -- Backup completed for "UPDATES"!

And a other:
============================== 11-09-2009 12:29:04 -- ghettoVCBg2 LOG START ==============================

11-09-2009 12:29:09 -- Initiate backup for "Helpdesk" found on 10.3.1.24!
11-09-2009 12:29:21 -- Helpdesk has 1 VMDK(s)
11-09-2009 13:07:02 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-11-09" not found
11-09-2009 13:07:18 -- Backup completed for "Helpdesk"!

============================== 11-09-2009 13:07:22 -- ghettoVCBg2 LOG END ==============================

How is the Beta going? Can I help?

Regards,

Rob

Nov 9, 2009 9:09 PM lamw

says: in response to: Koldijk

Hi Rob,

As mentioned in previous replies, the issue with backing up large VMs has been resolved in the next release of the script. I can't say much more but keep an eye out for the next version of the script soon

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Nov 18, 2009 5:27 PM adrian888 says: in response to: lamw

Hi,

to get the email to work I had to change line # 302
my $smtp = Net::SMTP->new($EMAIL_HOST ,Hello => 'vMA - ghettoVCBg2',Timeout => 30,);
to
my $smtp = Net::SMTP->new($EMAIL_HOST,Hello => 'mycompanydomain.com',Timeout => 30,);
or just do this
my $smtp = Net::SMTP->new($EMAIL_HOST);

I was always getting the error that it could not connect if I did not put my local company domain in.

Nov 18, 2009 7:49 PM lamw

says: in response to: adrian888

Thanks for the info! I thought I had it working without specifying the entry and using a dummy string but I guess not .... well I've added an additional variable that'll specify the email domain and if you don't want to specify it, it'll default to localhost.localdomain as defined here: http://perldoc.perl.org/Net/SMTP.html

Glad to hear everything else is good

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Nov 18, 2009 11:00 PM adrian888 says: in response to: lamw

Great! Thanks for the quick fix.

Everything else is working great, no more snapshot problems! I also really like the selective backups!

Nov 19, 2009 2:11 AM Koldijk says: in response to: lamw

Hello William,

Thanks fore the new script! I have downloaded the new script and give the backup a GO.
Running bt hand gives me:
~~vi-admin@vma ~~~$ ./ghettoVCBg2.pl --config_dir /home/vi-admin/backupConfigs --vmlist helpdesk
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
~~vi-admin@vma ~~~$

the /tmp/ghettoVCBg2.log:

~~vi-admin@vma ~~~$ tail -f /tmp/ghettoVCBg2.log

11-19-2009 10:28:34 -- info: Initiate backup for Helpdesk found on 10.3.1.24
11-19-2009 10:28:44 -- debug: Helpdesk original powerState: poweredOn
11-19-2009 10:28:44 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2009-11-19" for Helpdesk
11-19-2009 10:28:54 -- info: Helpdesk has 1 VMDK(s)
11-19-2009 10:28:54 -- debug: backupVMDK: Backing up "NFS Helpdesk/Helpdesk.vmdk" to "vmbackup vmware/Helpdesk/Helpdesk-2009-11-19/Helpdesk.vmdk"
11-19-2009 10:28:54 -- debug: backupVMDK: Signal copyThread to start
11-19-2009 10:28:54 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
11-19-2009 10:28:54 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
11-19-2009 10:28:54 -- debug: CopyThread: Start backing up VMDK(s) ...
11-19-2009 10:43:57 -- debug: backupVMDK: Backup progress: Elapsed time 15 min
11-19-2009 10:59:00 -- debug: backupVMDK: Backup progress: Elapsed time 30 min
11-19-2009 11:04:22 -- debug: copyTask: send copySuccess message ...
11-19-2009 11:04:22 -- debug: copyTask: waiting for next job and sleep ...
11-19-2009 11:05:02 -- debug: backupVMDK: Successfully completed backup for NFS Helpdesk/Helpdesk.vmdk Elapsed time: 36 min
11-19-2009 11:05:18 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2009-11-19" for Helpdesk
11-19-2009 11:05:21 -- debug: checkVMBackupRotation: Starting ...
11-19-2009 11:05:40 -- debug: Purging vmbackup vmware/Helpdesk/Helpdesk-2009-11-13--7 due to rotation max
11-19-2009 11:05:40 -- info: Backup completed for Helpdesk!

11-19-2009 11:05:40 -- debug: Main: Disconnect from: 10.3.1.24

11-19-2009 11:05:40 -- debug: Main: Login by vi-fastpass to: 10.3.1.25
11-19-2009 11:05:43 -- debug: Main: Disconnect from: 10.3.1.25

11-19-2009 11:05:43 -- debug: Main: Login by vi-fastpass to: 10.3.1.26
11-19-2009 11:05:45 -- debug: Main: Disconnect from: 10.3.1.26

11-19-2009 11:05:45 -- debug: Main: Calling final clean up
11-19-2009 11:05:45 -- debug: cleanUP: Thread clean up starting ...
11-19-2009 11:05:45 -- debug: cleanUp: Send exit to copyThread
11-19-2009 11:05:45 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit
11-19-2009 11:05:45 -- debug: copyTask: die ...
11-19-2009 11:05:45 -- debug: cleanUp: Join passed
11-19-2009 11:05:45 -- info: ============================== ghettoVCBg2 LOG END ==============================

Do you have a idea? The backup is running fine, with a large disk

Thanks!
kind regards,
Rob

Nov 19, 2009 9:44 AM lamw

says: in response to: Koldijk

It sounds like one of the logging variables is not configured correctly, can you please output one of the VM backup policy? The only supported logging option is either "info" or "debug" and hopefully that's defined properly in all your configs.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Nov 19, 2009 10:36 PM Koldijk says: in response to: lamw

William,

Thanks, you where right. I still used "verbose".
I'll change that right away.

Regards,

Rob

Nov 20, 2009 2:58 AM deniseche says:

Nice Script and Nice work !
I had trouble with previous version.
Now it's works fine ! I can backup all my vm and i receive a report status in my mailbox everytime !

Thanks a lot William.

Nov 20, 2009 7:57 AM lamw

says: in response to: Koldijk

Yea that was changed in this release with just having two logging options. Glad to see you got it working

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Nov 20, 2009 7:58 AM lamw

says: in response to: deniseche

Awesome to hear!

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Nov 21, 2009 1:39 AM boelama says:

hi all ,
I just installed this fine tool to evaluate is as main backup tool
(after being disapointed of other commercial tools)

i was wondering if it would be possible to add a feature to execute a script add the end of each VM backup
something like

POST_BACKUP_SCRIPT=/home/vi-admin/myscript.sh
POST_BACKUP_SCRIPT_ONESX=true

this would copy the script to the ESX/ESXi server and execute it after each backup.
i would provide 2 parameters :
- path of backup location
- name of VM

we could use this to compress the files in a rsync friendly way
so we can transfer the "big" files to a offsite storage for save keeping

Nov 21, 2009 2:47 AM roadfox says: in response to: boelama

Like the idea of post script. Another idea might be to add another option --vm vmname so i could create a backup script like:

./ghettoVCBg2.pl --vm vm_to_backup
my_post_script vm_to_backup
./ghettoVCBg2.pl --vm next_vm_to_backup
my_post_script next_vm_to_backup
....

so one could then choose if he wants to use --vmlist or just --vm

Nov 21, 2009 9:17 AM lamw

says: in response to: boelama

This kind of defeats the purpose of the script not requiring any SSH connections to either the classic ESX or unsupported busybox console on ESXi, else a user could just use [ghettoVCB.sh|ghettoVCB.sh - Free alternative for backing up VM's for ESX(i) 3.5 and 4.0+ which does support compression.

If you're looking for such functionality, I would recommend you checking out VMware's vDR or Veeams Backup/Recovery (which I've heard great things about, especially supporting change block tracking). There are no compression support within the VMware APIs ... minus change block tracking which I'm sure is not a trivial task to implement into a backup solution.

Having said that, you're more than welcome to modify/tweak the script to fit your environment but this is something that will not be supported.

Thanks for the comments.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Nov 21, 2009 9:20 AM lamw

says: in response to: roadfox

You also fail to understand the point of having either a global configuration for all VMs to be backed up or individual VM policy ...by having specifying individual VM to be backed up on the command line, you've introduced additional overhead which can be solved by using individual VM backup policy.

If the suggestion of allowing some type of post script to execute, this config would be stored in the VM policy file and you would still have 1 global vm list to backup but the specific params to each VM would be store in a configuration file which is why -c option is for.

Again, you can take a look at my previous reply, this defeats the purpose of this specific script not utilizing SSH console on either ESX(i) and you're more than welcome to modify/tweak the script to fit your environment but this is something that will not be supported.

Thanks for the commands.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Nov 21, 2009 11:56 PM boelama says: in response to: lamw

It was not my intention to break the "no SSH" requirements. i'm not yet on speed on what is possible with all
API's but my tough where going in the way of coping the script to the ESX(i) server and execute it there because
there is access to the backup files .

thinking longer on the subject , i guess that it would be hard to create scripts that can work in ESX/ESXi environment.
especially because ESXi is a very stripped busybox environment and VMWare would like it to be misused.

luc

Nov 22, 2009 1:34 AM roadfox says: in response to: lamw

What i fail to understand is what is option -c for, can't find a description for it.
My problem is that at the moment i have to maintain a configuration in the config dir and a vmlist file for each vm. My recomendation would just avoid this:

"Note: --vmlist is still a required parameter with the exception that backup policy files must exist for each VM in the list else the VM will be ignored in the backup process."

What i would like to achieve is that we just need to maintain one configuration file per backup policy (not per vm, thats to much work) and then call ghettoVCBg2 with the right policy for the right vm.

The postscripts im using do not connect to the ESXi host, they do manipulations in the NFS filer which is the datastore.

Nov 22, 2009 9:48 AM lamw

says: in response to: roadfox

If you don't understand the -c which specifies a configuration directory for your individual VM backup policy, you may want to go through the documentation carefully and I've provided some example output to show you each of the options.

With your logic, you're effectively duplicating the number of files that need to be created and maintain to accomplish the exact same task that is already available in the script today.

There are two core use cases that I've designed the script to handle:

1) Global configuration where all VMs are backed up in the same manner (1 vmlist, no need for -c option)
2) Individual VM backup policy, where you MUST create individual policy on a per VM basis, as each VM differs from one another (1 vmlist, -c option required)

What i would like to achieve is that we just need to maintain one configuration file per backup policy (not per vm, thats to much work) and then call ghettoVCBg2 with the right policy for the right vm.

That is effectively option #2 and again running any type of post/pre scripts is not supported as this defeats the purpose of the script not requiring any type of SSH connectively to the ESX(i) host as mentioned in the last replies.

Again, you're more than welcome to add/modify the script to fit your needs, its definitely doable but will not be supported.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Nov 30, 2009 7:34 AM KKrtz says:

Hello.
I have a few machines which cannot be backuped by ghettovcbg2. After starting the script only the "vmx" file will be copied. No vmdk files ?! (snapshot is created and deleted) - I doesn't make any difference by using the old or new version of the script.

Any ideas ???

(As you can see -> 0 min to copy the vmdk!)

The debug log from the email:
11-30-2009 16:25:50 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-30-2009 16:25:50 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-30-2009 16:25:50 -- info: CONFIG - VM_BACKUP_DATASTORE = nfs-storage
11-30-2009 16:25:50 -- info: CONFIG - VM_BACKUP_DIRECTORY = backup
11-30-2009 16:25:50 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
11-30-2009 16:25:50 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-30-2009 16:25:50 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-30-2009 16:25:50 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-30-2009 16:25:50 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-30-2009 16:25:50 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-30
11-30-2009 16:25:50 -- info: CONFIG - VM_VMDK_FILES = all

11-30-2009 16:25:50 -- debug: Main: Login by vi-fastpass to: esx01.short
11-30-2009 16:25:50 -- debug: copyTask: Task START
11-30-2009 16:25:50 -- debug: copyTask: waiting for next job and sleep ...
11-30-2009 16:25:52 -- info: Initiate backup for SH28S found on esx01.short
11-30-2009 16:25:59 -- debug: SH28S original powerState: poweredOff
11-30-2009 16:25:59 -- info: SH28S has 1 VMDK(s)
11-30-2009 16:25:59 -- debug: backupVMDK: Backing up "msa-storage SH28S/SH28S.vmdk" to "nfs-storage backup/SH28S/SH28S-2009-11-30/SH28S.vmdk"
11-30-2009 16:25:59 -- debug: backupVMDK: Signal copyThread to start
11-30-2009 16:25:59 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
11-30-2009 16:25:59 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
11-30-2009 16:25:59 -- debug: CopyThread: Start backing up VMDK(s) ...
11-30-2009 16:26:01 -- debug: copyTask: send copySuccess message ...
11-30-2009 16:26:01 -- debug: copyTask: waiting for next job and sleep ...
11-30-2009 16:27:00 -- debug: backupVMDK: Successfully completed backup for msa-storage SH28S/SH28S.vmdk Elapsed time: 1 min
11-30-2009 16:27:01 -- debug: checkVMBackupRotation: Starting ...
11-30-2009 16:27:01 -- debug: Purging nfs-storage backup/SH28S/SH28S-2009-11-30--1 due to rotation max
11-30-2009 16:27:02 -- info: Backup completed for SH28S!

11-30-2009 16:27:02 -- debug: Main: Disconnect from: esx01.short

11-30-2009 16:27:02 -- debug: Main: Login by vi-fastpass to: esx02.short
11-30-2009 16:27:03 -- debug: Main: Disconnect from: esx02.short

11-30-2009 16:27:03 -- debug: Main: Calling final clean up
11-30-2009 16:27:03 -- debug: cleanUP: Thread clean up starting ...
11-30-2009 16:27:03 -- debug: cleanUp: Send exit to copyThread
11-30-2009 16:27:03 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit
11-30-2009 16:27:03 -- debug: copyTask: die ...
11-30-2009 16:27:03 -- debug: cleanUp: Join passed
11-30-2009 16:27:03 -- info: ============================== ghettoVCBg2 LOG END ==============================

Nov 30, 2009 7:55 AM lamw

says: in response to: KKrtz

It looks like it might not have been able to locate the specific VMDK in question, right after locating the # of VMDK(s), you should see a line like this:

11-14-2009 00:19:45 --  debug: findVMDKFile: Found VMDK File: VIMA_1.vmdk

Can you paste the output of dryrun mode, please take a look at the documentation on how to enable dryrun and please wrap your output using {code} tags for readability

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Nov 30, 2009 8:05 AM KKrtz says: in response to: lamw

11-30-2009 17:02:23 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-30-2009 17:02:23 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-30-2009 17:02:23 -- info: CONFIG - VM_BACKUP_DATASTORE = nfs-storage
11-30-2009 17:02:23 -- info: CONFIG - VM_BACKUP_DIRECTORY = backup
11-30-2009 17:02:23 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
11-30-2009 17:02:23 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-30-2009 17:02:23 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-30-2009 17:02:23 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-30-2009 17:02:23 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-30-2009 17:02:23 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-30
11-30-2009 17:02:23 -- info: CONFIG - VM_VMDK_FILES = all

11-30-2009 17:02:23 -- debug: Main: Login by vi-fastpass to: esx01.short
11-30-2009 17:02:23 -- debug: copyTask: Task START
11-30-2009 17:02:23 -- debug: copyTask: waiting for next job and sleep ...
11-30-2009 17:02:25 -- info:

DRYRUN DEBUG INFO SH28S

11-30-2009 17:02:25 -- info: DEBUG - Host Build: VMware ESX Server 3i 3.5.0 build-123629
11-30-2009 17:02:25 -- info: DEBUG - Host: esx01.short
11-30-2009 17:02:25 -- info: DEBUG - Virtual Machine: SH28S
11-30-2009 17:02:25 -- info: DEBUG - VM ConfigPath: msa-storage SH28S/SH28S.vmx
11-30-2009 17:02:25 -- info: DEBUG - VMX File: SH28S.vmx
11-30-2009 17:02:25 -- info: DEBUG - BackupConfigPath: nfs-storage backup/SH28S/SH28S-2009-11-30/SH28S.vmx
11-30-2009 17:02:25 -- info: DEBUG - BackupPath: nfs-storage backup/SH28S/SH28S-2009-11-30
11-30-2009 17:02:25 -- info: DEBUG - VM Datastore: msa-storage
11-30-2009 17:02:25 -- info: DEBUG - VMDK(s):
11-30-2009 17:02:25 -- info: DEBUG - msa-storage SH28S/SH28S.vmdk
11-30-2009 17:02:25 -- info:

DRYRUN DEBUG INFO SH28S

11-30-2009 17:02:25 -- debug: Main: Disconnect from: esx01.short

11-30-2009 17:02:25 -- debug: Main: Login by vi-fastpass to: ESX02.short
11-30-2009 17:02:26 -- debug: Main: Disconnect from: ESX02.short

11-30-2009 17:02:26 -- debug: Main: Calling final clean up
11-30-2009 17:02:26 -- debug: cleanUP: Thread clean up starting ...
11-30-2009 17:02:26 -- debug: cleanUp: CopyTask was never started, send copyTaskStart
11-30-2009 17:02:26 -- debug: cleanUp: Send exit to copyThread
11-30-2009 17:02:26 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit
11-30-2009 17:02:26 -- debug: copyTask: die ...
11-30-2009 17:02:26 -- debug: cleanUp: Join passed
11-30-2009 17:02:26 -- info: ============================== ghettoVCBg2 LOG END ==============================

Nov 30, 2009 8:24 AM lamw

says: in response to: KKrtz

I see that you're using ESXi 3.5, do you know which Update this is? and are you using the free version of ESXi?

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Nov 30, 2009 8:26 AM KKrtz says: in response to: lamw

Version 3.5 U3 and license is free! Please keep in mind that 90 percent of the other machines are working ...

Nov 30, 2009 8:55 AM lamw

says: in response to: KKrtz

Interesting, I can't see anything wrong from the logs and it seems like the VMDK passes the important checks such as not being physical RDM nor independent disk ... but it's only backing up the .vmx file ... usually this occurs if you're using the free version of ESXi and vmkfstools is not allowing you to do the copy of the VMDK.

Does this path still exists on your ESXi host:

[nfs-storage] backup/SH28S/SH28S-2009-11-30

If so, can we try manually calling vmkfstools and see if you are in fact using U3 which has the API bug?

vmkfstools --server <server_name> --username <username> -i "[msa-storage] SH28S/SH28S.vmdk" "[nfs-storage] backup/SH28S/SH28S-2009-11-30/SH28S.vmdk"

Please fill in the fields that match your system and we'll see if you get access denied OR if its actually issuing the backup.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Nov 30, 2009 9:01 AM KKrtz says: in response to: lamw

the path still exists
after entering your command the following message appears on the vima cli: "Unable to clone virtual disk : A general system error occurred: Internal error"

Nov 30, 2009 11:22 AM lamw

says: in response to: KKrtz

I assume you took a snapshot prior to executing the command? else it will not work

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 1, 2009 12:55 AM KKrtz says: in response to: lamw

As I see in the GUI there is no open snapshot ...

Dec 1, 2009 4:45 AM lamw

says: in response to: KKrtz

For the command to work, you must have a snapshot else the disk is locked. Hopefully you understand why ...

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 1, 2009 5:34 AM KKrtz says: in response to: lamw

no open or existing snapshot !

"vmkfstools -i" from the console on the esx works fine ! (probably the command syntax from vima is not correct)

...my problem is that there is no failure message ...

Dec 1, 2009 5:39 AM lamw

says: in response to: KKrtz

If the VM is powered on, then you need a snapshot, hopefully you understand this point. If the VM is powered off, then you don't need a snapshot

Try this ( for vmA, you'll need to specify both adapter/disktype from what I remember):

vmkfstools --server <server_name> --username <username> -i "[msa-storage] SH28S/SH28S.vmdk" -a buslogic -d thin "[nfs-storage\ backup/SH28S/SH28S-2009-11-30/SH28S.vmdk"

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 1, 2009 7:49 AM KKrtz says: in response to: lamw

instead of using "[msa-storage..." i've used the full qualified path "/vmfs...." for both (source and destination" and now it works !!

and what does this mean for my backup problem ??

Dec 1, 2009 9:16 AM lamw

says: in response to: KKrtz

Are you running this on VIMA 1.0 or vMA 4.0?

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 1, 2009 9:17 AM KKrtz says: in response to: lamw

VIMA 1.0

Dec 1, 2009 10:30 AM lamw

says: in response to: KKrtz

I know there were some vmkfstools syntax issues on VIMA 1.0, can you deploy vMA 4.0 and try the backup on it? It should still work with ESXi 3.5u3 as the API is exposed from the host side. Let me know if that resolves the problem

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 2, 2009 3:17 AM KKrtz says: in response to: lamw

ok I've done the update to vMa and the backup still doesn't work ... but ... I've got now an new failure message in my task pane in the infrastructure client gui

Task: "Copy Virtual Disk - General fault caused by file. One of the parameters supplied is invalid"

Dec 2, 2009 5:12 AM KKrtz says: in response to: KKrtz

ok now i've found the problem with changing the script to give me an output when starting the copy command (like descriped before in the comments)

the problem was the diskformat ..like you wrote me I have to use "thin" but in the script was written "zeroedthick". At the moment I don't understand the reason why 95 percent of my VM work with zero and some only with thin ???

p.s. sorry for using your time to search a failure which was mine.....

Dec 2, 2009 6:19 AM KKrtz says:

New features/enhancements request:

eMail -> Possibility to change the subject (at the moment I do it manually
Output -> Possibility to get an output on the shell what the script is doing at moment (like adding the print command)

Thankz

Dec 2, 2009 6:49 AM lamw

says: in response to: KKrtz

No worries, glad we were able to get to the bottom of this since it did sound like a very 'odd' issue.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 2, 2009 10:57 AM fahrer says: in response to: lamw

Hey lamw,

nice scripts. it works fine for me so far. unless there is something wrong with the vmfs share.
if it doesn't exist or is not writeable, the script doesn't complain. it rather says the backup was ok and continuous. vms on the host, that have not been backed up yet, are then mentioned as "Unable to locate".

is this a known bug? let me know if u need any outputs of the logfiles.
i did not look at the source so far. just let me know if i should try any changes or whatever.

Kind regards

Julian

Dec 2, 2009 8:51 PM lamw

says: in response to: fahrer

Can you define "something wrong with vmfs share" ? I'll have to go with the assumption that your destination volume is valid and available ... if it's not then that should be up to the administrator to ensure it is and if not, properly alert on it.

There are different issues that could occur when a volume is not available, either its not mounted like an NFS volume or a volume that has been dropped (FC/iSCSI LUN)...etc. Currently the script does not run any type of verifications on the destination and it'll try to create directories/etc and prep for the backup and if it's unable to copy, you'll get an error in the logs.

If it's still occurring, please run the script with debugging flag on and if you have questions about this, please refer to the documentation and remember to wrap your output in {code} tags for readability.

Thanks

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 3, 2009 3:24 AM fahrer says: in response to: lamw

the definition for "something wrong" is: it doesn't exist or is not writeable
here is the log output when running a backup on a datastore which does not exist (same thing when the datastore is not writeable):

	12-03-2009 12:10:57 --  info: ============================== ghettoVCBg2 LOG START ==============================
	12-03-2009 12:11:08 --  info: CONFIG - USING CONFIGURATION FILE = necksolvminfra
	12-03-2009 12:11:08 --  info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
	12-03-2009 12:11:08 --  info: CONFIG - VM_BACKUP_DATASTORE = netapp_b02
	12-03-2009 12:11:08 --  info: CONFIG - VM_BACKUP_DIRECTORY = ghettoVCB
	12-03-2009 12:11:08 --  info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
	12-03-2009 12:11:08 --  info: CONFIG - ADAPTER_FORMAT = lsilogic
	12-03-2009 12:11:08 --  info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
	12-03-2009 12:11:08 --  info: CONFIG - VM_SNAPSHOT_MEMORY = NO
	12-03-2009 12:11:08 --  info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
	12-03-2009 12:11:08 --  info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-12-03
	12-03-2009 12:11:08 --  info: CONFIG - VM_VMDK_FILES = all

	12-03-2009 12:11:08 --  info: Initiate backup for necksolvminfra found on necksrvviesx02.comany.int
	12-03-2009 12:11:18 --  debug: necksolvminfra original powerState: poweredOn
	12-03-2009 12:11:18 --  debug: Creating Snapshot "ghettoVCBg2-snapshot-2009-12-03" for necksolvminfra
	12-03-2009 12:11:28 --  info: necksolvminfra has 1 VMDK(s)
	12-03-2009 12:11:28 --  debug: backupVMDK: Backing up "[MR1_V00_P0A0_A] necksolvminfra/necksolvminfra.vmdk" to "[netapp_b02] ghettoVCB/necksolvminfra/necksolvminfra-2009-12-03/necksolvminfra.vmdk"
	12-03-2009 12:11:28 --  debug: backupVMDK: Signal copyThread to start
	12-03-2009 12:11:28 --  debug: backupVMDK: Backup progress: Elapsed time 0 min
	12-03-2009 12:11:28 --  debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
	12-03-2009 12:11:28 --  debug: CopyThread: Start backing up VMDK(s) ...
	12-03-2009 12:11:35 --  debug: copyTask: send copySuccess message ...
	12-03-2009 12:11:35 --  debug: copyTask: waiting for next job and sleep ...
	12-03-2009 12:12:29 --  debug: backupVMDK: Successfully completed backup for [MR1_V00_P0A0_A] necksolvminfra/necksolvminfra.vmdk Elapsed time: 1 min
	12-03-2009 12:12:32 --  debug: Removing Snapshot "ghettoVCBg2-snapshot-2009-12-03" for necksolvminfra
	12-03-2009 12:12:34 --  debug: checkVMBackupRotation: Starting ...
	12-03-2009 12:12:34 --  info: Backup completed for necksolvminfra!

	12-03-2009 12:12:34 --  debug: Main: Disconnect from: necksrvviesx02.comany.int

	12-03-2009 12:12:34 --  debug: Main: Login by vi-fastpass to: necksrvviesx03.comany.int
	12-03-2009 12:12:39 --  debug: Main: Disconnect from: necksrvviesx03.comany.int

	12-03-2009 12:12:39 --  debug: Main: Login by vi-fastpass to: necksrvviesx04.comany.int
	12-03-2009 12:12:43 --  debug: reConfigureBackupParams: VM - NeckSRVTermPST
	12-03-2009 12:12:43 --  info: CONFIG - USING CONFIGURATION FILE = NeckSRVTermPST
	12-03-2009 12:12:43 --  info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
	12-03-2009 12:12:43 --  info: CONFIG - VM_BACKUP_DATASTORE = netapp_b02
	12-03-2009 12:12:43 --  info: CONFIG - VM_BACKUP_DIRECTORY = ghettoVCB
	12-03-2009 12:12:43 --  info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
	12-03-2009 12:12:43 --  info: CONFIG - ADAPTER_FORMAT = buslogic
	12-03-2009 12:12:43 --  info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
	12-03-2009 12:12:43 --  info: CONFIG - VM_SNAPSHOT_MEMORY = NO
	12-03-2009 12:12:43 --  info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
	12-03-2009 12:12:43 --  info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-12-03
	12-03-2009 12:12:43 --  info: CONFIG - VM_VMDK_FILES = all

	12-03-2009 12:12:43 --  info: Initiate backup for NeckSRVTermPST found on necksrvviesx04.comany.int
	12-03-2009 12:12:53 --  debug: NeckSRVTermPST original powerState: poweredOn
	12-03-2009 12:12:53 --  debug: Creating Snapshot "ghettoVCBg2-snapshot-2009-12-03" for NeckSRVTermPST
	12-03-2009 12:13:03 --  info: NeckSRVTermPST has 1 VMDK(s)
	12-03-2009 12:13:03 --  debug: backupVMDK: Backing up "[MR2_V03_P1S1_A] NeckSRVTermPST/NeckSRVTermPST.vmdk" to "[netapp_b02] ghettoVCB/NeckSRVTermPST/NeckSRVTermPST-2009-12-03/NeckSRVTermPST.vmdk"
	12-03-2009 12:13:03 --  debug: backupVMDK: Signal copyThread to start
	12-03-2009 12:13:03 --  debug: backupVMDK: Backup progress: Elapsed time 0 min
	12-03-2009 12:13:03 --  debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
	12-03-2009 12:13:03 --  debug: CopyThread: Start backing up VMDK(s) ...
	12-03-2009 12:13:10 --  debug: copyTask: send copySuccess message ...
	12-03-2009 12:13:10 --  debug: copyTask: waiting for next job and sleep ...
	12-03-2009 12:14:04 --  debug: backupVMDK: Successfully completed backup for [MR2_V03_P1S1_A] NeckSRVTermPST/NeckSRVTermPST.vmdk Elapsed time: 1 min
	12-03-2009 12:14:06 --  debug: Removing Snapshot "ghettoVCBg2-snapshot-2009-12-03" for NeckSRVTermPST
	12-03-2009 12:14:08 --  debug: checkVMBackupRotation: Starting ...
	12-03-2009 12:14:08 --  info: Backup completed for NeckSRVTermPST!

	12-03-2009 12:14:08 --  debug: Main: Disconnect from: necksrvviesx04.comany.int

	12-03-2009 12:14:08 --  error: getFinalList: ERROR - Unable to locate VM: NeckLNXVMvma01

	12-03-2009 12:14:08 --  debug: Main: Calling final clean up
	12-03-2009 12:14:08 --  debug: cleanUP: Thread clean up starting ...
	12-03-2009 12:14:08 --  debug: cleanUp: Send exit to copyThread
	12-03-2009 12:14:08 --  debug: copyTask: Wake up and follow the white rabbit, with status: exit
	12-03-2009 12:14:08 --  debug: copyTask: die ...
	12-03-2009 12:14:08 --  debug: cleanUp: Join passed
	12-03-2009 12:14:08 --  info: ============================== ghettoVCBg2 LOG END ==============================

i cannot find an error in the log, or am i just blind?
And the machine NeckLNXVMvma01 is there. After a job on a host "fails", the loop which goes through the vms on a host seems not to finish.

Dec 3, 2009 10:40 AM lamw

says: in response to: fahrer

I'll see if I can do a check on the destination datastore, there is an API property that checks to see if its accessible...but that may just mean if the ESX(i) host can see the datastore, it may not check to see if its actually mounted or if there are issues.

Again, I would probably say this is a corner case as I would hope that this type an issue where a volume is not available or having issue is properly monitored by the infrastructure VI team ... the expectation/assumption is that your environment has no issues when trying to run the backup

I'll let you know if I update the script, need to find some free time

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 4, 2009 12:17 AM fahrer says: in response to: lamw

Hey,

u are right. The datastore should be monitored by the vi team. but maybe there is a typo in the backup configuration.
I don't know much about threads in perl, but can't u just check for the exit code of the vmkfstools?

[vi-admin@NeckLNXVMvma01 ~]$ vmkfstools --server necksrvviesx01.company.int --username root --password xxx -i "[MR2_V03_P1S1_A] NeckSRVVMPrt03/NeckSRVVMPrt03.vmdk"  -a lsilogic -d zerodthick  "[netapp_b02] ghettoVCB/NeckSRVVMPrt03/NeckSRVVMPrt03-2009-12-03/NeckSRVVMPrt03.vmdk"
Unable to clone virtual disk : vim.fault.InvalidDatastore
[vi-admin@NeckLNXVMvma01 ~]$ echo $?
1

the exit code is also != 0 when the store is not writeable.

something like that:

                if ($copyThreadStatus eq "doCopy"){
                        &log("debug", "CopyThread: Start backing up VMDK(s) ...");
                        eval {
                                my $vmkfstools_cpy = `$copyCmd` ;
                                if ($?) {
                                    die "Could not backup VMDK(s): $?, $vmkfstools_cpy";
                                }
                                &log("debug", "copyTask: send copySuccess message ...");
                                $copyThreadStatus= "copySuccess";
                        } or do {
                                &log("error", "CopyTask: ". $@ );
                                &log("error", "CopyTask: Cmd " .$copyCmd);
                                $copyThreadStatus= "copyFail";
                        }
                }

as i said, i don't know nothing about threads. so i have no clue if the example would work. at least it drops an error in my test. but the remaining machines on the host are still skiped. i dind't look into that yet.

Dec 4, 2009 12:31 AM KKrtz says:

Regarding the eMail log...

When the backup didn't completed well could this also be reflected in the subject ??? e.g "Incompete"

Dec 9, 2009 2:38 AM aelus says:

Hi,
I tried the new exciting script, but without success. Both powerOn and powerOff is not functional. Did I make a configuration mistake? The script connects to the ESXi, creates the user but make no snapshot or create the folders in the destination drive (NFS network storage, it's included at the source and vMA ESXi). Any idea? Thanks a lot.

12-09-2009 11:16:04 -- info: ============================== ghettoVCBg2 LOG START ==============================
12-09-2009 11:16:04 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
12-09-2009 11:16:04 -- info: CONFIG - VM_BACKUP_DATASTORE = nfs_storage_backup_35
12-09-2009 11:16:04 -- info: CONFIG - VM_BACKUP_DIRECTORY = vmbackups
12-09-2009 11:16:04 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
12-09-2009 11:16:04 -- info: CONFIG - ADAPTER_FORMAT = lsilogic
12-09-2009 11:16:04 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
12-09-2009 11:16:04 -- info: CONFIG - VM_SNAPSHOT_MEMORY = YES
12-09-2009 11:16:04 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
12-09-2009 11:16:04 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-12-09
12-09-2009 11:16:04 -- info: CONFIG - VM_VMDK_FILES = all

12-09-2009 11:16:04 -- debug: Main: Login by vi-fastpass to: 192.168.3.94
12-09-2009 11:16:04 -- debug: copyTask: Task START
12-09-2009 11:16:04 -- debug: copyTask: waiting for next job and sleep ...
12-09-2009 11:16:07 -- debug: Main: Disconnect from: 192.168.3.94

12-09-2009 11:16:07 -- debug: Main: Login by vi-fastpass to: 192.168.3.38
12-09-2009 11:16:09 -- info: Initiate backup for ASP-VM-ADMIN found on 192.168.3.38
12-09-2009 11:16:16 -- debug: ASP-VM-ADMIN original powerState: poweredOn
12-09-2009 11:16:16 -- info: ASP-VM-ADMIN has 1 VMDK(s)
12-09-2009 11:16:16 -- debug: backupVMDK: Backing up "datastore1 Test4Bkp/Test4Bkp.vmdk" to "nfs_storage_backup_35 vmbackups/ASP-VM-ADMIN/ASP-VM-ADMIN-2009-12-09/Test4Bkp.vmdk"
12-09-2009 11:16:16 -- debug: backupVMDK: Signal copyThread to start
12-09-2009 11:16:16 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
12-09-2009 11:16:16 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
12-09-2009 11:16:16 -- debug: CopyThread: Start backing up VMDK(s) ...
12-09-2009 11:16:19 -- debug: copyTask: send copySuccess message ...
12-09-2009 11:16:19 -- debug: copyTask: waiting for next job and sleep ...
12-09-2009 11:17:16 -- debug: backupVMDK: Successfully completed backup for datastore1 Test4Bkp/Test4Bkp.vmdk Elapsed time: 1 min
12-09-2009 11:17:16 -- warn: WARNING: Snapshot "ghettoVCBg2-snapshot-2009-12-09" not found
12-09-2009 11:17:18 -- debug: checkVMBackupRotation: Starting ...
12-09-2009 11:17:18 -- debug: Unable to purge nfs_storage_backup_35 vmbackups/ASP-VM-ADMIN/GET https:192.168.3.38foldervmbackupsASP-VM-ADMIN?dsName=nfs_storage_backup_35 unsuccessful : 404 Not Found due to rotation max
12-09-2009 11:17:18 -- info: Backup completed for ASP-VM-ADMIN!

12-09-2009 11:17:18 -- debug: Main: Disconnect from: 192.168.3.38

12-09-2009 11:17:18 -- debug: Main: Login by vi-fastpass to: 192.168.3.41
12-09-2009 11:17:19 -- debug: Main: Disconnect from: 192.168.3.41

12-09-2009 11:17:19 -- debug: Main: Calling final clean up
12-09-2009 11:17:19 -- debug: cleanUP: Thread clean up starting ...
12-09-2009 11:17:19 -- debug: cleanUp: Send exit to copyThread
12-09-2009 11:17:19 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit
12-09-2009 11:17:19 -- debug: copyTask: die ...
12-09-2009 11:17:19 -- debug: cleanUp: Join passed
12-09-2009 11:17:19 -- info: ============================== ghettoVCBg2 LOG END ==============================

EDIT:
Found my mistake, the license is missing... But foundation license from vCenter? My ESXi doesn't know, if I have a licensed vCenter, does it? confused

Dec 9, 2009 8:57 AM lamw

says: in response to: aelus

Please refer to the documentation, your ESX(i) host must be fully licensed, you can NOT use a free version. The host can be a standalone or managed by vCenter, but if it's managed by vCenter, then it should also be properly licensed else you will not be able to use this script.

In a future update, I'll probably implement this validation check at the very beginning, I find that not everyone always goes through the documentation on first try.

Thanks

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 18, 2009 10:02 AM boelama says:

hi,

i'm seems to have some "logout" errors after a failed purge of rotated directory.
After backup is finished , directories are rotated and old are purged .
the purge of oldes directory seems to fail, which would lead to a logout of the VIM library.

I have a list of VM's and it happens on different places :
12-18-2009 12:16:05 -- info: ============================== ghettoVCBg2 LOG START ==============================
12-18-2009 12:16:09 -- info: CONFIG - USING CONFIGURATION FILE = ECL110
12-18-2009 12:16:09 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
12-18-2009 12:16:09 -- info: CONFIG - VM_BACKUP_DATASTORE = backup
12-18-2009 12:16:09 -- info: CONFIG - VM_BACKUP_DIRECTORY = ESX
12-18-2009 12:16:09 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
12-18-2009 12:16:09 -- info: CONFIG - ADAPTER_FORMAT = buslogic
12-18-2009 12:16:09 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
12-18-2009 12:16:09 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
12-18-2009 12:16:09 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
12-18-2009 12:16:09 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = WEEKLY
12-18-2009 12:16:09 -- info: CONFIG - VM_VMDK_FILES = all

12-18-2009 12:16:09 -- info: Initiate backup for ECL110 found on esx104.eclipseinternational.com
12-18-2009 12:16:27 -- debug: ECL110 original powerState: poweredOn
12-18-2009 12:16:27 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2009-12-18" for ECL110
12-18-2009 12:16:37 -- info: ECL110 has 1 VMDK(s)
12-18-2009 12:16:37 -- debug: backupVMDK: Backing up "Storage1 ECL110/ECL110 - DC3.vmdk" to "backup ESX/ECL110/ECL110-WEEKLY/ECL110 - DC3.vmdk"
12-18-2009 12:16:37 -- debug: backupVMDK: Signal copyThread to start
12-18-2009 12:16:37 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
12-18-2009 12:16:37 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
12-18-2009 12:16:37 -- debug: CopyThread: Start backing up VMDK(s) ...
12-18-2009 12:31:39 -- debug: backupVMDK: Backup progress: Elapsed time 15 min
12-18-2009 12:46:42 -- debug: backupVMDK: Backup progress: Elapsed time 30 min
12-18-2009 13:01:47 -- debug: backupVMDK: Backup progress: Elapsed time 45 min
12-18-2009 13:16:49 -- debug: backupVMDK: Backup progress: Elapsed time 60 min
12-18-2009 13:31:51 -- debug: backupVMDK: Backup progress: Elapsed time 75 min
12-18-2009 13:46:54 -- debug: backupVMDK: Backup progress: Elapsed time 90 min
12-18-2009 14:01:56 -- debug: backupVMDK: Backup progress: Elapsed time 105 min
12-18-2009 14:16:58 -- debug: backupVMDK: Backup progress: Elapsed time 120 min
12-18-2009 14:32:01 -- debug: backupVMDK: Backup progress: Elapsed time 135 min
12-18-2009 14:47:03 -- debug: backupVMDK: Backup progress: Elapsed time 150 min
12-18-2009 15:02:06 -- debug: backupVMDK: Backup progress: Elapsed time 165 min
12-18-2009 15:17:08 -- debug: backupVMDK: Backup progress: Elapsed time 180 min
12-18-2009 15:32:10 -- debug: backupVMDK: Backup progress: Elapsed time 195 min
12-18-2009 15:44:37 -- debug: copyTask: send copySuccess message ...
12-18-2009 15:44:37 -- debug: copyTask: waiting for next job and sleep ...
12-18-2009 15:45:12 -- debug: backupVMDK: Successfully completed backup for Storage1 ECL110/ECL110 - DC3.vmdk Elapsed time: 208 min
12-18-2009 15:45:35 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2009-12-18" for ECL110
12-18-2009 15:45:42 -- debug: checkVMBackupRotation: Starting ...
12-18-2009 15:46:32 -- info: Backup completed for ECL110!

12-18-2009 15:46:32 -- debug: Main: Disconnect from: esx104.eclipseinternational.com

12-18-2009 15:46:32 -- error: getFinalList: ERROR - Unable to locate VM: ECL131
... a list of

the output during backup shows this error :

ERROR on delete of a old directory

RESPONSE: $VAR1 = '<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<RetrievePropertiesResponse xmlns="urn:vim25"><returnval><obj type="Task">haTask--vim.FileManager.deleteFile-21879</obj><propSet><name>avail
/vmfs/volumes/b416f3a1-85912734/ESX/ECL110/ECL110-WEEKLY--2</localizedMessage></error><reason xsi:type="TaskReasonUser"><userName>vi-admin00
</soapenv:Body>
</soapenv:Envelope>';

followed by a logout

REQUEST: $VAR1 = '<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<Logout xmlns="urn:vim25"><_this type="SessionManager">ha-sessionmgr</_this>
</Logout></soapenv:Body></soapenv:Envelope>
';

after this , no other VM can be backuped. i located the problem to this perl line :

my $vm_view = Vim::find_entity_view(view_type => 'VirtualMachine',filter => {"config.name" => $vm_name});

this lines seems to fail after the logout.
the storage used for backups is a NFS mounted ReadyNas server.

luc

Dec 18, 2009 10:16 AM lamw

says: in response to: boelama

The reason it fails after this disconnect as you know, you're disconnecting from the server, so that line will error out since it can't locate the VM.

I've not seen this before, but it looks like it had issues trying to purge the old files. I know that uses of ghettoVCB see this time to time and generally it's been isolated to their NFS server timing out or having some input/output errors and is generally an issue with the NFS server. You may want to review the vmkernel logs during the time of the backup to get further insight on what happened.

To further isolate this, you can backup w/o rotating to see if you hit this error again and if you don't then its most likely the delete operation thats taking awhile or even commits of snapshots that's causing issues with your NFS. You may also want to review your NFS server logs to ensure you're not saturating the link or resources.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 20, 2009 11:15 PM boelama says: in response to: lamw

Hi William ,

i'll check my ESX server logs & NFS server log to see i can found some more info.
as far i can see , the rotation seems successful . files are moved correctly and 2 rotated backups exist .

luc

Dec 21, 2009 2:01 AM jlorenzo says: in response to: boelama

Hello William,
we are recently discovered your 4 version of ghettoVCBg2 and we are trying to use it from Vma, exists another more recent version than 4? if exists, where can we download it?
We have 3 Esxi Update 3 free version servers and one Esxi update 2 Foundation server, all with Vm´s on the local storage and with a shared NFS datastore in a Windows 2003 Server. We don´t have virtual center.
Our intentions are copy the local storage Vm´s of each server to NFS datastore. First we added all servers to Vma without problems with sudo vifp addserver IP and the password... when we execute the script from Vma "./ghettoVCBg2.pl --vmlist todas", the file todas contain the name of the Vm that we are using to test this proccess (owa in this case).. we obtain this log:

12-16-2009 07:05:22 -- info: ============================== ghettoVCBg2 LOG START ==============================
12-16-2009 07:05:22 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
12-16-2009 07:05:22 -- info: CONFIG - VM_BACKUP_DATASTORE = sntbck
12-16-2009 07:05:22 -- info: CONFIG - VM_BACKUP_DIRECTORY = SntwebservicesTEST
12-16-2009 07:05:22 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
12-16-2009 07:05:22 -- info: CONFIG - ADAPTER_FORMAT = lsilogic
12-16-2009 07:05:22 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
12-16-2009 07:05:22 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
12-16-2009 07:05:22 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
12-16-2009 07:05:22 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-12-16
12-16-2009 07:05:22 -- info: CONFIG - VM_VMDK_FILES = all

12-16-2009 07:05:22 -- debug: Main: Login by vi-fastpass to: 134.134.20.202
12-16-2009 07:05:22 -- debug: copyTask: Task START
12-16-2009 07:05:22 -- debug: copyTask: waiting for next job and sleep ...
12-16-2009 07:05:25 -- debug: Main: Disconnect from: 134.134.20.202

12-16-2009 07:05:25 -- debug: Main: Login by vi-fastpass to: 10.20.222.3
12-16-2009 07:05:27 -- debug: Main: Disconnect from: 10.20.222.3

12-16-2009 07:05:27 -- debug: Main: Login by vi-fastpass to: 10.20.222.2
12-16-2009 07:05:29 -- info: Initiate backup for owa found on 10.20.222.2
12-16-2009 07:05:45 -- debug: owa original powerState: poweredOn
12-16-2009 07:05:45 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2009-12-16" for owa
12-16-2009 07:05:56 -- info: owa has 1 VMDK(s)
12-16-2009 07:05:56 -- debug: backupVMDK: Backing up "datastore1 owa/owa.fm.es.vmdk" to "sntbck SntwebservicesTEST/owa/owa-2009-12-16/owa.fm.es.vmdk"
12-16-2009 07:05:56 -- debug: backupVMDK: Signal copyThread to start
12-16-2009 07:05:56 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
12-16-2009 07:05:56 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
12-16-2009 07:05:56 -- debug: CopyThread: Start backing up VMDK(s) ...
12-16-2009 07:06:01 -- debug: copyTask: send copySuccess message ...
12-16-2009 07:06:01 -- debug: copyTask: waiting for next job and sleep ...
12-16-2009 07:06:56 -- debug: backupVMDK: Successfully completed backup for datastore1 owa/owa.fm.es.vmdk Elapsed time: 1 min
12-16-2009 07:06:59 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2009-12-16" for owa
12-16-2009 07:07:02 -- debug: checkVMBackupRotation: Starting ...
12-16-2009 07:07:07 -- info: Backup completed for owa!

12-16-2009 07:07:07 -- debug: Main: Disconnect from: 10.20.222.2

12-16-2009 07:07:07 -- debug: Main: Calling final clean up
12-16-2009 07:07:07 -- debug: cleanUP: Thread clean up starting ...
12-16-2009 07:07:07 -- debug: cleanUp: Send exit to copyThread
12-16-2009 07:07:07 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit
12-16-2009 07:07:07 -- debug: copyTask: die ...
12-16-2009 07:07:07 -- debug: cleanUp: Join passed
12-16-2009 07:07:07 -- info: ============================== ghettoVCBg2 LOG END ==============================

Everything seems to have gone well, but only the .vmx file is copied to NFS datastorage. In the vsphere console we can see that folder of the destination backup is created, snapshot is created end in the copy file proccess fail with "error general fault cause by file. One of the parameters supplied is invalid", in the forum we have seen that this can be an error with the value of variable $DISK_BACKUP_FORMAT, in our case we have test with all values without best results.
Thanks a lot William
Regards

Dec 21, 2009 8:46 AM lamw

says: in response to: boelama

Something caused it to fail the connection. Again, I would recommend isolating it local VMFS backup to ensure you're not having issues with your NFS server and monitor your NFS server during the period of backups and also monitor your ESXi host.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 21, 2009 9:26 AM jlorenzo says: in response to: lamw

Thanks William,
I'm trying to copy the Vm to another local datastore and I'm getting the same error at the same point, it seems that the problem can not be the NFS server... Create the directories, copy the .vmx file, make snapshot and fail before start to copy..
what can be wrong?
Regards

Dec 21, 2009 10:05 AM lamw

says: in response to: jlorenzo

If you look at my previous reply, the comment was not directed to you.

I'm not sure why it's not working for you .. potentially something might have changed between u2/u3. We'll probably need to manually issue the vmkfstools copy manually from vMA and see if it's actually cloning the disk.

Try this from vMA:

vmkfstools --server <host> --username <username> -i "[datastore] vmfolder/source.vmdk" -d zeroedthick -a lsilogic "[datastore] vmfolder/destination.vmdk"

This is on the assumption your VM is powered off or if it's still on, you need to issue a snapshot before you can clone the disk and see if this makes it to your destination datastore. This should hopefully help figure out what the issue might be.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 21, 2009 11:11 PM boelama says: in response to: boelama

It seems there are some connection errors between ESX server and NFS host.
this what i could find on ESX4 server

vmkernel:Dec 21 12:03:58 esx104 vmkernel: 32:03:43:00.213 cpu6:4112)WARNING: NFS: 277: Lost connection to server 10.1.0.200 mount point /backupesx4, mounted as b416f3a1-85912734-0000-000000000000 ("backup")
vmkernel:Dec 21 12:04:10 esx104 vmkernel: 32:03:43:12.678 cpu1:4397)NFS: 286: Restored connection to server 10.1.0.200 mount point /backupesx4, mounted as b416f3a1-85912734-0000-000000000000 ("backup")
vmkernel:Dec 21 13:10:43 esx104 vmkernel: 32:04:49:45.530 cpu4:4112)WARNING: NFS: 277: Lost connection to server 10.1.0.200 mount point /backupesx4, mounted as b416f3a1-85912734-0000-000000000000 ("backup")
vmkernel:Dec 21 13:13:03 esx104 vmkernel: 32:04:52:05.060 cpu2:4397)NFS: 286: Restored connection to server 10.1.0.200 mount point /backupesx4, mounted as b416f3a1-85912734-0000-000000000000 ("backup")
vmkernel:Dec 22 04:37:36 esx104 vmkernel: 32:20:16:36.130 cpu12:4112)WARNING: NFS: 277: Lost connection to server 10.1.0.200 mount point /backupesx4, mounted as b416f3a1-85912734-0000-000000000000 ("backup")
vmkernel:Dec 22 04:37:39 esx104 vmkernel: 32:20:16:39.680 cpu7:4383)NFS: 286: Restored connection to server 10.1.0.200 mount point /backupesx4, mounted as b416f3a1-85912734-0000-000000000000 ("backup")

It seems to reconnect , as i can browse the datastore.

The NFS server is a ready nas , no special errors there. maybe a network problem , may just a slow NFS server. it's only used for backup purposes.

Dec 22, 2009 3:21 AM jlorenzo says: in response to: lamw

Thanks William,
the same error with vmkfstools to a local datastore or a NFS too..

~~vi-admin@localhost ~~~$ vmkfstools --server 134.134.20.2 --username root -i "datastore1 vMA-ovf-4.0.0-161993/vMA-ovf-4.0.0-161993.vmdk" -d zeroedthick -a LSIlogic "datastore2 vma/vma.vmdk"
Enter password:
Unable to clone virtual disk : General fault caused by file.
One of the parameters supplied is invalid

Dec 22, 2009 6:45 AM lamw

says: in response to: boelama

Yea if even the connection was lost for few seconds could cause issues with the backup, I would probably look into this before continuing, either the host can't handle it or your NAS server can't handle it. You may want to do a search in the general VMTN communities to see if there's any specific configurations you may need to set. I know for some NAS providers, you may need to increase the timeout value, again a search or new post in the general forum may yield better results. This has nothing to do with the script.

Good Luck

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 22, 2009 6:48 AM lamw

says: in response to: jlorenzo

I think the error might be that you're using an invalid key for the adapter "LSIlogic" .... it's case sensitive and the supported strings are: buslogic, lsilogic or ide

Can you please re-try the command on your vMA 4.0 host

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 22, 2009 7:38 AM jlorenzo says: in response to: lamw

Thanks William,
wea are using the correct form, i think, the first lines of the script are:

###############################################################

NAME OF THE BACKUP DATASTORE

###############################################################

my $VM_BACKUP_DATASTORE = "Sntback";

###############################################################

BACKUP DIRECTORY NAME ON DATASTORE

###############################################################

my $VM_BACKUP_DIRECTORY = "pruebasvlan1";

####################################################

Number of backups for a given VM before deleting

####################################################

my $VM_BACKUP_ROTATION_COUNT = "3";

###################################################

Supported backup types
'zeroedthick' 'eagerzeroedthick' 'thin' '2gbsparse'

###################################################

my $DISK_BACKUP_FORMAT = "zeroedthick";

###################################################

Supported adapter types
'buslogic' 'lsilogic'

###################################################
my $ADAPTER_FORMAT = "buslogic";

###################################################################################################

Shutdown guestOS prior to running backups and power them back on afterwards
This feature assumes VMware Tools are installed, else hard power down will be initiated
1=enable, 0=disable (disable by default)

###################################################################################################

my $POWER_VM_DOWN_BEFORE_BACKUP = "0";

##############################################################

VM BACKUP DIRECTORY NAMING CONVENTION (default -YYYY-MM-DD)

##############################################################

my $VM_BACKUP_DIR_NAMING_CONVENTION = timeStamp('YMD');

###################################################

VM Snapshot Memory & Quiesce
1=enable, 0=disable (disable by default)

###################################################
my $VM_SNAPSHOT_MEMORY = "0";
my $VM_SNAPSHOT_QUIESCE = "0";

################################################

LOG LEVEL VERBOSITY : "debug" or "info"

################################################

my $LOG_LEVEL = "debug";

DO NOT MODIFY PAST THIS LINE ##########################

We have tested on NFS datastores and local, with all vmdk format...
Any idea?

Dec 22, 2009 8:25 AM lamw

says: in response to: jlorenzo

You did not carefully read my last comment. I know it's a little hard to follow the threads in the comments section, but I do expect that users read through my replies carefully and answer all questions posed.

The comment referred to your manual execution of the backup:

Your execution:

vmkfstools --server 134.134.20.2 --username root -i "[datastore1] vMA-ovf-4.0.0-161993/vMA-ovf-4.0.0-161993.vmdk" -d zeroedthick -a LSIlogic "[datastore2] vma/vma.vmdk"

and I'm saying that your *-a" (adapter) format is incorrect. It should be lsilogic and NOT LSIlogic

Try this command on vMA:
{code}

vmkfstools --server 134.134.20.2 --username root -i "[datastore1] vMA-ovf-4.0.0-161993/vMA-ovf-4.0.0-161993.vmdk" -d zeroedthick -a lsiogic "[datastore2] vma/vma.vmdk"

The other question I have about the VM you're trying to clone is it looks like it's the default vMA 4.0 VMDK that's downloaded from VMare's site? If that is the case, the VMDK may not be supported format directly on ESX, generally you'll need to import which does a conversion OR using something like VMware converter.

I would recommend you use an actual VMDK that is fully functional or if you want to do testing, create a dummy VM on the ESXi host itself.

Please report back on the comments made from above

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 22, 2009 8:59 AM jlorenzo says: in response to: lamw

Tnaks again William,
the Vm we are trying to clone manually was imported from the .OVF of vmware and it was working fine. We turn off the Vm and test to clone it.. We don´t have more Vm´s that we can turn off..
The previous LSIlogic was a copy/paste failure, i had written lsilogic really..
I run it again with the same result:

~~root@localhost sntvlan1~~# vmkfstools --server 134.134.20.2 --username root -i "datastore1 vMA-ovf-4.0.0-161993/vMA-ovf-4.0.0-161993.vmdk" -d zeroedthick -a lsilogic "datastore2 vma/vma.vmdk"
Enter password:
Unable to clone virtual disk : General fault caused by file.
One of the parameters supplied is invalid

On all our Esxi server, we have open SSH and telnet to run GhettoVCB with Plink from Windows, any problem with this?
Thanks

Dec 22, 2009 10:05 AM lamw

says: in response to: jlorenzo

You're executing this on vMA 4.0 right? and not on ESXi?

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 22, 2009 11:45 AM jlorenzo says: in response to: lamw

Of course William,
i am executing this on Vma 4.0, not on Esxi.. But all servers managed by this Vma are Esxi 3.5 U2 and U3. I am using the same Vma to run vmwarevSphereHealthCheck.pl without any problem
Thanks

Dec 22, 2009 11:55 AM lamw

says: in response to: jlorenzo

That's quite odd ... basically what we need to do is to be able to issue the vmkfstools clone from vMA 4.0 ....if you can't manually do that, then there's no reason to troubleshoot further into the backup script. I'm just wondering if it's the VM that you're trying to clone (which I agree should not have issues as I've done clones of vMA many times) OR there's something funny that's going on in your environment. Once you can do this, then you should be able to use the script w/o issues.

If you have a lower priority system, you can take a snapshot fo the VM and see if you can clone, if you can start the clone process w/o issues. You can ctrl+c the clone and remove the snapshot and commit it back.

This is what we need to do as next steps, else I won't be able to help more.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 23, 2009 12:37 AM jlorenzo says: in response to: lamw

I created a new vm with a small 2 Gb disk and the result is the same, i tried with absolute and relative paths..

~~root@localhost tmp~~# vmkfstools --server 134.134.20.2 --username root -i "datastore1 /vmfs/volumes/4975853e-4f77538d-9ce8-002219860e82/test/test.vmdk" -d zeroedthick -a lsilogic "datastore2 /vmfs/volumes/493fc889-4d865435-e8af-002219860e84/prueba/prueba.vmdk"
Enter password:
Unable to clone virtual disk : General fault caused by file.
The system cannot find the file specified

Dec 23, 2009 6:45 AM lamw

says: in response to: jlorenzo

Sorry you're doing something wrong or you're missing something.

When you're issuing the clone operation, it accepts the following format:

"[datastore] vmfolder/vm.vmdk"

Please take screenshots of the available datastores on your ESXi host and also on the VM that is to be cloned, click into it's configuration and select the Hard Disk and take a screenshot, to the right of that, I should see a box containing the diskFile path to the VMDK

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 23, 2009 11:18 AM fahrer says: in response to: lamw

just to let u know, i've seen the same thing as jlorenzo with a bunch of esx 3.5 u3 hosts.
different vms, newest vma, it was all correct... i could not get rid of the problem.
i didn't invetigate a lot of time. we just decided to try again after an upgrde to vphere 4...

Dec 23, 2009 1:48 PM lamw

says: in response to: fahrer

Have you guys tried it with VIMA 1.0? I know there's been significant amount of API changes and it would not surprise me if there might be something funky going on. The last time I tested against ESX 3.5 was on Update4 which worked using vMA 4.0 (unfortunately the testing was very limited) and I don't have additional resources to host any 3.5 systems.

I agree, it's definitely better to move to vSphere, there are huge benefits but I can see why some may want to stay on 3.5 ...specifically on U2/U3

I think for those that don't want to move to ESX(i) 4.0, a better backup system is probably using ghettoVCB.sh

Thanks for the comments

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 28, 2009 1:35 AM jlorenzo says: in response to: lamw

It´s possible use ghettoVCB from Vma?, we are using it from Windows with plink..
We are not using vsphere version for hardware requirements that are not met.
Thanks

Dec 28, 2009 10:44 AM lamw

says: in response to: jlorenzo

No, ghettoVCB runs on either the ESX Service Console or ESXi unsupported console. Please go through ghettoVCB.sh - Free alternative for backing up VM's for ESX(i) 3.5 and 4.0+ documentation carefully for usage. You can however issue the backup from a Windows system using plink, again this is referenced in the 'additional info' in the documentation.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Dec 29, 2009 3:33 AM jlorenzo says: in response to: lamw

We are using ghettovcb from Windows with plink, but we want to improve performance with ghettovcbg2 from Vma 4 without success at this time.
Could you see the screenshots?

Dec 30, 2009 12:54 PM lamw

says: in response to: jlorenzo

I've not really tested performance between the two scripts, with g2, you're utilizing the API's to make the backup which might be slightly slower than issuing the commands from the Service Console of ESX or unsupported busybox console with ESXi. Performance is always an iffy subject and much of it will depend on the setup, you may or may not get better performance. The real point of g2 is to provide similar backup functionality of ghettoVCB without the require the use of SSH console, more importantly on ESXi where this console is unsupported.

As mentioned earlier from another user, you may or may not see issues working with u2/u3 ... I would recommend either using VIMA 1.0 and see if you still have these issues or moving to ESXi 4.0 (though this script will not work w/o proper licenses)

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Jan 4, 2010 5:33 AM boelama says: in response to: boelama

i currently fixed the issue by adding this function and
relogin on each backup iteration

sub loginESX {
&log("info", "loginESX: Login by vi-fastpass to: " . $host);
VIFPLib::login_by_fastpass($host);
my $viuser = vifplib_perl::CreateVIUserInfo();
my $vifplib = vifplib_perl::CreateVIFPLib();
eval { $vifplib->QueryTarget($host, $viuser); };
if(!$@) {
$host_username = $viuser->GetUsername();
$host_password = $viuser->GetPassword();
}
}

sub backUpVMs {
my @vm_backup_list = @_;
my $vm_backup_dir;
my $snapshot_name;
my $original_vm_state;
my $returnval;

my $count = 0;
foreach my $vm_name (@vm_backup_list) {
if($success_backups{$vm_name} ne -1 && $success_backups{$vm_name} ne 1) {
loginESX(); <==== ADDED
my $vm_view = Vim::find_entity_view(view_type => 'VirtualMachine',filter => {"config.name" => $vm_name});
if(defined($vm_view)) {

this way each backup start with a login into the ESX server, and failed connections are restored.
this is now running since 23/12/2009 without any problem. before i couldn't run it 2/3 times without getting errors.

luc

Jan 4, 2010 11:45 AM lamw

says: in response to: boelama

Interesting .... I'm wondering if this only affects ESX(i) 3.5 hosts? This is the first time I've heard of this where you may have a potential connection issue as this handled automatically with the most updated version of the script. Good to hear you've found a work around, let me know if this does fix the issue permanently.

Thanks for the info

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Jan 5, 2010 4:08 AM jlorenzo says: in response to: lamw

With VIMA 1.0 we obtain identical results, the .vmx file is copied in the target datastore, the snapshot is created and after one minute is deleted without copy it to the target..
Any idea?
Thanks

Jan 5, 2010 8:42 AM lamw

says: in response to: jlorenzo

From all the comments/replies, it sounds like you cannot use either VIMA 1.0 or vMA 4.0 to clone a VMs VMDK that's running on a supposedly ESX(i) 3.5u2/u3 hosts? I can see why the .vmx gets copied because it uses vifs which is basically utilizing HTTP to do the transfer, if you're using the free version of ESXi and you're not on ESXi 3.5u2 or u3, then the APIs are locked down.

I remember asking you much earlier in the initial replies what version and you stated you were running ESXi 3.5 u2 or u3, can you please confirm that is the case and you've not updated or patched? This would make sense if you're outside of either of those builds causing you not to use the script as it requires a properly licensed version of ESXi.

Please confirm

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Jan 7, 2010 4:00 AM jlorenzo says: in response to: lamw

I am using ESXi 3.5 Update 2 Foundation version in one server and Free Esxi 3.5 Update 3 in another 3 servers without any additional patch or updates.
Thanks

Jan 12, 2010 7:12 AM jlorenzo says: in response to: lamw

William, I have read the information about the license requirement to use this script, but i still have doubts..
what is the minimum required license to run ghettoVCBg2 in a Esxi 3.5 Update 4 or 5? Foundation license no longer exists.. and in a Vsphere 4 server?.. i suppose that Standard license is valid like the essentials.. if it isn´t valid, what version of essentials it´s necessary, normal or plus?
Ghettovcbg2 uses vstorage APIs for backups?
Thanks

Jan 12, 2010 8:32 AM lamw

says: in response to: jlorenzo

Basically any license for your ESXi host that is NOT the free version will work. I know VMware has changed and updated additional license types and it can get confusing.

Also, no, the script does not use the vStorage API which is basically a marketing term from VMware. vStorage API is composed of vSphere API (which ghettoVCBg2 uses) + VDDK (which I don't make use of).

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Jan 15, 2010 5:25 AM jlorenzo says: in response to: lamw

In Esxi (free license and Foundation license) update 2 and 3, is not possible for me to use different disk format than thin with this script.. with zeroedthick fails, and my vm´s has the default disk format (zeroedthick) so if i use thin format in case of having to restore i will to convert the format disk again to zeroedthick..
With large size files often fails, any workaround or new version available to correct this?
Thanks

Jan 15, 2010 6:48 AM lamw

says: in response to: jlorenzo

Remember the ESXi 3.5u2/u3 functionality is more of a hack due to a VMware bug and also note that their were some fixes in vmkfstools script pre vSphere. This script is primarily supporting licensed version of ESX(i) and I'm not sure why you're running into these issues but it's working for the majority of the users. I'm thinking this is more specific to u2/u3 builds than the script. I would recommend going to ESXi 4.0 as it does provide further stability and feature enhancements.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Feb 2, 2010 9:08 AM biekee says: in response to: lamw

Thanks for the script, works well towards ESX datastores.
Ideal would be, if the vm files would be written towards a cifs share, in my case on mount point /mnt/backup.
Problem is, that nothing happens. The log says:
02-02-2010 17:54:03 -- info: ============================== ghettoVCBg2 LOG START ==============================
02-02-2010 17:54:03 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
02-02-2010 17:54:03 -- info: CONFIG - VM_BACKUP_DATASTORE = /mnt/backup
02-02-2010 17:54:03 -- info: CONFIG - VM_BACKUP_DIRECTORY = backups
02-02-2010 17:54:03 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
02-02-2010 17:54:03 -- info: CONFIG - ADAPTER_FORMAT = buslogic
02-02-2010 17:54:03 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
02-02-2010 17:54:03 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
02-02-2010 17:54:03 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
02-02-2010 17:54:03 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2010-02-02
02-02-2010 17:54:03 -- info: CONFIG - VM_VMDK_FILES = all

02-02-2010 17:54:09 -- info: Initiate backup for <VMNAME> found on <SERVERNAME>
02-02-2010 17:54:31 -- info: <VMNAME> has 1 VMDK(s)
02-02-2010 17:55:37 -- info: Backup completed for <VMNAME>!

I know the mount is there, mount through mount -t cifs -o, username=.....etc.

Any ideas about that? Or are only ESX Datastores supported?

BK

Feb 2, 2010 9:16 AM lamw

says: in response to: biekee

Only valid ESX(i) datastores are supported in the form of FC/iSCSI LUN or NFS Datastore ... even though cifs can be mounted, it's not understood by the APIs.

If you want to backup to cifs share that is mounted on your ESX host, you may want to look at ghettoVCB.sh - Free alternative for backing up VM's for ESX(i) 3.5 and 4.0+ which is the same script but using the Service Console for classic ESX or unsupported Busybox console for ESXi. Since this run's on one of the consoles, you just need to specify a valid path.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Feb 4, 2010 8:17 AM biekee says: in response to: lamw

I solved it by moving all te files to the cifs share in a seperate action.
Thanks for the script!
BK

Feb 4, 2010 8:58 AM lamw

says: in response to: biekee

np.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
Twitter: @lamw
vGhetto Script Repository
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
VMware Developer Comuunity

If you find this information useful, please award points for "correct" or "helpful".

Search VMware Communities Advanced Search

Actions

View as PDF

Incoming Links

More by lamw

Up to Documents in vSphere Management Assistant (vMA)

ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Created on: Apr 22, 2009 9:36 PM by lamw - Last Modified: Nov 18, 2009 7:47 PM by lamw

Table of Contents:

Description:

Features

Requirements:

ESX/ESXi Version Support Table

Setup:

Addings ESX/ESXi host(s) to VIMA/vMA

Configurations

Usage:

Sample Execution

Backup VMs located in a list

Backup VMs based on individual VM backup policies and log output to /tmp/ghettoVCB.log

Dry run Mode (no backup will take place)

FAQ

Change Log

Actions

Incoming Links

More by lamw

Post a comment

Communities

Community Resources

VMworld

Get Support

Other Links

ghettoVCBg2 - Free alternative for backing up VMs in ESX(i) 3.5 and 4.x (no SSH console required!)

Created on: Apr 22, 2009 9:36 PM by lamw - Last Modified: Nov 18, 2009 7:47 PM by lamw

Table of Contents:

Description:

Features

Requirements:

ESX/ESXi Version Support Table

Setup:

Addings ESX/ESXi host(s) to VIMA/vMA

Configurations

Usage:

Sample Execution

Backup VMs located in a list

Backup VMs based on individual VM backup policies and log output to /tmp/ghettoVCB.log

Dry run Mode (no backup will take place)

FAQ

Change Log

Actions

Incoming Links

More by lamw

VMware Beta Programs

VMware Developer

Developer Sample Code

VMworld Sessions & Labs

Purchase PSO Credits Online

Community Hardware Software

Only VMware ... Delivers Nexus 1000V

Post a comment

Communities

Community Resources

VMworld

Get Support

Other Links

Developer
Sample Code

VMworld
Sessions & Labs