ESXi Backup Script using RCLI

[KBuchanan]

There are a lot of discussions about the performance of ESXi backups using scripted backup solutions. Most people (including me) have posted questions about the performance over the VMKernel management interface. It would appear that the IO bandwidth has been capped at some value by VM software engineers...that is speculation since no one has confirmed it...but everyone's testing seems to indicate there is some truth in it.

HOWEVER...I have a script that is allowing me to get an average of 50MB/sec throughput. The script will create a log file with timed entries that will allow you to calculate the time require to complete the transfer and subsequently you can calculate the transfer speed based on the size of the *.vmdk and *.vmx files that were transferred.

So - here, in a nutshell, is the setup/configuration:

1. Configure all ESXi, NFS server NICs, and network switches to be 1000/Full-duplex. (I don't rely on auto-negotiation.)

2. Install the VMWare RCLI on a Windows system where you run the backup as a scheduled task. (URL: )

3. The RCLI will (by default) install in this folder: "c:\Program Files (x86)\VMware\VMware VI Remote CLI\bin"

4. Download the plink.exe file and install it in the "c:\Program Files (x86)\VMware\VMware VI Remote CLI\bin" folder. (URL: )

5. One of the perl scripts in the "c:\Program Files (x86)\VMware\VMware VI Remote CLI\bin" folder was giving me an error about not finding "chcp.com". This file is a windows DOS file that displays the "active code page number". The "chcp.com" file should be in the folder: "c:\windows\sytem32". I copied this file to the "c:\Program Files (x86)\VMware\VMware VI Remote CLI\bin" folder even though the environment path included the system32 path.

6. I have attached my scripts - use them at your own risk!! They work VERY WELL for our environment - but I'll give the typical disclaimer: I make absolutely no guarantee if they will work for your environment and will not guarantee their reliability. (Open the files in an editor - ie, notepad - and be sure you understand what is happening in the scripts).

7. The "runbackup.bat" file is the file that you will schedule to run on a periodic basis. We run it daily at 11pm. (There is a performance hit; we have run it during normal business hours and it doesn't seem to affect end-users at all.)

8. The "runbackup.bat" will pass the FQDN (or IP address) of the ESXi server, root username, and root password into the "backupscript2Celerra.bat" batch file.

9. The "backupscript2Celerra.bat" file is where all the work is done. You will need to modify the variables in this script. We are creating 2 copies of the VMImage...one is on local storage and the other is on a remote NFS server. If you aren't using a second location, you will have to modify the script.

10. The script has commands to perform a shutdown of the VM before it clones it and then start it up after the copy is completed...but we don't use it - so these commands are "commented-out" in the script.

11. When the script runs, it will create a log file. (This is defined in the "runbackup.bat" script by piping the output of backupscript2Celerra.bat to a file.)

These scripts will take a snapshot of the VM and then create two cloned images. It will keep 2 copies of the images on local storage and on remote NFS storage. I'll be glad to try to answer any questions about it...but you need to have a novice understanding of DOS batch and UNIX scripting.

I have renamed my scripts as text files...you will have to rename them to *.bat files. If you have to ask what this means...you better leave them and walk away!

Here is a previous posting of my performance measurements: http://communities.vmware.com/message/1168198#1168198

Good luck!

-Kevin

[brichrist]

Kbuchanan,

Thank you for posting this info. I think I understand what is going on however I am new to RCLI. I edited the variables in both these bat files, but when I run the script for some reason I get a few errors. Could you help me out a bit please?

First, when your script calls Plink.exe it gives me errors creating the directories...which is wierd because if I cut and paste the exact same command into a cmd windows plink.exe will create the directories without any problems. Its only errors when I run it from a .bat file.

My second problem is when it tries to run the perl commands I get "perl" is not recognised as internal or external command. Do I have to download a perl program for my XP box. I figured Perl would be included in the RCLI installer.

[KBuchanan]

Regarding Perl:

Check that Perl is the PATH environment variable. It should install in "c:\Program Files\VMware\VMware VI Remote CLI\Perl" directory by default.

If the script gives you a problem, you can see what variables are passed into the script by removing or commenting the "@echo off" command. By doing this, you can see all of the comands...it is very verbose, but it will show you what is happening.

This is a bit of cat-and-mouse play because you are mixing some DOS and LINUX scripting in the batch file. I working on an updated version of the backup script...it just does a few "extra" things that I wanted and has some extra variables to help with configuration.

[Vishy1]

Hi There,

Thanks for the script. I'm looking for a backup script that are DRS aware and will work in a VM cluster enviroment. Haven't found one yet.

Thanks,

Vishy

[lamw]

VMware vCenter Data Recovery will have this ability, it won't be free but it'll have some nice capabilities when it's released

http://vmware.com/products/vcenter-data-recovery/

=========================================================================

--William

VMware ESX/ESXi scripts and resources at:

[tailwindALWAYS]

KBuchannan,

Thanks for the script! I'm excited to test it out. I saw similar performance issues that are "rumored" and it pretty much made me look into other solutions. Now I'm excited to test this out again!

A few questions before I dive into them too much ...

• Why did you create a script that just calls a script?

• If the password has any special chars in it, do they have to be negated somehow? For instance in my script (perl) I had to put a "\" before any special char.

Thanks again!

[KBuchanan]

The second script calls the first script JUST to get the output piped to a file. In my case, I wanted to create a file that had a time stamp in the file name...so, I used another script to call the backup script.

Plus, the script is "portable". I just change the server name, root, and pw in the "Run_XXX.bat" file and it is ready to go. BUT...the key is consistency. There are no spaces in any of the VM names. In my environment, I am running ESXi and the images are hosted locall. All the datastores (locally) are named with the same naming convention, and all the images are backed up to the same backup datastores.

You can call the backup script and pipe the outputput from the command line...but it was just easier/cleaner and more intuitive to see the calls to the backup script in my opinion to separate it.

Regarding the password - that's an good point and intersting question, but I havne't tried it. We have a long passwords using alpha/numberic. We may (or may not) use some punctuation - but so far I haven't had any problem.

I'm going to post another revision of the script. I'll post the revisions also.

[tailwindALWAYS]

The second script calls the first script JUST to get the output piped to a file. In my case, I wanted to create a file that had a time stamp in the file name...so, I used another script to call the backup script.

OK, I figured that out (assumed that) as soon as I hit submit.

Plus, the script is "portable". I just change the server name, root, and pw in the "Run_XXX.bat" file and it is ready to go. BUT...the key is consistency. There are no spaces in any of the VM names. In my environment, I am running ESXi and the images are hosted locall. All the datastores (locally) are named with the same naming convention, and all the images are backed up to the same backup datastores.

I'm just stepping through the script via CMD prompt (manually typing in some of the commands) to not ASSuME what it's doing. When I run the following command I keep getting "Access Denied"

plink.exe root@servername.domain.com -pw mypassword ls /vmfs/volumes/vmachines/

I then tried doing an ls on /vmfs/volumes/ to make sure the name was set right and got the same response. Any idea what I'm doing wrong off hand?

I'm going to post another revision of the script. I'll post the revisions also.

Cool! So are you saying I should wait for that? Any time frame? Thanks again for helping us all out!!!

[KBuchanan]

I'll try to have the revised script posted tonight - that will give me time to update some "documentation comments" in the script.

Kevin

[KBuchanan]

Sorry for the delay...I reformatted the script and move this to another discussion:

I changed some of the logging to better suit what I wanted to see. ...but feel free to modify and mutilate it as you want to!

Good luck with it!

[rcpop]

Hi KBuchanan ,

I setup a ESXi 4.0 Machine and setup XP for backup the VM.

When I use RCLI to run the batch file on XP , it have error message :

run this line with Error1 :

perl vmware-cmd.pl -H %ESXHOST% -U %USER% -P %PASS% %DATASTOREPATH%/ihaemail/ihaemail.vmx createsnapshot "SS1" "Nightly

Error1 :

Snapshot" 1 1

Fault:

SOAP Fault:

-

---

Fault string: fault.RestrictedVersion.summary

Fault detail: RestrictedVersionFault

run this line with Error2 :

perl vmkfstools.pl --server %ESXHOST% --username %USER% --password %PASS% -a lsilogic -d thin -i %FIRSTCOPYPATH%/ihaemail/ihaemail.vmdk %SECONDCOPYPATH%/ihaemail/ihaemail.vmdk

perl vmkfstools.pl --server %ESXHOST% --username %USER% --password %PASS% -a lsilogic -d thin -i %FIRSTCOPYPATH%/ihaemail/ihaemail_1.vmdk %SECONDCOPYPATH%/ihaemail/ihaemail_1.vmdk

Error2:

unable to clone virtual disk : fault.RestrictedVersion.summary

Could you tell me hoe to solve this problem?

Thanks !!!!!!!!

[js-hacki]

the RCLI in the recent ESXi version is now limited. Search the forum, you will find the answer.

Take a look at the ghettoVCB Backup script