Re: Free ESXi Backup Solution for Windows - Page 17

ChrisInTexas · ‎08-20-2008

I have spent the last few days trying to find a free backup solution to the newly free ESXi for windows only enviroments (in particular Windows XP). The solution for me was the following:

1. Installing Windows Services for UNIX (WSFU)

2. Copying the ESXi Server password and group files to Windows

3. Configuring WSFU for accepting ESX Server connections

4. Sharing the Windows folder for NFS compatibility

5. Configuring the ESXi Server to mount the Window NFS Share as Datastore.

6. Setup Backup Script

Attached is the complete steps.

I take NO credit for any of this. This is just a complation of others work formated to suit my needs and felt others could benift from it as I have.

by Jason Mattox from Vizioncore (direct copy of his work, I just added more information to make it work in Windows XP)

(NFS Server port information)

by robink (The backup script)

(ssh on ESXi)

HarisB · ‎01-12-2009

I just enabled ENABLE_2GB_SPARSE and now backups are a bit smaller, but still not where they should be : I have a 8GB vmdk, with 4 GB free inside VM, but backup is still at 5.8GB (better than 8GB thick but still above what's really used)

Looking forward to gziping this on the fly...

lamw · ‎01-12-2009

You may want to review over the different type of disk types: http://sanbarrow.com/vmdk/disktypes.html

2gb sparse just splits the disks from your source into individual 2gb size files that equate back to your original VMDK

(e.g)

Source VMDK = 8gb

Destination = 4x2gb

This is primary used to support legacy applications or system that can not exceed the 4gb file limit.

=========================================================================

--William

VMware ESX/ESXi scripts and resources at:

HarisB · ‎01-12-2009

Yes but these 2GB files are not full - on backup I'm getting *.vmdk to *.S005-vmdk or something so I have 6 VMDK files (even though original VMDK is 8GB), but all 6 exported files together are in the region of 5.8GB or so, still more than what's really used within the VM.

I also tried gzipping all those files, I get 3.5GB archive, which is a bit better. I know VCB would make this into under 2GB archive, I've had 70+ VM backed up with VCB 1.0.1 and few of them were over 2GB, and there were all sorts of functions there. How does VCB compress / export data? Any chance we can use the same method? 5.8 beats 8 but 1.3 sounds much better

HarisB · ‎01-12-2009

Right - another thing I wanted to bring up - if we can't use vmware-cmd on esxi, how about we do crude "find -name *.vmx" from /vmfs/ and capture all VMX files on all datastores, then use that as input, filtering out exclusions? We can probably do some sanity check against the list of registered VMs on the host to ensure we are not backing up ghosts?

lamw · ‎01-12-2009

This work on ESXi, if you enable the unsupported SSH console

~
# find /vmfs/volumes/ -name *.vmx

You can also use RCLI's version of vmware-cmd.pl to get a listing or VI Perl Toolkit vminfo.pl. There are many many ways of accomplishing this task, depending on what you need to do or with what toolset.

btw, there is a sanity check occurring when a user provides the VM input list. It will try to sanitize the input file such as removing blank lines or not process any VMs that it can not find under the given ESX/ESXi host.

=========================================================================

--William

VMware ESX/ESXi scripts and resources at:

aelus · ‎01-12-2009

lamw & harrisb,

thanks for your help, but nor the execution with debug information either the flag at DEVEL_MODE gave usefull information to me. You can find the debug info attached. I also checked the vmlist for ^M (I had this several weeks before and even created a complete new list with vi, no change. I use ESXi 3.5 Update 3. Can I give you any further information, propably it's only a tiny damn failure

ESXiaddicted · ‎01-13-2009

hi lamw

first, congratulations for your script wich I look after since weeks today ...

I actually try the backup way of my ESXi solution, and your script works really fine for this

But I have a little disappointment, concerning the VM's state when the backup script occurs. Indeed, if it's ON, it's turned OFF and the backup is done, and then the VM is turned back ON.

But in my case, there are some VMs which are rarely powered on, and I regret that the state before backup isn't stored and every VM backuped is always turned ON at the end of script.

I think my problem is really poor concerning the improving of this script but it will be more user-friendly if you store the state before the backup to keep back the VM in.

Greetings

lamw · ‎01-13-2009

Looking at your debug output, it shows it's unable to locate the VmId for the specific VM, I think we had this finding the last time you posted. This script should be executied on the ESX or ESXi host containing the VM you're looking to backup, if the host is not managing the VM or the VM is not registered to that given host, it will fail to backup.

One easy way of checking is logging in using the VI Client and ensuring the ESXi host is managing the VM in question or you can also run the following command which is how I locate the VM and extract it's VmId:

vim-cmd vmsvc/getallvms

You can either redirect the output to a file or just search for your VM.

Hopefully this clears some things up.

=========================================================================

--William

VMware ESX/ESXi scripts and resources at:

jlorenzo · ‎01-14-2009

Hi mvalpreda,

I have the same problem:

2009-01-14 17:36:42 -

Copy VM files to local storage -

GET unsuccessful : 500 Internal Server Err

Could you fix it??

Thanks

fdhd · ‎01-14-2009

Thanks Eng. Lamw for the great script.

how to restore - the splited VMDK into 2GB sparse files - back to the esxi server?

lamw · ‎01-14-2009

If you use the normal clone disk flag, it will reassemble the sparse files.

vmkfstools -i <src.vmdk> <dst.vmdk>

=========================================================================

--William

VMware ESX/ESXi scripts and resources at:

BeNrOo · ‎01-16-2009

Hi !

I want to know if its possible to save Vm's of a server1 and copy them to a server2 if the server1 will be sick

Sorry for my english but if you don't understand me, say me!

i use the ESXi free and i've activate the SSH

is it possible to do that with snapshot? and then transfer them to the second server? and frequentialy?

Tx

fahrer · ‎01-17-2009

hi,

check out http://communities.vmware.com/message/1146868#1146868

it actually for linux, but since i'm using putty and perl, it should also run under windows. just change the path to the binaries

fdhd · ‎01-19-2009

Dear Eng William 'lamw'

I use many Virtual Hard disk ' vmk ' files that is :

linked to the Virtual Machine 'vmx'

Are not located on the same Virtual Machine 'vmx' folder

Are located on other DataStore than Virtual Machine 'vmx'

your great script ' ghettoClone.sh' did not backup linked - Virtual Hard disk ' vmk ' files -

Is ther is any resulation to my problem ?

thanks

lamw · ‎01-20-2009

I see the issue with the script, there is an assumption that all VMDK(s) will be stored in the same directory as the VM. Even though it's allows you to store the VM disk files on other datastores, in terms of management it can be pretty difficult if you don't remember certain VMs have their disks spread across datastores.

You can apply the following changes to the code during the backup process to check whether the VMDK is residing locally within the VM directory of if it's in another datastore and set the appropriate variables. I've only done very limited testing, prior to using this in production, please do go and validate thoroughly.

for j in ${VMDKS};
do
		VMDK="${j}"
		#check to see if the VMDK is local or on another datastore
		echo "${VMDK}" | grep "^/vmfs/volumes" > /dev/null 2>&1
		if [ $? -eq 0 ]; then
		        DEFAULT_VMX_DIR=${VMX_DIR}
		        VMX_DIR=$(echo ${VMDK%*/*.vmdk})
		        VMDK=$(echo ${VMDK##*/})
		fi
		
		if [ ${ENABLE_2GB_SPARSE} -eq 1 ]; then
		        vmkfstools -i "${VMX_DIR}/${VMDK}" -d 2gbsparse "${VM_BACKUP_DIR}/${VMDK}"
		else
		        vmkfstools -i "${VMX_DIR}/${VMDK}" "${VM_BACKUP_DIR}/${VMDK}"
		fi
		VMX_DIR=${DEFAULT_VMX_DIR}
done

Now, even though this will allow you to backup your VMs that may have their disks spread across multiple datastore, it will still be your responsibility to ensure on recovery the backup is either restored as before or you'll have to manually edit the .vmx file on where the VMDK is located. In my opinion it's best not to separate your disk files, if you have VMotion license, I would suggest using SVMotion to migrate the VM's disk back to the VM directory or allocate a larger LUN and Storage VMotion your VM to the new LUN.

Any ways, good luck.

=========================================================================

--William

VMware ESX/ESXi scripts and resources at:

oalkatib · ‎01-21-2009

Hi Guys,

I have been sifting through the 23 pages in this thread, and I have come across different backup solutions prvided from snabe, kchawk, and other.

It seems like everyone has gotten the scripts to work exceprt for me. I have tired everything within my n00bish powers to get snabe's script, or Kchawk's scriptsto work.

I would like to schedule a nightly snapshot of running VM's on my ESXi server to another Windows Server that has lots of disk space.

Here is what I have Done:

- Enabled SSH on the ESXi Server

- Installed Active Perl on my desktop

- and Installed VI Remote CLI

- I have edited the files to my settings

But I cant get the snapshotiing to work properly.

Can someone please help me get the following script by kchawk, a tutorial of some sort, please HELP!!

I feel like I'm paralized!

-

sanbe, script works great. I have added to it to allow the tar gzip from gnuwin for compression. Do with it what you will.

Working on the restore script now.

I wrote a php script using the rcli on linux. It gets the list of vms

and files, creates snap if needed and backups the files, then

removes snaps.

I have tested on three esxi servers that have anywhere from two vm to three.

Message was edited by: kchawk

Attachments:

(5.0 K)
(1.8 K)

kchawk · ‎01-21-2009

oalkatib, I don't use the php script anymore myself, the vifs speed was killing me. If you are talking about esxibak.sh, that one I use everynight.

Let us know a little more about the errors you are getting about the snapshots and maybe someone can help.

oalkatib · ‎01-22-2009

Alright, so I grabbed the esxibak.sh, and put in my parameters, and I'm getting the following error:

-

C:\Program Files\VMware\VMware VI Remote CLI\bin>vmware-cmd.pl -H 10.170.250.**2 -U root -P ****** -l

Error connecting to server at 'https://10.170.250.**2:443/sdk':

500 Connect failed: connect: Unknown error; Unknown error

-

Now I have noticed two things

1- This location on my server does not exist:

In particular the /sdk portion, is that something I need to install, or upload via winscp????

Please bear with me, and thank you for the quick response!!!

kchawk · ‎01-22-2009

Try

Quick edit. I am using the attached script to backup from datastore to another datastore, not the perl script, although it should work fine.

oalkatib · ‎01-22-2009

(Edit: Yes we are using the script you attached above)

Alright is not accessible via browser, and it does not exist.

We where able to find the file, and here is what we found

-

~ # find / -name "sdk"

/usr/lib/vmware/hostd/docroot/sdk

~ # cd /usr/lib/vmware/hostd/docroot/sdk

/usr/lib/vmware/hostd/docroot/sdk # ls

vim.wsdl vimService.wsdl

/usr/lib/vmware/hostd/docroot/sdk # more vimService.wsdl

-ash: more: not found

/usr/lib/vmware/hostd/docroot/sdk # less vimService.wsdl

<?xml version="1.0" encoding="UTF-8" ?>

<!--

-->

<definitions targetNamespace="urn:vim25Service"

xmlns="http://schemas.xmlsoap.org/wsdl/"

xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"

xmlns:interface="urn:vim25"

<soap:address location="https://localhost/sdk/vimService" />

</port>

</service>

</definitions>

vimService.wsdl (END)

-

now We determined our root cause of the problem can be that we don't have access to vmware-cmd

lastly, Does your script only copy from one Datastore to Another?? so One ESXi Server to Another, or can it be setup to snapshot, and copy from ESXi to Windows 2003 Server??

please Guide us!!!

Thank You

All

Free ESXi Backup Solution for Windows