Free ESXi Backup Solution for Windows

[ChrisInTexas]

I have spent the last few days trying to find a free backup solution to the newly free ESXi for windows only enviroments (in particular Windows XP). The solution for me was the following:

1. Installing Windows Services for UNIX (WSFU)

2. Copying the ESXi Server password and group files to Windows

3. Configuring WSFU for accepting ESX Server connections

4. Sharing the Windows folder for NFS compatibility

5. Configuring the ESXi Server to mount the Window NFS Share as Datastore.

6. Setup Backup Script

Attached is the complete steps.

I take NO credit for any of this. This is just a complation of others work formated to suit my needs and felt others could benift from it as I have.

by Jason Mattox from Vizioncore (direct copy of his work, I just added more information to make it work in Windows XP)

(NFS Server port information)

by robink (The backup script)

(ssh on ESXi)

[kchawk]

the script does copy to another shared datastore. NFS, Fiber, or iSCSI. I use nfs off the same linux server that runs the script.

I am afraid I am a little confused, sorry. If you are using my script it doesn't require that you enter the vimService parm.

vmware-cmd is ran from the host the script is on(rcli) not the esxi host to get the list of vms and for snapshots.

Then vmkfstools is ran on the esxi host to copy the vmdks to the shared datastore using auth keys so no password is required for ssh.

Do you have the auth keys setup?

[oalkatib]

humm... Okay lets start from scratch, and review the process I'm following (I probably should of explained this in my first post, really sorry about that)

here is what I have setup

• An HP Proliant Server with ESXi installed on it, and all the data stores are local storage (i.e. internal drives, not external mounts what so ever)

• I have a windows 2003 server with 1.5 TB of disk space, i want to use that server as a backup location for all my ESXi Server, just incase one of the ESXi server physically crashes, I can push a backed up VM to another ESXi server to lower down time.

So I searched, and Came across your script, and a few others, but yours seemed to be the most appealing, as it takes a snap shot, and copies, and also many have had success with it.

So this is what I have done,

• I have enabled SSH on the HP ESXi Server

• And kicked off the script from my workstation to see if it works, and all I keep getting is the error I mentioned in my previous post, which is:

-

---

C:\Program Files\VMware\VMware VI Remote CLI\bin>vmware-cmd.pl -H 10.170.250.**2 -U root -P ****** -l

Error connecting to server at 'https://10.170.250.**2:443/sdk':

500 Connect failed: connect: Unknown error; Unknown error

-

---

• Now I have only installed Remote CLI on my workstation and that is it,

-I have only enabled SSH on my ESXi server, and that’s about it.

• I didn't upload any scripts or any additional files nor did I do any modifications to my esxi server, I have only and only enabled SSH, everything else is at its default setting.

Now that you know what my settings are, and what I'm trying to accomplish, please guide me through, as I'm struggling to get this fixed, and working

Thank You Kchawk, you have been very responsive

[kchawk]

Great detail, the bad news is that my script will only export the vmdk files to another datastore and in your case, since they are local only, this doesn't seem like a good solution.

If you loose that host you loose the backups. Other than that I will do some testing tonight on the vmware-cmd. I don't know if there are any diffs between the rcli on windows and linux, but I would expect you should

get the list of vms with vmware-cmd.pl -H 10.170.250.**2 -U root -P ****** -l

Maybe someone else will answer, if not I will test a few things tonight and let you know what I find.

Hopefully my script has been working as I will have to restore my windows vm

[kchawk]

Ok, didn't have a lot of time last night, forgot about family game night. First, where are you running the script from? It is a bash script so it won't run in windows unless you

are using gnuwin or something. I was able to run the vmware-cmd.pl from my xp vm to pull the list of vms so the cmd functions the same as in linux.

If you are running the script on the esxi host it won't work. I think there is a script here that may run completly from the esx host but not sure.

[jwall04]

sanbe, if I have more than one vm running on my server does the following code not specify to backup just the listed vm?

my @VMNames;

$VMNames[0] = "Test2"; #Uncomment the next lines if you like to backup more vm's

#$VMNames[1] = "";

When I run the script it starts backing up the flat.vdmk from a server I haven't listed.

UPDATE

I'll be honest, I haven't read through every post in this thread, but after skimming through several pages I see there are a few solutions here, and maybe sanbe's solution isn't the best for me. I apologize if the answer to my question is in this thread, if that's the case please direct me.

What I'm looking for is a script, which I can run from a Windows box, to backup running vms on an ESXi server, and copy them to the Windows box I'm running the script from. Actually, the vms don't necessarily have to be copied to the Windows box, but I do want them off the ESXi Host. If I need to setup a file share or turn SSH one please tell me where to find the info.

And once I have this working how would I restore the vms if, for example the ESXi Host, died?

And can someone please explain what actually happens (with the .vmx file etc) when I take a snapshot and what a .vmx, .vmdk and flat .vmdk files are?

Thanks so much

[oalkatib]

Hi kchawk, sorry I took forever to respond back, well we are trying to push

the script from a windows, and its a shell script. So we tried some of the Linux

emulators, and still we didn’t have much luck.

Is there a way to run this script purely from windows. Because like I

mentioned before, we want to kick off the script to run daily from a windows

server that has almost 2TB of space, to do what it does, take a snap shot, and

transfer over, and delete old snap shot. Then backup that windows server via net

backup onto tape.

So if any ESXi server fails, I simply run the converter on my windows

server, or transfer the files to a data store, and turn on the VM, thus

reducing down time....

HELP!! AHH!

I'll be looking forward to your reply!

[kchawk]

To use my script you need a local ds and remote ds, iscsi, nfs, or fiber, so it probably isn't the solution.

Have you tried the ESXi-Backup.pl script? It uses the vifs.pl cmd to export. It is slow but does the job.

[lamw]

Have you taken a look at plink, it's by the makers of putty. It's a small exe that performs like an SSH client. Also if you're looking for more storage replication option, not just local datastore you can take a look at ghettoVCB.sh. It is more Linux/UNIX oriented but there isn't too much to configure to get it working. I would caution you since you're used to Windows, to ensure that you don't have any Window ^M characters within the script or the input file.

Either way, if you need to automate an SSH connection and you're using SSH keys, plink can help.

=========================================================================

--William

VMware ESX/ESXi scripts and resources at:

[kchawk]

Just for clarification, for me and others, ghettoVCB, esxibak.sh, others?, backup the vms only to another datastore. I think the issue is he is only running on MS box so unless he installs

unix services or iscsi target, he doesn't have another ds to connect to. The scripts that use vifs should work and I think there are some posted in this thread by Christian Gruetzner called ESXi-Backup.pl.

Please correct me if I am mistaken.

[lamw]

I can't speak for the other scripts, but ghettoVCB.sh supports all storage medium (LOCAL, NFS and iSCSI/FC SAN) backups. I believe you're correct, that he's trying to execute this backup process on a Microsoft Windows server, but, the script itself is not being executed on the Windows host. It's executed on either ESX or ESXi itself. If you upload the script to the host, you can remotely kickoff the process so long as the configuration on the script is set properly and you have a list of valid VM(s) to backup. The backup can take place locally from local->local datastore, there are no restrictions on what type of datastores the backup process requires, nor should it.

=========================================================================

--William

VMware ESX/ESXi scripts and resources at:

[kchawk]

Ok, I understand the option of running the script on the host, didn't catch that one. I think I will take a look at using it. The only thing I am trying to make sure there is no confusion on (again for me and others) is the comment about "all storage medium". Even if the script is ran from the host he isn't going to be able to backup directly to his MS box because NFS, iSCSI/FC SAN are not going to be exported from the MS box without further install/config.

I believe the only options to go directly to a MS box is installing unix services for NFS, iSCSI target like StarWind, WinTarget, MS Storage Edition or use a script that uses vifs.

Maybe I am missing something...

[kchawk]

oalkatib, another suggestion would be to use openFiler to turn your MS box into a SAN to export NFS or iSCSI LUNS. You could then easily use ghettoVCB.sh to backup to your lun then to tape.

[lamw]

Nope you're totally correct, his "ms box* box unfortunately is not a valid datastore or at least at it's current present state. As you've already suggested, he can install UNIX services and export that out as NFS or create an iSCSI target. There are many options, even using the RCLI but if you're going to download/upload using vifs, that could be kind of slow.

=========================================================================

--William

VMware ESX/ESXi scripts and resources at:

[kchawk]

Has there been any info on the slow perf of vifs? Did a quick search but didn't see anything concrete that smacked me in the face.

[lamw]

I haven't checked the forums, but I had tried to upload few files and it took longer than it should but then again I haven't really dived into this type of backup solution so I can't say 100% sure.

=========================================================================

--William

VMware ESX/ESXi scripts and resources at:

[willmarmn]

I tried the command and everything works, except for the log output when using nohup. Without nohup, it works fine, but when I run it as stated, the file get created but is 0 bytes. When I run the command directly on the ESXi host, it seems that is continues to want to output the log to the nohup.out file. Anyone experience this issue?

[SeanT77]

I'm not sure if this has been offered yet, too many posts to read them all. I have used software raid1 to script the following: mirror the vmdk1, break the mirror, disconnect the vmdk2, copy off the vmdk2 to a backup location, reconnect the vmdk2 and remirror the vmdk1/vmdk2. You can even mirror the vmdk2 on a network store to be backed up.

[shechtl]

Hello lamw,

in your ghettoVCB-enhance I get sometimes this error message: 79% done.Failed to clone disk : Input/output error (327689)

I get this error with different VMs, sometimes, not always. I use the feature to shutdown the VM first.

I have this problem on two ESXi Server. The storage is a open-e DSS mounted per NFS. The backup storage is also DSS with NFS.

The VMs are XP and 2003 Server. HDDs are between 15 and 120GB.

Do you know what the problem could be ?

[lamw]

Nope, not sure. I would take a look at your storage since it's showing I/O error. Try to manually vmkfstools your disks a couple of times manually and see if you get the error, if you do, then it's your storage. This most likely is something within your environment than the script.

=========================================================================

--William

VMware ESX/ESXi scripts and resources at:

[ipadl]

Has anybody spent any amount of time working on the restore functionality of each of these scripts, especially in a disaster recovery type environment?

I am currently doing a two-stage backup using the following:

1. ESXi-Backup_mod_by_khaliss.pl to snapshot and copy my server to an openfiler NAS attached to the ESXi server as an iSCSI target

2. A third party beta software designed to transfer files from the NAS to a USB local to a windows XP machine that allows me to walk away from the office with my server on a drive.

Network Information:

1x HP Proliant DL360G5 Production Server with ESXi 3.5.0 w/ Intel + Broadcom Gigabit NICs

1x HP Proliant DL360G5 Test Server with ESXi 3.5.0 w/ Broadcom Gigabit NIC

Gigabit network

Guest: Windows SBS Server 2003 ~600GB in size

Openfiler NAS w/ Gigabit NIC and 1x 1TB Sata HD, no raid currently

Current Backup procedures:

1. Nightly ESXi copy from ESXi to Openfiler NAS

2. Nightly file-level recovery, exchange, and system state backups using ntbackup

3. Weekly copy of Openfiler NAS to walk away USB drive

So here's the scenario that I am currently running in my mind. The Production Server hardware goes kaput for some unknown reason in the middle of the day. We fire up the Test Server, attach the NAS as an iSCSI target, and create a new VM pointing at the vm files on the NAS. I have tested and this works perfectly. However, a second folder is created called Servername_1 on either the NAS datastore or in the local datastore, depending on where you choose when creating the new vm using the the "New Virtual Machine" wizard in ESXi. I would assume that we will see a bit of a performance drop using the NAS as our datastore than if the data were local to the ESXi machine, so we will probably want to transfer the data from the NAS datastore to the test server datastore (or back to the Production server if it is fixed at this time) during the weekend. How is this effectively accomplished?

Do we need to pull all of the files from the NAS datastore Servername folder(servername.vmx, servername.nvram, servername.vmdk, vmware-core.gz, servername.vmsd, servername.vmxf) AND the files from the new Servername_1 folder (Servername_1.vmx, vmware.log, Servername_1.nvram, Servername_1.vmxf, Servername_1.vmsd)? I am assuming that the files in the new folder (Servername_1) are simply pointing to the files in the old folder on the NAS (Servername) and any changes are made on the files in the old folder on the NAS, so do we just create a new local vm with the right attributes and replace the data files in the datastore folder with the files from the NAS?

If that's confusing, let me know and I'll try to rephrase it properly. Thanks for your help!

Seth