Free ESXi Backup Solution for Windows

[ChrisInTexas]

I have spent the last few days trying to find a free backup solution to the newly free ESXi for windows only enviroments (in particular Windows XP). The solution for me was the following:

1. Installing Windows Services for UNIX (WSFU)

2. Copying the ESXi Server password and group files to Windows

3. Configuring WSFU for accepting ESX Server connections

4. Sharing the Windows folder for NFS compatibility

5. Configuring the ESXi Server to mount the Window NFS Share as Datastore.

6. Setup Backup Script

Attached is the complete steps.

I take NO credit for any of this. This is just a complation of others work formated to suit my needs and felt others could benift from it as I have.

by Jason Mattox from Vizioncore (direct copy of his work, I just added more information to make it work in Windows XP)

(NFS Server port information)

by robink (The backup script)

(ssh on ESXi)

[kpc]

I've attached my backup script - it's written using Shell and is pretty simple. I use it on a SUSE 10 Linux server that has the RCLI installed on it, but you can use any variety or even the appliance that VMware provides you with.

It's fast, doesn't use vifs, uses vmkfstool to do an export locally to a temp directory inside the VM's folder, then SCP's (PULLS) the export to a remote NFS share on another server, supports compression, backups VM's multiple datastores, gives you the option of doing weekly backups and will email you a report when the backup completes.

You need to setup automatic ssh key, however it's all documented in the comments section at the top of the script.

It will backup a 14GB VM that has used 4.7GB in about 13 minutes (3 minutes for the export and then 10 minutes to copy that export to the NFS share roughy 15MB/s.) If I could get the full write speed I could write the export directly to the NFS share it would cut the backup time by another 5 minutes, but haven't figured out why the slow speed yet.

If your new to Linux and have some free time give it a whirl. It won't work on Windows!

[s1xth]

KPC-

Nice looking script, I am new to using the Vmware RCLI and I downloaded the vmware provided appliance and I am running it now in vmware player. When I try to run the script I get 'command not found'. I wouldnt call my self an entire linux noob but I havent been using Linux non-stop so I forgot some of the commands. What needs to be typed to run this script in the appliance? I just uploaded it to the appliance via winscp and put it in the root directory (may not be best practice) but I would like to try this out.

Thanks!

[kpc]

Hi s1xth

You can run it from root no problem, just type the following:

./esxi_backup_script.sh

That's a full-stop or period in front Let me know if you get stuck at any other point. You can also run the script without having to setup the SSH keys, just enter them manually when prompted (I think)

[kchawk]

make sure to chmod with execute perms.

[s1xth]

KPC-

Thanks! I will try this out tonight!

[solutiond]

Hi kpc,

Have a question regarding your great script. I want scp to pull the export to a second ESXi server. Can I still use the nfs strategy (nfs share on second ESXi server) or must I use something else?

Cheers

[solutiond]

Hey kpc,

In your script you say for restore, copy files back to esxi. How do you achieve this?

[kpc]

Hi Solutiond

To copy files back to esxi there are a number of ways, however I do all my moving of files using SCP. To move the export directly to another ESXi server all you have to do is either do a SCP (push) from the first esxi server, something like:

scp -r /vmfs/volumes/datastore1/myvm

or a SCP (pull) from the second (command is slightly different)

Cheers

[s1xth]

I do the same, I use SCP to copy vm's to another esxi server. It can be alittle slow but it works. Hopefullly FastSCP has their ESXi program out soon to help with this.

[Johny009]

Hello again.

Sorry but it is very important for me.

I use the script with my ESXi 3.5 Server and it works very good, but when i want to use it on my ESX 3.0.1 Server, than it doesn't work.

I get everytime this error message:

Get https://192.168.0.1/folder/WINXP?dsName=datastore1 unsuccessful : 404 Not Found

What can i do now? I want to use this script for both esx server.

And what is the quickest restoring methode?

Thanks!

[peedy]

KPC,

Thank you for the wonderful script. I have not actually tried it but I think its gonna be exactly what I am looking for. I am a total noob when it comes to programming so I was wondering how I could, or you could, add support to export the ESXi config as someone mentioned there is a command provided that does just that.

thanks,

[kpc]

Hi Peedy. I may look at exporting the ESXi config, however before I do that I need to rewrite part of the export process, I've found out why I get such poor write speeds to my NFS server.

Cheers

[peedy]

Fantastic KPC, I look forward to seeing the new developments. Your script only works on linux with CLI or the RCLI VM right? I'd love to see someone port it to windows since the clients Im setting things up for are mostly a Windows shop.

[peedy]

Also, I'd like to get people's opinion on how I should backup some specific servers.

Client just bought a new Proliant DL320 G5p, ESXi doesn't support the raid so I have two SATA drives. I also have the main file server running windows server 2003. I was thinking about backing up the VM's to the 2nd disc in the new server then copy to via nfs or iscsi to the main file server, essentially having two backups. Is there more performance using iSCSI (gigabit ethernet) over local sata storage or NFS?

Sorry if any questions are RTFM, I have played with VMware a lot but this is my first attempt at trying to do a production box with it.

[kpc]

Hi Peedy

You say ESXi doesn't support the Proliant RAID card? I personally would think twice before using this for production, there would be no resiliance in your VM's, however sometimes you have to do what the client asks! Generally local is always going to be faster than NFS or iSCSI if you're not using an ultrafast network and something like a RAID 10 on your iscsi array.

From what you're saying you'll have two independent disks in your ESXi server. ESXi will probably use both to store VM's on, creating two datastores. Instead of wasting a datastore with backups I would create a windows NFS share and export your backups directly to there.

Hope this helps a little

[solutiond]

It all depends on the size of the vmdk's, your max transfer speed, what you want to protect yourself against and how long the restore process will be that management except.

Lots of options.

As for local backups I would be creating a second datastore on the second disk and doing an export to this location. you would have to modify the script so as instead of deleting the export once the offsite copy completes, it stays and the following day it would increment the folder name for the next backup. keep this going to you get however many version your happy with. You'd expect 3 versions would be plenty. On the fourth day the script would have to then trigger a remove command of the oldest backup so it can be replaced.

As for the second backup, I'd definatly go either SCP to a iSCSI LUN or WINSCP to your windows file server.

You definately need a standby copy that you can fire up in minutes in case the first disk dies (export to second disk). Then you need the copy on iscsi in case you lose the whole server.

Next question is how long is the client down if they lose the whole server? major part issue or server needs to be replaced. Thats why it's best to somehow come up with a plan of using 2 esxi servers. then the copy/restore process is incredibly easy.

I do this at most places: whatever reason one new box to acted as a esxi host is being purchased, put in your plan to virtualise an existing server onto the new esxi server, then leverage this free'd up hardware to act a a esxi dr server. then you export directly to the dr esxi server. in the case of an emergency you can have the site back up in 10 minutes.

sd

sd

[s1xth]

Solutiond-

You mention at the end of your last post that you can use/or have a second ESXi server as a DR server, housing all of your VM's that were copied from the live server. How do you achieve this? This is something that I want to be able to do in the future, utilize another ESXi server as a DR/failover server and have last nights backup ready to fire up.

Thanks!

[solutiond]

Scripting credit goes to kpc. All I do is twist it for my needs. kpc's script is available several posts back.

I haven't proved this 100% but will be doing the final stages tomorrow. So far things look good. I'll let you know my final tests.

Limitations: the SCP process from esxi01 to esxi02 for me on gigabit infrasrtucutre give me an average transfer speed of about 8mb a second. This means you might be lucky and be able to transfer around 300GB an hour. if you have a large amount of VM's or very large vmdk's then, this might not be the solution for you.

once you have 2 esxi servers, setup authorisation keys for scp. next we don't want to apply the 2gbsparse switch as we are just moving to another esxi server. As a result remove "-d 2gb sparse" from the clone disk command. ssh $HOSTNAME "vmkfstools -i $VM_DIR/$VM_NAME.vmdk $VM_DIR/backup/$VM_NAME.vmdk"

as for the weekly backup, comment it out for now. next you need to change the "BK_DAILY=" to the new destination value of your scp command. remember we are copying it directly to the datastore of the second esxi server

Keep it a little simplier and try all that without authorisation keys for now. I'll post my script changes after testings complete tomorrow.

Let me know if that all made sense and I explained it properly.

sd

sd

[s1xth]

Solutiond-

I am going to try this out today, what you say makes some sense but I am not 100% on the exact way you wrote the script. In KPC's script it asks for the location of backups/log files, what did you use? Something like this? root@esxiservername pathtodatastore' same as if you were scp files to that other esxi server?

Edit- I just re-read your post and noticed you said you would be posting the script once you are done with it, I will take a look at it once you post it and I am sure it will answer my question. I think copying to another ESXi server is exactly what I want to do, so I have a copy on another ESXi server and I could fire that right up.

Thanks

[Cellolein]

Hi kpc,

I know i already asked this a few weeks ago. But did you in the meantime find a way out, how to backup VM's including white spaces in it's names?

This would be great.

Thanks in advance,

Cello