VMware Communities: Message List - Free ESXi Backup Solution for Windows

Re: Free ESXi Backup Solution for Windows

cincinnerdi — Wed, 30 Sep 2009 15:55:27 GMT

This is an automated response to email sent to scott.ledyard@nerdsonsite.com
I am out of town presently, but will be checking messages once each day.
In the event of an emergency, please contact my backup eNerd, Jeremy Laughlin at 859-380-8983 or jeremy.laughlin@nerdsonsite.com.
Or contact our dispatch (please mention your a client, if you are) at 1-877-696-3737.

Scott Ledyard
513-708-7682

Re: Free ESXi Backup Solution for Windows

Cellolein — Wed, 30 Sep 2009 15:45:26 GMT

Ich bin bis einschließlich 02.10.09 nicht im Hause. E-Mails werden nicht bearbeitet oder weitergeleitet. In dringenden Angelegenheiten wenden Sie sich bitte an meinen Kollegen Herrn Venker. Email: venker@schmitz-werke.com Telefon: +49 2572 927 0. Über unsere Telefonzentrale werden Sie an Herrn Venker weitergeleitet.

Mit freundlichen Grüßen,

Marcel Ruholl
Schmitz-Werke GmbH + Co. KG
Abteilung EDV
Hansestrasse 87
48282 Emsdetten - Germany
fon: +49 (0) 2572 927-198
fax: +49 (0) 2572 927-9198
mailto:MRuholl@Schmitz-Werke.com
http://www.schmitz-werke.com
KG, Sitz Emsdetten, Amtsgericht Steinfurt HRA 3133
Komplementärin: C.H. Schmitz Beteiligungsges. mbH
Sitz Emsdetten, Amtsgericht Steinfurt HRB 3612
Geschäftsführer: Justus Schmitz

Diese E-Mail ist revisionssicher archiviert.
This e-mail is legal audit proof archived.

Re: Free ESXi Backup Solution for Windows

cincinnerdi — Wed, 30 Sep 2009 15:45:25 GMT

Re: Free ESXi Backup Solution for Windows

banthorpe — Wed, 30 Sep 2009 15:41:14 GMT

OK got it all working now

However, I am trying to backup a VM where the disks are thinly provisioned. So when it takes the snap and then does the backup it has to expand the file to it's full size and I end up with 3 VMDK files:

test.vmdk, test-flat.vmdk and test-000001.vmdk

if I blow away the VM and then copy everything back from the backup location it tells me that the test.vmdk is an invalid file. any ideas what is going on here?

Re: Free ESXi Backup Solution for Windows

banthorpe — Tue, 15 Sep 2009 15:36:42 GMT

ok - so I have recently downloaded your script to get it to work. However, I have only uncommented 1 VM name in the array (see below), but the script still copies the snaps of all running VMs as opposed to just those specified in the array?

my $DSPath = "ESXIDMZ01_1"; #Datastore name on ESX Host, example datastore
my @VMNames;
$VMNames[0] = "C2C"; #Uncomment the next lines if you like to backup more vm's
#$VMNames[1] = "";
#$VMNames[2] = "";
#$VMNames[3] = "";
#$VMNames[4] = "";
#$VMNames[5] = "";
#$VMNames[6] = "";
#$VMNames[7] = "";
#$VMNames[8] = "";
#$VMNames[9] = "";
my $RCLIPath = "C:/PROGRA~1/VMware/VMWARE~2"; #VI Remote CLI Path (Windows: Use ONLY Short Folder Names!!!!)
my $DestPath = "f:/esxidmz01/";

Re: Free ESXi Backup Solution for Windows

mennouth — Wed, 26 Aug 2009 22:01:53 GMT

Ok i finaly got the script to run using VMwares new version of the RCLI,only to get prompted on the creation of the snapshots that the version is restricted

in the vi client it says that i m running version 3.5.0 build 158874 I am presuming this is the version after they locked the RCLI down to read only. so i did a little digging and it seams that there is anouther way to create the snapshots using vm vix 1.6 so i could schedual a few diffrent jobs one to go in and make the snapshot using the VIX api and the other to go in and copy the files using the RCLI then execute anouther script to open up the VIX api and remove the snapshot on the server in question.

I use windows NT backup for all but one of the servers on the that way i can have incrimental backups, and i would use it on the exchange server, however there is a major issue with using the enhanced backup built in with exchange on a VMware platform. which leaves me hunting for a script.

the question i have is there anyone willing to help woddle my way through the code to set it up.

Re: Free ESXi Backup Solution for Windows

tranceking — Fri, 14 Aug 2009 11:58:23 GMT

Is there a modification where we can backup other vmdk in a vm and skipping only rdm disk instead of skipping the whole vm.

This will make the script very usefull

Re: Free ESXi Backup Solution for Windows

jllopezg — Tue, 14 Jul 2009 10:37:21 GMT

Hello everybody,
I am trying to do the backup of a esxi with this script, it is working perfecty, but I need to backup the vm´s with rdp too.
I have tried the ghettoVCBg2.pl without success.
Anybody has the code to add to the script to backup the vm´s ignoring the rdp?

Thanks a lot for yor help!
Regards.
Jose

Re: Free ESXi Backup Solution for Windows

AlbertWT — Wed, 17 Jun 2009 23:20:02 GMT

Thanks for the reply argyle. we don't have the license to implement Win2008 R2 yet though. But it is good to know that such feature does exist.

Kind Regards,
AWT

Re: Free ESXi Backup Solution for Windows

bones*79 — Wed, 17 Jun 2009 02:23:51 GMT

Sehr geehrte Damen und Herren,

Ich befinde mich in der Zeit vom 11. Juni bis zum 21. Juni nicht im Hause.

Ich werde Ihre Mail nach meiner Rückkehr beantworten.

Bei dringenden Anliegen wenden Sie sich bitte an Herrn Eisenberg.

Mail:
eisenberg@hospital-fritzlar.de

Telefon:
05622 997-7729

Mit freundlichen Gruessen

Martin Bolz

Leiter EDV-Abteilung
Fon: (05622) 997-650
Fax: (05622) 997-654
Mail: bolz@hospital-fritzlar.de

Hospital zum Heiligen Geist
Am Hospital 6
34560 Fritzlar
www.hospital-fritzlar.de

Re: Free ESXi Backup Solution for Windows

Argyle — Wed, 17 Jun 2009 02:18:24 GMT

When it comes to NFS on Windows take a look at the Release Candidate for Windows 2008 R2. It can be downloaded for free. There is a new NFS option called "Unmapped UNIX User Access".

"An Unmapped UNIX User option is now available for NFS shared folders. Windows servers can be used for storing NFS data without creating UNIX-to-Windows account mapping. Mapped user accounts use standard Windows security identifiers (SIDs) and unmapped users use custom NFS SIDs."

More info:

What's New in Services for NFS in Windows Server 2008 R2
http://technet.microsoft.com/en-us/library/dd540762(WS.10).aspx

Services for NFS Step-by-Step Guide for Windows Server 2008 R2
http://technet.microsoft.com/en-us/library/dd758767(WS.10).aspx

Re: Free ESXi Backup Solution for Windows

sanbe — Mon, 08 Jun 2009 12:14:41 GMT

I´m not in the office until 17.06.2009. Your E-Mail won´t be forwarded.
In urgent cases contact "PTD DataCenter" oi-sd-datacenter-mailbox.ptd@siemens.com

With best regards,

Bernd Sandner
Siemens AG GIO IT SHS 94
Freyeslebenstr. 1
91058 Erlangen, Germany
Tel.: +49 (9131) 7-35210
Fax: +49 (9131) 7-31537
mailto:sandner.bernd@siemens.com

Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Gerhard Cromme; Managing Board: Peter Loescher, Chairman, President and Chief Executive Officer; Wolfgang Dehen, Heinrich Hiesinger, Joe Kaeser, Barbara Kux, Hermann Requardt, Siegfried Russwurm, Peter Y. Solmssen; Registered offices: Berlin and Munich, Germany; Commercial registries: Berlin Charlottenburg, HRB 12300, Munich, HRB 6684; WEEE-Reg.-No. DE 23691322

Important notice: This e-mail and any attachment thereof contain corporate proprietary information. If you have received it by mistake, please notify us immediately by reply e-mail and delete this e-mail and its attachments from your system. Thank you.

Re: Free ESXi Backup Solution for Windows

Cellolein — Mon, 08 Jun 2009 12:14:40 GMT

Ich bin bis einschließlich 17.06.09 nicht im Hause. E-Mails werden nicht bearbeitet oder weitergeleitet. In dringenden Angelegenheiten wenden Sie sich bitte an meinen Kollegen Herrn Venker. Email: venker@schmitz-werke.com Telefon: +49 2572 927 0. Über unsere Telefonzentrale werden Sie an Herrn Venker weitergeleitet.

Re: Free ESXi Backup Solution for Windows

spex — Mon, 08 Jun 2009 12:14:40 GMT

Guten Tag,

ich bin am 15.06.2009 wieder im Haus.

Bei wichtigen Problemen wenden Sie sich bitte an Herrn Herbert Bachmann.

Ihre E-Mail wurde nicht weitergeleitet

Danke
Stefan Holzwarth

Re: Free ESXi Backup Solution for Windows

AlbertWT — Mon, 08 Jun 2009 12:04:19 GMT

Hi All,

Does this Method still working fine in ESXi 4 ?
Because at the moment I'm looking for a fast and free way to backup my 2x ESXi 4 servers into he SAN.

Kind Regards,
AWT

Re: Free ESXi Backup Solution for Windows

krrunal — Sun, 03 May 2009 05:48:27 GMT

i found the issue as the it was the short windows path name in declaring the variable. It took me 2 day to just find that. It should suppose to "C:\prog~\VMWare~\VMWare~4" instead of 1 at the end..Thanks to my manager Dennis Williams for finding that out for me.

Re: Free ESXi Backup Solution for Windows

KBuchanan — Sat, 02 May 2009 20:27:09 GMT

Gothca. If you're running the VIMs locally, then, I suggest to load ESXi on 1 partition, so that you can separate the Host and guest o different partitions. If you have to reload ESXi, then it will not mess with the VMs.

Also, get a written documentation of the network config.

Good luck!!

Kevin Buchanan
Chief Information Officer
Lexington Memorial Hospital
336-238-4286
kbuchanan@lexmem.org

Re: Free ESXi Backup Solution for Windows

N20Steve — Sat, 02 May 2009 18:38:09 GMT

Hi,

That's exactly why I'm undertaking this exercise. It's for a new office installation - Brand new Proliant server, RAID5, with a 'hot spare' drive. I want to implement a system that will give the fastest possible recovery time in the event of a problem. The server will be virtualised purely to ease the backup and recovery burden.

At the moment I'm experimenting with options as I want it to be as bomb-proof as possible. If there's a neat and safe way to take a regular backup of the running server then that would be great. The data will (regardless of this issue at hand) be backed up daily to two remote (another building, but on the same LAN) NAS devices, and weekly to tape. I just want to have a 'belt and braces' arrangement with a complete backup of the server that could be restored in a (virtual!) blink of an eye.

The NAS boxes already exist, and there isn't budget/scope for multiple replicated servers, so this would be a reasonable halfway house.

Cheers.

Steve.

Re: Free ESXi Backup Solution for Windows

KBuchanan — Sat, 02 May 2009 18:07:09 GMT

Good advice. Test backup and restore NOW. And have a written recovery plan. You don't want to have to fly by the seat of your pants if you have an unplanned downtime. I've scripted all of our backup and recoveries.

Kevin Buchanan
Chief Information Officer
Lexington Memorial Hospital
336-238-4286
kbuchanan@lexmem.org

Re: Free ESXi Backup Solution for Windows

N20Steve — Sat, 02 May 2009 17:20:23 GMT

Many thanks for such a fast reply. Much appreciated.

And for my next question...

What is the best method to backup a server without taking it down? The solution doesn't need to be free!

Thanks again.

Regards.

Steve.

Re: Free ESXi Backup Solution for Windows

KBuchanan — Sat, 02 May 2009 17:17:08 GMT

Yes...but if you restore, it will be as though the server was "unplugged"...ie, a hard shutdown.

That may not be good for some databases or other open files. There isn't a quiesce for the data.

Kevin Buchanan
Chief Information Officer
Lexington Memorial Hospital
336-238-4286
kbuchanan@lexmem.org

Re: Free ESXi Backup Solution for Windows

N20Steve — Sat, 02 May 2009 17:04:53 GMT

Hello,

New to VMWare, and I'd like to say thanks to everybody who has contributed to this thread - Incredibly useful.

I've got the script running, and it seems to be exactly what I need. However, I have a question that will confirm my 'newbie' status with virtualisation:

Is it the case that this script can be used to back up a running server, with no need for any server downtime?

Thanks a lot for reading.

Steve.

Re: Free ESXi Backup Solution for Windows

KBuchanan — Thu, 30 Apr 2009 06:27:10 GMT

Did you install the RCLI software?

Kevin

Re: Free ESXi Backup Solution for Windows

krrunal — Thu, 30 Apr 2009 03:59:32 GMT

Gooday Sanbe,

I just came around thisuseful thread of backingup VMs using VMware Remote CLI used along with the perl. The start of the question is i install the Remote CLI from vmware and when i run the attached.pl script in windows from cmd prompt it brings the error saying that

"Can't open perl script "C:/Progra~1/VMware/VMware~1/Perl/apps/vm/snapshotmanager.pl: No Such file or directory. Do i have to change anything in variable path my $RCLIPath = "C:/Progra~1/VMware/VMware~1". I tried to change the path to but then it doesnt recognise at all. Help please as i am not with scrpting at all.

Re: Free ESXi Backup Solution for Windows

DSTAVERT — Fri, 24 Apr 2009 02:52:13 GMT

Spelling in backup file or backup destination in script.

Re: Free ESXi Backup Solution for Windows

lamw — Fri, 24 Apr 2009 02:30:34 GMT

Yea it looks like the system is having issue with it makes a call internally and it's failing. Have you made any changes to the system lately or do anything on the unsupported console. I would recommend rebooting the host if you can to see if the issue goes away.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
http://twitter.com/lamw

If you find this information useful, please award points for "correct" or "helpful".

Re: Free ESXi Backup Solution for Windows

lamw — Fri, 24 Apr 2009 02:28:53 GMT

Thanks for the confirmation.

Re: Free ESXi Backup Solution for Windows

holgate — Fri, 24 Apr 2009 02:22:44 GMT

Hm. Looks as if the problem isn't with the script, but with some element of the ESXi internals: The logs are showing this:

[2009-04-24 02:10:35.736 'ha-eventmgr' 81926 info] Event 20879 : Failed to login user root@127.0.0.1: No permission
[2009-04-24 02:10:35.736 'Vmomi' 81926 info] Activation [N5Vmomi10ActivationE:0xac72610] : Invoke done [login] on [vim.SessionManager:ha-sessionmgr]
[2009-04-24 02:10:35.736 'Vmomi' 81926 info] Throw vim.fault.NoPermission
[2009-04-24 02:10:35.737 'Vmomi' 81926 info] Result:
(vim.fault.NoPermission) {
   dynamicType = <unset>, 
   object = 'vim.Folder:ha-folder-root', 
   privilegeId = "System.View", 
   msg = ""
}

But the only reference to that particular error is in relation to VMWare Server 2. I'll do some more digging.

Re: Free ESXi Backup Solution for Windows

DSTAVERT — Fri, 24 Apr 2009 01:57:11 GMT

lamw

I have had it running on U4, I also have no problem with the command line either.

Re: Free ESXi Backup Solution for Windows

lamw — Fri, 24 Apr 2009 01:10:48 GMT

hmmm, if you're logged in as root, you should have full permissions. I'm wondering if the fix also somehow disabled vimsh wrapper from executing any local commands? It's odd because you're already logged in as root. Without a system of my own, I can't really do further debugs, I was not aware of any changes to the unsupported console that would prevent you from executing the vim-cmd as you would on classic ESX Service Console with vmware-vim-cmd

I'll need to ask around to see if theres something more to this.

Re: Free ESXi Backup Solution for Windows

holgate — Fri, 24 Apr 2009 01:08:14 GMT

Thanks for such a quick response:

*vim-cmd vmsvc/getallvms*

from the command line produces the same error: "Failed to login: vim.fault.NoPermission."

Re: Free ESXi Backup Solution for Windows

lamw — Fri, 24 Apr 2009 01:00:57 GMT

Interesting ...

Can you run the following manually without any errors:

vim-cmd vmsvc/getallvms

I don't have an ESXi 3.5u4 system to test with, so I'll have to rely on some steps to see what might be the issue. I'm hoping this has nothing to do with the ESXi 3.5u4 lockdown, it should only affect remote API calls to the host.

Re: Free ESXi Backup Solution for Windows

holgate — Fri, 24 Apr 2009 00:36:32 GMT

Hi lawm: thanks for the script(s)! A quick question: I'm getting the following error on ESXi 3.5U4 (logged in as root):

./ghettoVCB.sh vmbackups
Failed to login: vim.fault.NoPermission
Error: failed to extract VM_ID for datastore1/foobar

The problem is apparently with the initial "vim-cmd /vmsvc/getallvms", which produces the same error when run separately -- any suggestions? (I'd reply on the dedicated thread, but that doesn't appear to be working now.)

Re: Free ESXi Backup Solution for Windows

IT_Architect — Fri, 17 Apr 2009 15:16:08 GMT

False

Good to hear. I will be using a Windows 2008 Web Edition server to schedule the backups using your script as per the instructions. With no compelling reason to change, I'll be sticking with U3 for awhile, not only for feature reasons, but also because it has a good record built up for performance and reliability.

Thanks!

Re: Free ESXi Backup Solution for Windows

lamw — Fri, 17 Apr 2009 14:32:11 GMT

Hi IT_Architect,

False, my ghettoVCB.sh will continue to work because it does not rely on the VI API, it's using the unsupported SSH console to initiate it's backups. Let me know if you run into any issues

IMHO it's a bad move to go to ESXi 3.5u4 if you heavily rely on automating configuration or backup process using the RCLI/VI Perl Toolkit/Powershell binding to the VI API, I would probably stay on U3 if you're quite happy with the full r/w access to the VI API.

Re: Free ESXi Backup Solution for Windows

IT_Architect — Fri, 17 Apr 2009 09:19:31 GMT

With regard to this script, t appears upgrading to ESXi U4 would be a bad move. I would assume methods used in your script would no longer work.

Re: Free ESXi Backup Solution for Windows

Tonyjansen — Fri, 17 Apr 2009 09:24:50 GMT

There are idd snapshot files in the backup, but they are only 1 kb and the originals where 5 to 100 GB. It also seems that the snapshots are always 1 tot 5 and never more than that. Altough i notice that in the datastore that snapshops where going from 1 tot 13.

Re: Free ESXi Backup Solution for Windows

KBuchanan — Tue, 14 Apr 2009 12:40:59 GMT

Wasn't there some delta files? Each snap creates a delta file and xxxxxxx_0000# numbered file...I cant remember the exact naming format...but there should be more than 1 kb.

Kevin Buchanan
Chief Information Officer
Lexington Memorial Hospital
336-238-4286
kbuchanan@lexmem.org

Re: Free ESXi Backup Solution for Windows

Tonyjansen — Tue, 14 Apr 2009 05:04:57 GMT

Hi,

I'm using the backup script on a esxi server. Yesterday we had a complete server crash. Now i noticed that the script made backups, but that i downloaded a ful flat file and 5 snapshot files of 1 kb. This is the case for all backups.

Is there any way to recover the data, i opened the file with vmware workstation, and it only shows me the orinigal state of the server;, and not the uptodate version

Many thx

Re: Free ESXi Backup Solution for Windows

SNAFU58 — Fri, 10 Apr 2009 16:02:25 GMT

Chris,

I had my SFU and NFS set up the same. But your mention of giving the logged in user full rights got me to thinking.

I had mapped the Unix root account to my domain account which is the account that I am logged in with and is a local admin. It was the "local" admin that got me to thinking so I added a mapping of the root account to my local account on the WSFU machine, and even though I am still logged in with my domain account I can now establish the NFS storage. Seems a bit backwards to me but it works.

Thanks for your help. Now I get to try the script.

Re: Free ESXi Backup Solution for Windows

ChrisInTexas — Fri, 10 Apr 2009 01:19:17 GMT

I am not sure if this will help or not. The attached pics are how I have my windows NFS share and SFU set. One thing I remember that messed me up was that I had to give the logged in user full rights to the folder that I have set as the NFS share.

I am still using it just as I have originally set it up as, to be honest I have not touched it since. Though I am planning to try Veeam Backup as it is now ESXi capable.

Re: Free ESXi Backup Solution for Windows

Cellolein — Thu, 09 Apr 2009 22:40:53 GMT

Ich bin bis einschließlich 17.04.09 nicht im Hause. E-Mails werden nicht bearbeitet oder weitergeleitet. In dringenden Angelegenheiten wenden Sie sich bitte an meinen Kollegen Herrn Venker. Email: venker@schmitz-werke.com Telefon: +49 2572 927 0. Über unsere Telefonzentrale werden Sie an Herrn Venker weitergeleitet.

Re: Free ESXi Backup Solution for Windows

sanbe — Thu, 09 Apr 2009 22:40:53 GMT

I´m not in the office until 14.04.2009. Your E-Mail won´t be forwarded. In urgent cases contact "PTD DataCenter" oi-sd-datacenter-mailbox.ptd@siemens.com

With best regards,
Bernd Sandner

Siemens AG
GIO IT SHS 94
Freyeslebenstr. 1
91058 Erlangen, Germany
Tel.: +49 (9131) 7-35210
Fax: +49 (9131) 7-31537
Mobile: +49 (171) 2296136
mailto:sandner.bernd@siemens.com

Re: Free ESXi Backup Solution for Windows

spex — Thu, 09 Apr 2009 22:40:51 GMT

Guten Tag,

ich bin am 20.04.2009 wieder im Haus.

Bei wichtigen Problemen wenden Sie sich bitte an Herrn Dr. Matthias Weger.

Ihre E-Mail wurde nicht weitergeleitet

Danke
Stefan Holzwarth

Re: Free ESXi Backup Solution for Windows

SNAFU58 — Thu, 09 Apr 2009 22:33:24 GMT

ChrisInTexas,

I don't know if you are still monitoring this thread or not. I was following your directions for using WSFU and NFS to do backups in ESXi. Everything was going well until I got to adding the NFS storage in the VMI Client. I keep getting this error: "error during the configuration of the host cannot open volume vmfs volumes", So I did some searching and found posts saying that to get around this error you needed to allow anonymous access to the NFS folder and give Anonymous Logon Full Control of NTFS permissions. I have done that and now the error I get is "error during the configuration of the host nfs error unable to mount filesystem unable to connect to nfs server"

Any ideas what I may be doing wrong. It seems I am on the verge but cannot get this to work.

Re: Free ESXi Backup Solution for Windows

bones*79 — Sat, 28 Mar 2009 14:02:15 GMT

Sehr geehrte Damen und Herren,

Ich befinde mich in der Zeit vom 28. März bis zum 13. April nicht im Hause.

Ich werde Ihre Mail nach meiner Rückkehr beantworten.

Bei dringenden Anliegen wenden Sie sich bitte an Herrn Eisenberg.

Mail:
eisenberg@hospital-fritzlar.de

Telefon:
05622 997-7729

Mit freundlichen Gruessen

Martin Bolz

Leiter EDV-Abteilung
Fon: (05622) 997-650
Fax: (05622) 997-654
Mail: bolz@hospital-fritzlar.de

Hospital zum Heiligen Geist
Am Hospital 6
34560 Fritzlar
www.hospital-fritzlar.de

Re: Free ESXi Backup Solution for Windows

tahmad — Sat, 28 Mar 2009 13:50:48 GMT

thanks sanbe your script work as a charm,without any problem

Re: Free ESXi Backup Solution for Windows

duonglt — Wed, 25 Mar 2009 19:25:46 GMT

Hi,

Have you tried using the sparse option in the ghettoVCB.sh script?

VMware ESX 3.x and ESXi Scripts & Resources:
http://www.engr.ucsb.edu/~duonglt/vmware

Re: Free ESXi Backup Solution for Windows

dotcom — Wed, 25 Mar 2009 18:11:46 GMT

Does anyone know of a backup script that correctly handles the sparse files that ESXi creates?

A sparse file is a file that does not have all of it's space allocated. So for example, if I create a new 8g VM, and I don't install any data on it, it may look like it has 8g of space in use, but in reality it is close to 0. "ls -l" should show the 8g, "du" should show something close to 0 (if you have a real du that reports actual space used).

I tried using ghettoVCB.sh, and while the script works great, it is not correctly handling the sparse files. an empty 8g sparse file should take a couple seconds to cross the network, instead, it is sending 8g of data.

If ghettoVCB were to encapsulate and/or compress the sparse file in a manner that the sparse file was created as a sparse file upon restore, that would be the best solution, and it would reduce the needless bandwidth consumption, pushing gig after gig of 0's across the wire.

If you don't understand what a sparse file is, check out this link:

http://administratosphere.wordpress.com/2008/05/23/sparse-files-what-why-and-how/

I have moved 35 real machines into 35 virtual machines. My backups have gone from using 500g to using several terabytes. This is not a compression issue, it is an issue of storing data that should not exist.

And worse, now that non-existant data is real data, the ESXi server has to handle large blocks of 0's that used to not exist because the backups have filled-in what is supposed to be sparse data. This is having a performance impact on the network, and the drives themselves. Before a backup, vmware could clone a system in a couple seconds, and after restoring the backup, the cloning operation takes much longer. We must preserve the sparse files sparse state, otherwise it's not a real backup.

George

Re: Free ESXi Backup Solution for Windows

IT_Architect — Mon, 16 Mar 2009 23:35:26 GMT

Does anybody know if you convert to 6.5 while backing up if it preserves the ESXi snapshots?

If you ever want to see "Gigs Fly", and you are copying a VM between local disks or volumes, or moving/renaming a VM on the same volume, use the cp or mv commands at the ESXi command prompt. Volumes show under /vmfs.

Re: Free ESXi Backup Solution for Windows

DSTAVERT — Mon, 16 Mar 2009 23:22:14 GMT

StuartLittle

If you only need occasional copies for a single machine you can use the current Converter 4.0 tool. It doesn't schedule but you can basically rerun a previous conversion. It is reasonably quick and as an added benefit you can launch the "backup" and verify that it is viable.

Re: Free ESXi Backup Solution for Windows

lamw — Mon, 16 Mar 2009 15:40:42 GMT

Please try contacting the original author on that script, he/she may not be consistently monitoring this thread.

Re: Free ESXi Backup Solution for Windows

StuartLittle — Mon, 16 Mar 2009 15:32:42 GMT

Hello all,

Does anyone have any advice on modifying the Esxibackup.pl script to simply snapshot and backup a single VM server?

Thanks - in anticipation.

Stuart

Re: Free ESXi Backup Solution for Windows

IT_Architect — Sun, 15 Mar 2009 21:45:16 GMT

I just had a thought that may have already been considered. To make a long story short, the VI Client messed up my vmdks, and I was able to salvage them by running the VM through the converter to Workstation 6.5 and back to ESXi. However it did not escape my attention that the Workstation vmdk that was created was only 5.5 gigs uncompressed compared to the previous 42 gigs worth of files that came in from ESXi. The conversion to workstation was a lot faster than converting back to ESXi's 36 gig vmdk. We do a lot more backups than restores. If that concept plus compression could be applied to the backup process...

Re: Free ESXi Backup Solution for Windows

lamw — Sun, 15 Mar 2009 19:02:33 GMT

Yes, those are consider write operations, any changes to the system using the RCLI are considered write operations. Per the ESX vs ESXi comparison chart, it looks like you'll need at least the Foundation license to allow both r/w operations. This is actually something we're still investigating and hopefully sales will be able to give us a definitive answer, I've heard from various sources that there might be an API license for ESXi because Foundation by default provides vCenter and not all users may need or want to run vCenter. I can update once we get a definitive answer, but from that document it looks like you need at least the Foundation to have full access to the RCLI.

Re: Free ESXi Backup Solution for Windows

fahrer — Sun, 15 Mar 2009 18:59:37 GMT

Does this effect creating snapshots and shutdowns of vms over the rcli?

Re: Free ESXi Backup Solution for Windows

lamw — Sun, 15 Mar 2009 18:47:06 GMT

When you unlock the unsupported ESXi console you'll notice in red that it's called Tech Support Mode:

From http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1003677&sliceId=1&docTypeID=DT_KB_1_1&dialogID=9014578&stateId=0%200%2010259445

Tech Support Mode provides a command-line interface that can be used to diagnose and repair VMware ESXi hosts. Tech Support Mode should only be used at the request of VMware technical support.

When you have issues with ESXi, VMware support may be using the Tech Support Mode to help troubleshoot issue. It's unsupported in the fact that user's should not be opening up the console and using it as they have in the past with the full Service Console in the classic ESX.

As you may or may not be aware, RCLI read/write operations currently works in ESXi 3.5u3, this was done inadvertently while fixing an internal VMware API bug. Either in a future release or patch update, the RCLI will only have read access if you're using the free version of ESXi. This is important to note when the fix is out and the backup solution that uses the RCLI may no longer function unless you're paying for an ESXi license.

In terms of the speed of the RCLI, it's well known by VMware and hopefully performance will be fixed in a future release.

Re: Free ESXi Backup Solution for Windows

IT_Architect — Sun, 15 Mar 2009 18:42:37 GMT

ghettoVCB is kept up-to-date at: http://communities.vmware.com/docs/DOC-8760 and there is also detailed instructions on setting this up in a Windows environment and scheduling backups provided by Raj Perumal.

Now there's something I can use.

Thanks tons!

Re: Free ESXi Backup Solution for Windows

KBuchanan — Sun, 15 Mar 2009 18:19:35 GMT

The RCLI is supported, but the remote console on ESXi isn't. I had a support case e weeks ago. After the tech asked about the problems, they asked if I could start a console with putty. NOW...isn't interesting that they asked if I could start a remote console when SUPPOSEDLY, it isn't supported!?!

I figure they don't want to open the OS on the "free" version...maybe because there is an assumption that the admins will not be trained at a reasonably competent level (compared to an ESX admin). I don't know, that's just my assumption.

That was my first ESXi support case. He didn't asked if I had enabled the remote console...he assumed it.

SO...even though it says it isn't supported, the tech went straight to it!

Kevin Buchanan
Chief Information Officer
Lexington Memorial Hospital
336-238-4286
kbuchanan@lexmem.org

Re: Free ESXi Backup Solution for Windows

kchawk — Sun, 15 Mar 2009 17:45:18 GMT

All of the scripts are great and the repsonse is great. I use this forum as an example when I am trying to get management to use more open source apps and to use the community support model. Anyway, my question is about the supported methods for rcli, is the speed issue wih vifs still an issue or is there another method without using the unsupported console method?

Re: Free ESXi Backup Solution for Windows

lamw — Sat, 14 Mar 2009 22:52:40 GMT

You should create a VMTN document on your script and processes, that way it can be easily accessible and kept up-to-date for others to find

FYI - The next revision of ghettoVCB will be using supported APIs/platforms by VMware =)

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Re: Free ESXi Backup Solution for Windows

StuartLittle — Sun, 15 Mar 2009 11:58:41 GMT

Hello all,

Firstly thank you to everyone who made these scripts available - they are a god send! I understand the scripts have been developed and have evolved since their first creation and believe the GhettoVCB is the most recent version however I currently use the Esxibackup.pl perl script to backup my VM's and it works great.

The one question I have and was hoping someone could help with was: how or what code do I need to backup a single VM?

At the moment the Esxibackup.pl script creates a snapshot for every powered on VM which is currently 12 servers and backups whatever one I label in the code but I only want to backup a single server without all the snapshots ... the reason I'm asking is because I'm running low on local storage space and won't have enough for all the snapshot's it creates for the 12 servers.

Is there a way I can simply code the script to snapshot and backup a single server please?

Look forward to any help and thanks in advance!

Kind regards,

Stuart

Re: Free ESXi Backup Solution for Windows

KBuchanan — Sat, 14 Mar 2009 22:49:36 GMT

I'm away from my PC, but I'll post my script. It uses the RCLI and the SDK - both of which ARE SUPPORTED by VMWARE.

My script will backup (clone) from multiple sources and will backup TO multiple targets.

Sure they are many options available...but ultimately, the admin needs to assess the options on THEIR OWN to be sure the scripts work in their environment and meets their own specific requirements.

Kevin Buchanan
Chief Information Officer
Lexington Memorial Hospital
336-238-4286
kbuchanan@lexmem.org

Re: Free ESXi Backup Solution for Windows

lamw — Sat, 14 Mar 2009 22:33:52 GMT

I agree, initially I was just providing a quick script to run a backup, but there were some very interesting feature requests and this thread just started to grow. I have not used the other solutions provided in this thread and I'm not sure what are the pro/con(s) of each and the latest revision of the scripts.

ghettoVCB is kept up-to-date at: http://communities.vmware.com/docs/DOC-8760 and there is also detailed instructions on setting this up in a Windows environment and scheduling backups provided by Raj Perumal

At the end, it's really up to the end user's and their requirements for their environment to decide what is the best solution. I know Veeam Backup is another popular tool that also supports both ESX/ESXi and the free version of ESXi. It would be nice for someone that has used all the tools and provide a small post on the pros/cons but even then that may spawn an even larger post, not mention other 3rd party backup tools that are not free.

=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Re: Free ESXi Backup Solution for Windows

peedy — Sat, 14 Mar 2009 18:00:08 GMT

Amen to that! I'm trying to make heads or tails of all the forks and Im lost.

Re: Free ESXi Backup Solution for Windows

sanbe — Sat, 14 Mar 2009 17:49:36 GMT

I´m not in the office until 16.03.2009. Your E-Mail won´t be forwarded. In urgent cases contact "PTD DataCenter" oi-sd-datacenter-mailbox.ptd@siemens.com

With best regards,
Bernd Sandner

Siemens AG
GIO IT SHS 94
Freyeslebenstr. 1
91058 Erlangen, Germany
Tel.: +49 (9131) 7-35210
Fax: +49 (9131) 7-31537
Mobile: +49 (171) 2296136
mailto:sandner.bernd@siemens.com

Re: Free ESXi Backup Solution for Windows

IT_Architect — Sat, 14 Mar 2009 17:46:02 GMT

This thread was a fine discussion to hammer several different methods out, with many changes and fixes. The problem with it is, at least separte 4 forks evolved, and changes to each. After reading 26 pages, nobody will have any idea what to do or where to find the most up to date scripts and instructions. There needs to be a new thread that is a summary of this thread, the lists the methods, the advantages and disasdvantages of each approach, and where to find the most current code and instructions for each.

Re: Free ESXi Backup Solution for Windows

lamw — Sat, 28 Feb 2009 21:40:01 GMT

There are some instances where snapshots created from the CLI using vimsh will not allow you to remove the snapshot from VIC.

What you can do is SSH to console (if you're comfortable) and you'll want to search for your VM using the vim-cmd.

1. Locate the VmId for the VM in question, looking at your screenshot I'll assume the name is Windows2003

vim-cmd vmsvc/getallvms | grep -i "Windows2003"

This should return you a single row and the first column will be a numeric value representing the VmId for this virtual machine

2. You will then take that VmId and pass it to the following command to remove the snapshot, I assume you only have 1 snapshot which was created by the backup script? (please double check prior to running)

vim-cmd vmsvc/snapshot.remove [VmId]

You also have the option of committing all snapshots if you have more than one and also if you only have one you can also use the following command:

vim-cmd vmsvc/snapshot.removeall [VmId]

This should commit your snapshot and allow you to completely process it.

=========================================================================
--William
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Re: Free ESXi Backup Solution for Windows

rgarzar — Sat, 28 Feb 2009 19:56:14 GMT

Hi....

I'm testing this script and I have a little problem. The script its stuck (a week) removing an snapshot from a Windows 2003 guest and I don't want to damage this VM.

If I try to remove the snapshot from the VIC, it doesn't finish (95%). What can I do to cancel the script and remove correctly the snapshot?

Any ideas?

Re: Free ESXi Backup Solution for Windows

shechtl — Thu, 26 Feb 2009 04:24:08 GMT

Re: Free ESXi Backup Solution for Windows

ing2k-simone — Fri, 27 Feb 2009 14:00:18 GMT

i found a solution with this command

./ghettoVCB.sh VM_backup_list 2>&1 | tee ./logfilename &

Re: Free ESXi Backup Solution for Windows

ing2k-simone — Fri, 27 Feb 2009 11:00:38 GMT

I've the same problem with nohup command!

it seems that the nohup command under ESXi only redirect output on nohup.out!

and if i try to run the command with plink.exe, the file nohup.out is not created!

my command is:

plink.exe ~~root@host~~ -pw mypasswd "nohup ./vmfs/volumes/DSiS01_BCK/VCB_Script/ghettoVCB.sh /vmfs/volumes/DSiS01_BCK/VCB_Script/VM_backup_list &"

anyone have a workaround?

Re: Free ESXi Backup Solution for Windows

johnnymo — Thu, 26 Feb 2009 04:15:27 GMT

Hi Sanbe,

I am trying your script but i have had no success until now. I have edited all variables that i think are necessary,
but no go, the snapshots are made in all VMs but when its time for the download of the actual files i get a 404 error.

2009-02-26 03:54:45 - Copy VM files to local storage -----

GET https://my_url_here/folder/datastore1 unsuccessful : 404 Not Found
GET https://my_url_here/folder/datastore1%20hsone unsuccessful : 404 Not Found

(hsone is one of the VMs)

After this the snapshots are deleted and script ends. I'm no command line or linux expert...
I'm running VMware ESX Server 3i 3.5.0 build-123629, can you help?

TIA

Problem solved... my stupidity. Instead of "datastore1" in $DSPath i put "datastore1".
It's all in the script... Thx!

Re: Free ESXi Backup Solution for Windows

spex — Thu, 26 Feb 2009 04:24:07 GMT

Guten Tag,

ich bin am 02.03.2009 wieder im Haus.

Bei wichtigen Problemen wenden Sie sich bitte an Herrn Dr. Matthias Weger.

Ihre E-Mail wurde nicht weitergeleitet

Danke
Stefan Holzwarth

Re: Free ESXi Backup Solution for Windows

spex — Mon, 23 Feb 2009 10:04:07 GMT

Guten Tag,

ich bin am 02.03.2009 wieder im Haus.

Bei wichtigen Problemen wenden Sie sich bitte an Herrn Dr. Matthias Weger.

Ihre E-Mail wurde nicht weitergeleitet

Danke
Stefan Holzwarth

Re: Free ESXi Backup Solution for Windows

shechtl — Mon, 23 Feb 2009 09:54:49 GMT

Hello lamw,

yes my problem are the SATA hdds in the storage. I changed to SAS and its running.
But since friday I have a new strange problem.

If I try to start ghetto-vb on the esxi 3.5.0 Build 143129 I get the message
ash: Cannot fork

This problem is now on all 3 ESxi server. The vm are all running, I can access the esxi per infrastructure client, but I can access only 2 esx server per ssh, the third is not accepting the ssh connection anymore.
I shutdown all vm and tried to reboot the esx. The reboot does not work. I had to make a reboot direct on the console. After that its running fine.
Maybe someone knows what the problem could be ?
With esxtop I could not see any problems.

Re: Free ESXi Backup Solution for Windows

Bigun — Wed, 11 Feb 2009 13:58:03 GMT

Found the problem:

The version of ghettoVCB.sh from this link:

http://communities.vmware.com/docs/DOC-8760

Seems to have some Windows "^M" characters in it.

I went to a Linux machine and used wget to retrieve the script. We fixed it by running it through a text editor that could get rid of them.

Re: Free ESXi Backup Solution for Windows

Bigun — Tue, 10 Feb 2009 21:51:46 GMT

We have no trouble running an older version of your script lamw.

Recently we tried to update the script and ran into this error:

"hs-vsbackup1 ~ # plink -pw ********* root@hs-vs1.hcso.hcsheriff.gov /vmfs/volumes/Backups1/backupvs1.sh
/vmfs/volumes/Backups1/backupvs1.sh: /vmfs/volumes/Backups1/backupvs1.sh: 2: /vmfs/volumes/Backups1/ghettoVCB.sh: not found"

So I logged into the ESXi server directly and ran it:

" /vmfs/volumes/63cf7fd9-fab2ac18 # ./ghettoVCB.sh
-ash: ./ghettoVCB.sh: not found"

What are we missing? The file is there:

" /vmfs/volumes/63cf7fd9-fab2ac18 # ls -la ghettoVCB.sh
-rwxrwxrwx 1 nfsnobod nfsnobod 11536 Feb 10 21:31 ghettoVCB.sh"

Re: Free ESXi Backup Solution for Windows

KBuchanan — Tue, 10 Feb 2009 04:23:05 GMT

Here is a script that I use to backup my ESXi servers with the VMs running on local DAS.

Basically, we have 2 separate RAID-5 volumes on the local server, and each is mounted as a datastore. We also have a NFS mounted datastore (Celerra NS20).

The script will snap the image, make the first copy to local storage, then make a second copy to the NFS storage. The script has the logic to also create an "archive"...which is the previous image backup.

In short - the script will provide 2 of recent image backups on local storage and 2 on the NFS remote storage.

The script will also generate a timed log file with a lot of output...but you can modify the output to minimize to what you want. The "runbackup.bat" calls the "backupscript2Celerra.bat". We have these batch files installed on a server with RCLI and putty/plink to run as a scheduled task. So far, it has run for nearly 1 month without problem.

This script is derived from another script I found...but I don't remember where, so I apologize for not being able to "give credit". I'm quite certain that it was contributed to the communities.vmware.com forum. SO >>> Thanks to whomever gave me the ideas to build on!

-Kevin

Re: Free ESXi Backup Solution for Windows

ipadl — Mon, 09 Feb 2009 23:52:45 GMT

Hey guys,

I've been using Sanbe's ESXi-Backup.pl script modified by Khaliss (ESXi-Backup_mod_by_Khaliss.pl) to copy a running VM to an openfiler NAS unit as stage 1 of my backup strategy. Unfortunately, I can't figure out how to output the "printed" lines of that script to a log file. I have been searching the web to see if there is an easy way to do this, but can't seem to make any of the suggestions work. Has anybody here been doing the same and outputting to a logfile? Thanks in advance,

Seth

Re: Free ESXi Backup Solution for Windows

HarisB — Sat, 07 Feb 2009 01:27:59 GMT

Hi guys,

My Perlese is definitely not at the level where I can understand all the greps, awks and regular expressions the way lawm wrote them. I've spent some time trying to do what I need and I just can't get there.

So I'm hoping someone already did this and they can share the code.

I need to setup an exclusion list rather than inclusion list as it is now. So my concept is this:

Get all the VM names from the exclusionlist.txt into an array, call it $Exclusions (or @Exclusions)
Get the list of all VMs on the system by using vmsvc/getallvms (as done already, captured in /tmp/vms_list
Now compare the two, remove any VMs that appear in exclusions from /tmp/vms_list
Proceed with backup based on /tmp/vms_list, one by one.

I could code this in AutoIT but it's useless in shell / ESX environment.

Much appreciated,

Thanks

Re: Free ESXi Backup Solution for Windows

IT_Architect — Sat, 07 Feb 2009 00:32:58 GMT

there's been plenty of user's trying NFS on a VM and saw less than up to par performance

Hmm...I had to have time to reflect on that. So to summarize, from your experience and anything you have seen, when a VM runs the NFS, there will be a performance problem whether that VM is local or remote? I assumed that across the network to a VM running NFS would be as fast as going to a dedicated NFS host and that the network would be the limiting factor.

What if I ran Xtravirt Virtual SAN Appliance on both machines, and did the cross backups that way? The backup would be going from one data store to another except it would be across the network. I know you mentioned it, but I don't know if you have experience with it or not. What's your best guess as to whether or not that might eliminate all or most of the VM performance penalty?

see if you can get some loaners

The servers are dedicated servers that we rent by the month. We can add or remove anything we want to including drives, memory, and CPUs. iSCSI costs $.75/gig and NFS is $.50/gig per month. If it ends up making more sense in the overall scheme of things, we will do this.

It's nice to get a design/plan out, but good part of the implementation is to also verify and test that the solution will in fact work as stated and in a timely manner that is expected.

True, but human progress depends on learning from the experiences of those who have gone before. I'm trying to narrow down the list of things to try to those that have the highest probability of success.

Re: Free ESXi Backup Solution for Windows

lamw — Fri, 06 Feb 2009 19:01:30 GMT

Well if you're planning on running an NFS Server whether Windows/Linux, just be warn if you don't get the speeds you're expecting. We've done tests in the past and having a physical NFS Server just gave us better performance, not to say you can't or you won't see decent speeds but there's been plenty of user's trying NFS on a VM and saw less than up to par performance. Cost is always a factor when deciding a solution but before you look at that, give some of these scenarios a run before committing to any purchases, I know it's hard if you're talking about local storage/etc. but see if you can get some loaners, I know HP has programs for their servers as such but not sure what it covers when talking about disk(s). If you happen to have spare hardware around, try this in a lab to see what type of speeds you're getting and what you're willing to deal with. I assume that may also be a driving factor balancing cost and performance.

It's nice to get a design/plan out, but good part of the implementation is to also verify and test that the solution will in fact work as stated and in a timely manner that is expected.

Good Luck

=========================================================================
--William
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Re: Free ESXi Backup Solution for Windows

IT_Architect — Fri, 06 Feb 2009 18:45:32 GMT

Thanks TONS for repsonding so rapidly.

So in my opinion, this is not a bad way to go about for recovery solution and if I understand you correctly you're looking at a 2 phase backup strategy? Both ESX-1/2 have mirrored 146 GB and they each have an additional SATA drive which will be local storage backup? Then you're also looking at implementing an NFS Server running on Windows as additional backup which can be mounted as a datastore to either host as an extra precautionary backup?

No, my thoughts were to use only local storage. One of the VMs would be a WIndows VM running Windows and presenting an NFS volume by using the local 250 SATA. However, the backups for each server would be stored on the opposite server. If one server goes off line, the other could run the VMs for both servers temporarily. I could also restore a new server from a backup on the opposite server since it would contain the backups.

I've not used Windows NFS Services before, but as you probably noticed from looking at this thread and at the other threads regarding NFS backup on either Linux/Windows, speeds will vary depending on configurations/etc.

Reading the thread, speeds are all over the map. I don't what is fast and what isn't. I haven't read all of the side threads, including the ones you started specifically for NFS and another related one yet. The private network where I'm at has reputation of people are ditching their local drives because they say the iSCSI and NFS is faster than local plus it's redundant. In my mind, that statement doesn't fly because the math doesn't work for a gigabit interface compared to local storage. However, it does make a statement about the quality of their private network. Thus, backing up from one ESXi server to the opposite server's NFS VM, should be about as good as it gets across a network, whatever that may be. I can use these 250s any way I want to or ditch them. If it makes more sense to backup to the local 250 and then ferry it across, I'm open to that also. 500 Gigs on their NAS would cost me about $250/mo. whereas 250 gigs on each server would cost me about $40/mo. I'd like to hear what you would do.

Re: Free ESXi Backup Solution for Windows

lamw — Fri, 06 Feb 2009 16:50:31 GMT

Hi IT_Architect,

Glad you took some time to go over the thread. Majority of the time questions can be answered by going over past comments, though this is a pretty hefty thread to have to go through.

So in my opinion, this is not a bad way to go about for recovery solution and if I understand you correctly you're looking at a 2 phase backup strategy?

Both ESX-1/2 have mirrored 146 GB and they each have an additional SATA drive which will be local storage backup? Then you're also looking at implementing an NFS Server running on Windows as additional backup which can be mounted as a datastore to either host as an extra precautionary backup?

The additional local storage backup is good idea but could also be redundant. If you have your boot/vm drive mirrored then that should protect against failure of one of those drives but having an additional drive within that same chassis will not protect against hardware failure of that system (cpu,memor,networking,psu,etc). Depending on the number of VM(s) you're looking to manage in this environment and if you plan for growth, what you could do is leverage the NFS Server for backups but also using the fact that it can be mounted on either ESX-1/2 to create a replicate copy of each hosts VM(s) on the other server's local storage. If you run into hardware failure, you automatically have the other set of VM(s) residing on the opposite host that can be booted up relatively quickly assuming all configurations are matched exactly the same and you still have all backups stored on the NFS Server.

Again, this strategy will only be as useful as the redundancy you have on external dependencies such as redundant network configurations and to external switches, power supply, etc.

I've not used Windows NFS Services before, but as you probably noticed from looking at this thread and at the other threads regarding NFS backup on either Linux/Windows, speeds will vary depending on configurations/etc.

Hopefully this answered some of your questions and there are many solutions to the problem, so by all means post in the overall ESXi forum if you want additional feedback about your back/recovery design.

=========================================================================
--William
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Re: Free ESXi Backup Solution for Windows

tman24 — Fri, 06 Feb 2009 16:24:15 GMT

I think I've actually resolved my own problem. I inserted the snip of code from a couple of pages back that will search for all vmdk's linked to the VM. This actually seems to work really well, and in my testing, it does indeed backup all the vmdk files, even if they are spread across multiple data stores. This has removed the error (25) I was getting, and the script now runs clean. This though, actually introduces another issue that I'd like to be considered.

A VM will often have a small O/S vmdk, and a larger DATA vmdk (at least that's how I configure them). This allows me to locate vmdk's for the VM in different datastores that are optimized for that particular type of storage. The issue is that the actual files in the DATA vmdk's are often backed up using traditional methods (i.e. Tape) at the filesystem level, and I'd rather not have these cloned via the script. These data vmdk's are ususally larger (>20GB). I also have seperate vmdk's to hold VM host swap files, which are generally smaller (<4GB), and I'd rather not have these backed up either as they can be easily re-created.

What I thought was, would it be possible to include upper and lower vmdk size limits to we can control which vmdk's get cloned by the script. I'm thinking if you set both the uppper and lower variables to 0, it would backup everything, while setting a lower limit would not backup vmdk's smaller than the value, likewise the uppper limit would skip vmdk's >= the value. This would allow granular control of which vmdk's are cloned even if they are spread across multiple stores. If I went back to the original script it would only backup the vmdk's in the primary VM folder, but we'd start getting the error (25)'s again when the script fails to find linked vmdk's.

Re: Free ESXi Backup Solution for Windows

IT_Architect — Fri, 06 Feb 2009 16:19:48 GMT

I’m in search of a good backup solution for ESXi as well as a high-availability strategy for a few VMs across two servers that are hosting a few web server VMs. I can administer them remotely via a VPN. I’ve read through every post, which took the better part of day to assimilate. Here is how my thoughts are gelling for me. I’d like feedback as to any “flys in the ointment” you may see in my thought process.

Situation:
Two web servers where I need good backups, rapid recover, and near HA on some of the VMs.

Thoughts:
1. Two servers with dual 146 SCSI’s in a mirror to run the VMs from and to boot ESXi. Install a 250 Gig SATA II on each server for backup. They each backup to the other’s SATA. I have 2 gigabit network interfaces, one private that can access a private network and all resources, and a public one. If one has a problem, the other can mount the VM backup of the other machine. I can reverse the source and destination in lamw script until I restore the other machine. I have a gigabit NFS and iSCSI available if they somehow are needed to play a role in the HA scheme, I would still need the 250 SATAs since I jwould still need a backup to have a plan B if something goes wrong with their array. The only thing that would change if I used the hoster’s NFS or iSCSI is I would not store the backups locally instead of on the opposite server.

2. For an NFS host, the best pick would be Windows Server 2008 Web Edition (32bit). It would inhabit the 250 SATA drive. It could run Windows Services for UNIX (WSFU) to present an NFS data store to ESXi. Windows is the only OS that can run the VI Client. A lot of the services of the client would be a lot more useful since they don’t need to traverse a WAN. I could connect to it using RDP over the VPN to run the client. I could simply map a drive to it via the VPN and drag files to and from it. It has an easy to use and easy to audit scheduler. All other VMware tools are available for both Windows and ?NIX. People report the 32 bit version runs much faster under ESXi than the 64 bit. Windows server is required when I upgrade to other VMware services. Windows Server 2008 Web Edition is free where I’m hosting. It also allows me to run Windows-based web sites for applications that require MS technologies. These technologies could also be shared by ?NIX hosts such as MSSQL etc.

3. Use Windows to kick off lamw’s script to backup to the opposite server across the private network.

I'd appreciate your thought about this strategy.

Re: Free ESXi Backup Solution for Windows

Cellolein — Fri, 06 Feb 2009 12:48:26 GMT

Ich bin bis einschließlich 13.02.09 nicht im Hause. E-Mails werden nicht bearbeitet oder weitergeleitet. In dringenden Angelegenheiten wenden Sie sich bitte an meinen Kollegen Herrn Venker. Email: venker@schmitz-werke.com Telefon: +49 2572 927 0. Über unsere Telefonzentrale werden Sie an Herrn Venker weitergeleitet.

Re: Free ESXi Backup Solution for Windows

tman24 — Fri, 06 Feb 2009 12:38:33 GMT

I'm testing ghettoVCB as a backup solution for our ESXi production servers. We currently have a two-node ESXi cluster with iSCSI SAN shared storage. There are two primary datastores on the SAN mounted into each ESXi host, one store for the VM O/S vmdk, the other (larger) store for the VM data vmdk. We also have an Openfiler 2.3 host between the two, and each ESXi host mounts a NFS share for temp storage. We use the temp storage for additional disks that are mounted into each VM specifically for swap files. We also use the NFS store for vswp files and also for snapshot creations. This is accomplished by using the 'WorkingDir' variable in the VM .vmx file. This works fine, and means unnecessary temp/swap data is kept off the SAN.

When ghettoVCB runs, the snapshot is created in the host folder on the NFS share ok, and the backup is created on another NFS mount (vmbackup) on another Openfiler 2.3 host, but I still get the following error when ghettoVCB starts backing up each host;

DiskLib_Check() failed for source disk The system cannot find the file specified (25)

The snapshots are being deleted ok after ghettoVCB finishes, so it's using the external volume ok. As we have the vmdk's split over multiple datastores, could this still be the problem?

Re: Free ESXi Backup Solution for Windows

kchawk — Wed, 04 Feb 2009 20:47:01 GMT

I attach the vm I am restoring from to a blank network, power it on and then change the ip. My admin vm is connected to the same blank network, two nics, which allows me to copy to where ever I need. Another way I have done it is to power down my admin vm and attach the vmdk of the vm I want to restore from. For me the first option is better because I am mostly working with linux vms but the admin pc is xp for VI Client.

Re: Free ESXi Backup Solution for Windows

ipadl — Wed, 04 Feb 2009 19:58:53 GMT

Wow kchawk, that's an awesome solution that I was unaware of. We might change our file level backup and recovery strategies with this information.

Obviously, it could cause major problems if you were to bring up two identical guests on your LAN, especially if they are AD servers. If you bring up the guest with the NICs disabled, is there an easy way to grab a file through the VIC and transfer it to your local machine? I am starting to research shared folders in VMware tools. Is this how you are accomplishing this?

Thanks in advance!

Seth

Re: Free ESXi Backup Solution for Windows

kchawk — Wed, 04 Feb 2009 19:20:10 GMT

Instead of creating a new vm you can browse the backup ds and select the vmx file of the vm you want
to run, right click and select add to inventory, make sure to leave the name field blank so it uses the same name.

Then after your prod box is back up reverse the source/dest datastores on your backup script to move the vms back, then readd to inventory again.

This is also how I do file level restores. I just add the backed up vm to inventory power on to another network and restore the files.

Re: Free ESXi Backup Solution for Windows

kchawk — Wed, 04 Feb 2009 19:18:31 GMT

double post

Re: Free ESXi Backup Solution for Windows

lamw — Wed, 04 Feb 2009 18:53:45 GMT

Has anybody spent any amount of time working on the restore functionality of each of these scripts, especially in a disaster recovery type environment?

Personally I believe the restore process is going to be up to the end user's to implement. There is no, one solution fits all for restoring your VM(s) whether it's a recovery from backup or a disaster recovery. Even the backup process itself will vary from environment to environment as you can see from the extensive feature requests within this thread on the various scripts. I think however you choose to restore your VM(s), if you're able to manually walk through that process following a document (which you should) then it can be automated with a script and should be designed/implemented by the end users. If you're looking for a more streamline approach with the ability to test your DR/recovery, take a look at VMware SRM. It's a great product if you're looking to do advanced testing and planning with a recovery/DR.

=========================================================================
--William
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Re: Free ESXi Backup Solution for Windows

ipadl — Wed, 04 Feb 2009 18:40:03 GMT

Has anybody spent any amount of time working on the restore functionality of each of these scripts, especially in a disaster recovery type environment?

I am currently doing a two-stage backup using the following:

1. ESXi-Backup_mod_by_khaliss.pl to snapshot and copy my server to an openfiler NAS attached to the ESXi server as an iSCSI target

2. A third party beta software designed to transfer files from the NAS to a USB local to a windows XP machine that allows me to walk away from the office with my server on a drive.

Network Information:

1x HP Proliant DL360G5 Production Server with ESXi 3.5.0 w/ Intel + Broadcom Gigabit NICs

1x HP Proliant DL360G5 Test Server with ESXi 3.5.0 w/ Broadcom Gigabit NIC

Gigabit network

Guest: Windows SBS Server 2003 ~600GB in size

Openfiler NAS w/ Gigabit NIC and 1x 1TB Sata HD, no raid currently

Current Backup procedures:

1. Nightly ESXi copy from ESXi to Openfiler NAS

2. Nightly file-level recovery, exchange, and system state backups using ntbackup

3. Weekly copy of Openfiler NAS to walk away USB drive

So here's the scenario that I am currently running in my mind. The Production Server hardware goes kaput for some unknown reason in the middle of the day. We fire up the Test Server, attach the NAS as an iSCSI target, and create a new VM pointing at the vm files on the NAS. I have tested and this works perfectly. However, a second folder is created called Servername_1 on either the NAS datastore or in the local datastore, depending on where you choose when creating the new vm using the the "New Virtual Machine" wizard in ESXi. I would assume that we will see a bit of a performance drop using the NAS as our datastore than if the data were local to the ESXi machine, so we will probably want to transfer the data from the NAS datastore to the test server datastore (or back to the Production server if it is fixed at this time) during the weekend. How is this effectively accomplished?

Do we need to pull all of the files from the NAS datastore Servername folder(servername.vmx, servername.nvram, servername.vmdk, vmware-core.gz, servername.vmsd, servername.vmxf) AND the files from the new Servername_1 folder (Servername_1.vmx, vmware.log, Servername_1.nvram, Servername_1.vmxf, Servername_1.vmsd)? I am assuming that the files in the new folder (Servername_1) are simply pointing to the files in the old folder on the NAS (Servername) and any changes are made on the files in the old folder on the NAS, so do we just create a new local vm with the right attributes and replace the data files in the datastore folder with the files from the NAS?

If that's confusing, let me know and I'll try to rephrase it properly. Thanks for your help!

Seth

Re: Free ESXi Backup Solution for Windows

lamw — Sat, 31 Jan 2009 22:08:56 GMT

Nope, not sure. I would take a look at your storage since it's showing I/O error. Try to manually vmkfstools your disks a couple of times manually and see if you get the error, if you do, then it's your storage. This most likely is something within your environment than the script.

=========================================================================
--William
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

Re: Free ESXi Backup Solution for Windows

shechtl — Sat, 31 Jan 2009 19:48:29 GMT

Hello lamw,

in your ghettoVCB-enhance I get sometimes this error message: 79% done.Failed to clone disk : Input/output error (327689)
I get this error with different VMs, sometimes, not always. I use the feature to shutdown the VM first.
I have this problem on two ESXi Server. The storage is a open-e DSS mounted per NFS. The backup storage is also DSS with NFS.
The VMs are XP and 2003 Server. HDDs are between 15 and 120GB.

Do you know what the problem could be ?

Re: Free ESXi Backup Solution for Windows - broken software mirror

SeanT77 — Thu, 29 Jan 2009 22:26:28 GMT

I'm not sure if this has been offered yet, too many posts to read them all. I have used software raid1 to script the following: mirror the vmdk1, break the mirror, disconnect the vmdk2, copy off the vmdk2 to a backup location, reconnect the vmdk2 and remirror the vmdk1/vmdk2. You can even mirror the vmdk2 on a network store to be backed up.

Re: Free ESXi Backup Solution for Windows

willmarmn — Thu, 29 Jan 2009 19:07:55 GMT

I tried the command and everything works, except for the log output when using nohup. Without nohup, it works fine, but when I run it as stated, the file get created but is 0 bytes. When I run the command directly on the ESXi host, it seems that is continues to want to output the log to the nohup.out file. Anyone experience this issue?

Re: Free ESXi Backup Solution for Windows

lamw — Tue, 27 Jan 2009 22:32:31 GMT

I haven't checked the forums, but I had tried to upload few files and it took longer than it should but then again I haven't really dived into this type of backup solution so I can't say 100% sure.

=========================================================================
--William
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/