Announcement: New (free) backup utility realease.

[admin]

Hello all,

I've written a backup utility, it's currently still in 'beta' but I think I've worked out most of the bugs and it's been working very well for me, so I thought I'd share my work with the community.

I'll be trying to update it fairly regularly, hot-clone of a VM or specific VMDK attached to a VM planned for next version.

Please see the readme file here for some initial instructions: http://users.ox.ac.uk/~alexm/README

You can download the utility here:

http://xtravirt.com/index.php?option=com_remository&Itemid=75&func=select&id=5

Once you've extracted the archive, please read the README file in the directory created for instructions on using the utility.

Any problems post them to this thread and I'll try and resolve them, I'll set up a proper website when I have the time.

Alex

p.s. Big thanks to acr for help with testing!

Added link to the latest release - 21st May 2007[/b]

Message was edited by:

MR-T: Updated with link to Xtravirt site

[PCS_Dan]

My password had special characters in it. I manually edited the config

file and added single quotes to my password and it seems to work now.

It is still spitting out the error "/usr/sbin/vcbSnapshot: line 1:

localhost: command not found" randomly but it seems to be backing it up

still though

Also - make sure your /etc/hosts has the correct IP for your

hostname. I had the issue where I changed the host IP but the hosts

file still pointed to a previous IP. After that change you need to

restart the network service

[CommVault]

Hello Mittel

The information helped me a lot. But I need some documentation and configuration for installing commvault on linux.

- Thanks in Advance

- Gopala

[dwightmccann]

Unexpected errors running VISBU: noticed a VM not getting backed up. From the vi-backup.log:

12/03/2009 01:44:41: rhel-dbtest log file = /vmfs/volumes/SAASB-Backup/visbu/rhel-dbtest.log

12/03/2009 01:44:41: Snapshotting VM rhel-dbtest....

12/03/2009 01:44:43: Snapshot of rhel-dbtest successful.

12/03/2009 01:44:43: Exporting disks for rhel-dbtest....

12/03/2009 01:47:26: Export complete. Disks exported without errors.

12/03/2009 01:47:26: Removing Snapshot for rhel-dbtest....

12/03/2009 01:47:30: Snapshot removed successfully from rhel-dbtest.

12/03/2009 01:47:30: Exporting VM config and log file...

12/03/2009 01:47:30: Config and log file exported successfully.

12/03/2009 01:47:30: SystemsAD3 log file = /vmfs/volumes/SAASB-Backup/visbu/SystemsAD3.log

12/03/2009 01:47:30: tomcat_dev log file = /vmfs/volumes/SAASB-Backup/visbu/tomcat_dev.log

12/03/2009 01:47:30: Snapshotting VM tomcat_dev....

12/03/2009 01:47:33: Snapshot of tomcat_dev successful.

12/03/2009 01:47:33: Exporting disks for tomcat_dev....

12/03/2009 01:50:11: Export complete. Disks exported without errors.

12/03/2009 01:50:11: Removing Snapshot for tomcat_dev....

12/03/2009 01:50:15: Snapshot removed successfully from tomcat_dev.

12/03/2009 01:50:15: Exporting VM config and log file...

12/03/2009 01:50:15: Config and log file exported successfully.

12/03/2009 01:50:15: stormweb log file = /vmfs/volumes/SAASB-Backup/visbu/stormweb.log

12/03/2009 01:50:16: *** Script completed successfully. ***

From the SystemsAD3 log:

12/03/2009 01:47:30: Log file opened.

12/03/2009 01:47:30: NOTE: This is a dry run, VM will not be snapshotted or exported!

12/03/2009 01:47:30: Lock file present for SystemsAD3, appears to already be in a state of backup! Skipping VM.

12/03/2009 01:47:30: Log file closed.

So in the middle of a scheduled backup VISBU decides this one machine is in 'Dry Run' mode and skips it. I then noticed that it has happened for other VMs. Anybody else seen this and/or have a solution?

TIA,

Dwight

[cas5213]

Is there any way to specify a password file in the Visbu configuration and load the password this way? If the file could be encrypted it seems like it would be much more secure. Additionally (primarily?), it would make management on systems like ours much easier where we use multiple hosts and frequently change passwords.

If there is some kind of workaround where the passwords for every config could be loaded from only 1 file to maintain, could anyone explain this? I've exhuasted myself of ideas trying to find a work around myself.

Thank you in advance for your time and consideration...Look forward to finding a way to make this script that much better!

Really hoping someone has something for this!

As always - great script!

[tlyczko]

There must be a way for the script to read the values from within

another file, after all it's perl!!

There's probably a function to decrypt the file before reading, then

encrypt again after reading out the data.

HTH Tom

[cas5213]

Tom,

Thanks for the reply...I'm almost positive you're right...but I haven't had the time to dig through the actual script files. (I was trying bash input redirection and scripting to just pass a value from a file quick and dirty...). I'll go through if no other ideas come up - but I'm quite hopeful someone did something like this before - or knows the code well enough to point out the relevant ideas to look at. Unfortunately, I cant focus all of my time on this among other things I must take care of.

I'll keep waiting on a reply and digging through slowly and surely in the meantime haha.

Any further insight always appreciated

Thanks!

[admin]

Hi, ignore the DRY RUN messages in the log - they are false and get entered regardless if it's actually a dry run or not - it's a bug.

www.phdvirtual.com

[admin]

Hi there,

The file that contains the password is only readable by the root user...so fairly secure (the same way vmware 'secure' their backuptools.conf file used for command line vcb tools). The trouble with using encryption is that it would need to be unencrypted again for use - and as this is an un-compiled program the encryption key would be in plain text so not really much more secure! You could compile the code with Perlcc or something I guess...

www.phdvirtual.com

[dwightmccann]

OK, but if I ignore the Dry Run message then I'm left with the Lock issue. I see no lock files and know of no reason they should exist. FYI: this (these) VM's were automatically moved from one machine to another by the VMware supervisor software. So, is there something I can look for to figure out why this occurs ... an apparently is "permanent" once it occurs? TIA, Dwight

[cas5213]

Mittel - thank you for the explanation, that actually makes sense - and did not occur to me that it was handled in that way for that reason. Nonetheless, it would be nice to have a "global" password file to reference per ESX server, and simply point each config to the file. At least then, when having multiple cron jobs calling different configurations wouldnt result in so much work when a server password is updated. Is there a way to modify the script to read the config file "password" field as a path pointing to a file with the password? Of course - its easy enough to lock down that file to root access only...

Hopefully you understand my question - It would do much to make management easier in this aspect. Any guidance appreciated!

Thank you for your reply, and once again - a wonderfully powerful script!

Regards,

-Cody

[admin]

Are no VM backups running? The lock (.lck) files are per VM in the VM home directories (with the .vmx), check for them and delete them. The global lock is in the vi-backup install directory under /usr/local.

www.phdvirtual.com

[admin]

You could put the vi-backup.cfg files on shared storage and then modify your cron jobs to point to that, then when you want to change just use a sed command to search and replace the passwords in all files in the cfg file directory at the same time. To read just the password field from there would require a code change but it wouldn't be too much work.

www.phdvirtual.com

[dwightmccann]

I included a log section in my post. There are several VMs on the host. The previous and subsequent VMs seem to backup with no issues in this particular case. I will again look for lock files as you suggest and report back. BTW, we are looking into esXpress but there is simply no money (well, less than no money at the moment here in California) in the very immediate future.

[tlyczko]

If you can get it somehow, esXpress is a good software in that it

doesn't have any dependencies on VCB or anything else, also if you have

Exchange or SQL or AD anything in a VM you should request their support

for advice on the best way to set up esXpress, there are some extra

steps involved.

However if you can wait for vSphere, it includes a backup solution

quite similar to esXpress's, but I have seen no information about what

it will cost, and VMware is unfortunately known for aiming at enterprise

companies with lots of $$$$$ unlike California

Finally you could also try this backup solution which is getting good

remarks and a lot of forum traffic:

http://communities.vmware.com/docs/DOC-8760

HTH Tom

[admin]

First off let me say that I now work for PHD Technologies so I obviously like esXPress. That said, I've had a good hands-on look at VMware vSphere's backup solution, and while it's nice to see VMware validating esXpress' approach to VM backup using the virtual machines to backup virtual machines, it's still a fair way behind on features from what I've seen and read. Not sure when vSphere is due to be released, but the file level backup/restore still wasn't part of it at VMworld Cannes so it'll be interesting to see how complete it is on vSphere GA.

Someone asked the same question on the esXpress support forum if you want to read more on the differences: http://support.esxpress.com/boards/read.php?4,12148,12148#msg-12148

Agree with you on the current esXpress setup but it's all going to be GUI driven very shorty, and have a VC plugin to integrate with the VI client, beta of it is available check out www.esxpress.com beta forum.

Alex

www.phdvirtual.com

[dwightmccann]

We're looking at esXPress. But in the meantime we seem to be getting more and more failures with VISBU (not blaming VISBU ... it's likely something we're doing) and wonder if you have any insight. Here's some log entries from last night:

16/03/2009 22:36:02: stormdbms log file = /vmfs/volumes/SAASB-Backup/visbu/stormdbms.log

16/03/2009 22:36:02: Snapshotting VM stormdbms....

16/03/2009 22:36:03: ERROR: Could not snapshot VM stormdbms. Cannot perform backup, moving to next VM.

And then from the stormdbms.log:

16/03/2009 22:36:03: 2009-03-16 22:36:02.561 'App' 3076453184 info Current working directory: /root

2009-03-16 22:36:02.641 'CreateSnapshot' 3076453184 info Creating snapshot

2009-03-16 22:36:02.764 'vcbSnapshot' 3076453184 error Error: The

managed object of type "vim.VirtualMachine" identified by the MoRef

"3568" could not be found.

17/03/2009 01:30:17: Log file opened.

17/03/2009 01:30:17: NOTE: This is a dry run, VM will not be snapshotted or exported!

17/03/2009 01:30:17: Lock file present for stormdbms, appears to already be in a state of backup! Skipping VM.

17/03/2009 01:30:17: Log file closed

I'm wondering what would have caused the original error and then wondering what would cause the second log entry at 1:30. The two entries could have been created from two different ESX hosts with the same registered VM? I'm the backup guy, not the ESX guy, so I'm not intimately familiar with the structure.

[tlyczko]

Contact esXpress support, they can tell you what the errors mean.

I have had good experience with their support, they are still a small

enough company that they can respond pretty quickly.

There might be a setting of some kind that prevents snapshotting etc.

Good luck, wish I were in that Santa Barbara sunshine now...

HTH Tom

[admin]

Looks like a lock file has been left in the VMs .vmx directory. If for some reason the script dies before a backup completes, or perhaps if the VM VMotions to another host during a backup and that host then tries to back it up at the same time (which is what the lock file is supposed to prevent) it's geting left. Check the VMs home directory for .lck files and delete them and you should then be able to back up the VMs again.

Alex

www.phdvirtual.com

[petedr]

Thanks Tom for the positive words regarding the esXpress support team.

Although looks like the issues posted were in regard to VISBU and not esXpress.

www.phdvirtual.com, makers of esXpress

[tlyczko]

Mittell said to look for lock files, which makes sense, one can't snap a

locked VM, correct??

When is 3.5 coming out??