Re: VDR 1.2 Reliability - Page 2

dustinn3 · ‎10-04-2010

Just curious if VDR 1.2 is improved over previous versions. I tried using all the previous versions and couldn't get any of them to work consistently. They would backup for a few days, then suddenly they would start failing and eventually all my backups would get corrupted. Consistency checks would take days. Supports only resolution was to reboot the VDR appliance and delete the restore points. I've been burned so far, so I'm hesitant to try it again until it's a bit more mature.

craiggunson · ‎10-11-2010

Hi John,

Move your VDR datastores to NFS based vmdk or FC based RDM. I have both now working well. CIFS was causing me all sorts of pain which seem to have gone away since.

I generally agree this product doesn't seem that mature yet. (For us it's better than the vcb framework we had plugged into Symantec tho). Be interested to hear what your solution provider settles on.

Cheers,

Craig.

John4567 · ‎10-11-2010

Hi,

Thanks, but we're already using NFS for the target location for VMDR to use as the backup respository.

I'm about to trash VMDR and go to an all 'ShadowProtect' backup solution pending a final discussion with the solutions provider, VMDR was a bit part of the 'pitch' when we decided to finally give VmWare a go and so far it's been nothing but grief. Meanwhile shadowprotect inside a VM just keeps belting along without even so much as a pause.

Here's what I'm thinking about:

Shadowprotect server running on all Server VM's

Shadowprotect ImageManager consolidating the incrementals and then replicating to a NAS appliance each day

Symantec to manage the 'Get it onto tape' aspect

How does that sound?

Our solutions provider has logged a call with VWARE, but if they come back and tell me to delete ShadowProtect and trust VMDR, no way...I know which product i'll be backing...

-John

craiggunson · ‎10-11-2010

Can't say for Shadow Protect (no experiece with it), however doesn't appear to support Linux?

(Might be ok if your currently 100% Windows, but if you start virtualizing linux you'll want cross-platform solution).

jasoncllsystems · ‎10-11-2010

I'm not familiar with ShadowProtect and you may try Acronis Backup & Recovery which provide similar function + centralize manage.

Regards,

jlchannel

MALAYSIA VMware Communities

http://www.malaysiavm.com

'If you found this or any other answer useful please consider allocating points for helpful or correct answers ***

http://www.malaysiavm.com

aarondovetail · ‎10-12-2010

I'm not very impressed with Veeam so far. I like the speed and the way it lays out files, but the restore function doesn't seem to actually work?

I have a few brand new Win 2008 R2 VMs that i've created, I ran a backup on them all and then did a restore as a full VM to a new VM as a copy. None of the Win 2008 R2 VMs will boot up, they all blue screen.

The one Win 2003 VM I restored did boot up and work properly but 0 of the 4 Win 2008 VMs. This product isn't very useful if I can't restore properly, Veeam hasn't come back with a solution as of yet.

Jelloir · ‎10-12-2010

We run VDR using a datastore in on a Debian NFS server so we can backup the VDR appliance offsite, and it has actually been stable from the start. But I have to agree that it's as if VMware have released an alpha/beta piece of software - I had to write my own email reporting script... seriously what the?

A big warning for anyone considering it for file level recovery.... File level recovery does not work with GPT disks. You simply cannot mount them. VMware support advised me of this.

John4567 · ‎10-12-2010

Hi All,

Well, so much for my ShadowProtect Idea...

The current (4.3) cersion of the Standalone converter tool doesn't officially support ShadowProtect 4.01.. they support 3.2...which is nearly two years old.

So, I can take a full backup of a server and the converter tool can read that.. but if I create a full ShadowProtect backup from a base+incrementals via teh ShadowProtect Image Conversion tool, the Standalone Vmware converter tool tells me it can't read the resulting full backup image, and it doesn't seem to properly support Shadowprotect incrementals anyway, it's almost like it fails to process more than about a dozen incrementals if you point it at a directory containing consolidated ShadowProtect incremetnals...

My SAN doesn't have the space to take multiple full backup images of all my VM's, the only way I could do this would be to have 1 backup of the server taken each day, stored onto the SAN and then that being written to tape, maybe get ImageManager to replicate all that off to a NAS each day, but then that would run of space as i'm dealing with full server images each day (hundreds of GB's of data moving around each day on the network).

So, it looks like we're in a bad place...I run continuous Incrementals in ShadowProtect with ImageManager taking care of all that, but it looks like I can't use this as a means to backup my virtual machines and recover them if I loose my SAN or have to recover a VM for some other reason.

My absolute last resort is to have ShadowProtect convert a consolidated backup image into a VMDK and then somehow load this into a VM on VMWARE...I'm still working out how to do this.

How the #@!$ are we supposed to back up our VM's !?!?? I'm told that Veeam is having problems as well... has anyone here had to actually recovery VM's or restore from a dead SAN?

I'm seriously pissed #!$#@!ing off right now, this is doing my head in... advice please?

Do I need to just suck it up and go pay Symantec a shit load of cash and be done with it?

-John

craiggunson · ‎10-12-2010

The plug-in for Symantec still relies on a whole bunch of disk to spool the images to tape, depending on the suite you have in mind. BackupExec just uses the vcb framework in the back-end which could be scripted in about 30 mins if you don't want to license that feature. We abandoned Symantec after a few months, due to "other" scaling issues and scripted it ourselves which worked very well for about a year). We're only moving on now coz we want to leverage VDR.

(VCB is our fallback plan though, and would force us back to weekly backups at best, as you say daily is bit much on VCB)

It's far from elegant, but it's free and I know restores work. On the more commercial side I'm told Vizioncore are quite good (recently aquired by Quest).

Back to VDR I just completed an integrity check on a corrupted VDR datastore (formerly on CIFS) now on FC (speedy) RDM and it is fully back online, with fully functional restore. It took 17 hours to cover 350GB (1.3TB Non-Deduped), but it worked... no file delete hacks, just let it go. This was done with a new VDR appliance pointed at copy of old dodgy datastore.

It's important to note the appliance doesn't really go flat out (and push the disk - at all) it just sort of crawls along. It seems tied up mostly in iowait... unsure why at this time.

coldtech · ‎10-13-2010

I have yet to attempt a restore from a Veeam backup. I'll have to try that today.

Supposedly PHD will be releasing their solution sometime this month into beta, will try that too.

Does anyone actually have a solid solution for backups?

When I did this new setup I thought I'd be fairly well covered with Dell's snapshot feature of the MD3000, VDR, a DPM Server and an additional

backup server pulling files via a robocopy script (yes I am paranoid).

Dell snapshots = FAIL. This software is crap, you can't tell what it's doing or not doing.

VDR = FAIL.

DPM Server=FAIL. Some machines get protected others not, at least my Exchange stores seem to be OK.

I've been doing this since the days of Netware ELS II and over the years my experience with backup recovery for both customers when I was a field tech and personally is dismal. I can only recall ONE instance where a tape actually restored anything. Restoring to different hardware has always been a nightmare. I was hopeful I had finally reached the goal of having an infrastructure that was truly capable of being archived.

WTF does it take these days?

John4567 · ‎10-13-2010

Hi All,

OK we got a solution here! Not for VDR1.2 but for backups in general..

We got ShadowProtect going and can restore to VMware wihtout any drama... in the interests of helping people here's out here's how it works...

To get ShadowProtect running.

1. Install ShadowProtect onto all the VM's we want to backup

2. Select the destination target

3. Set for 'Continuous Incremental' and fiddle with number of incrementals to take per day etc, we use VSS on all backups taken

4. That's pretty much it for backups.

To Prepare Restores.

1. Use an ISO converter tool to take the current ShadowProtect CD/DVD to an ISO or download the ISO from their website

2. Upload the ISO into a VMWARE DataStore (we have one specifically for ISO images)

To Do a Restore.

1. Create your Virtual Machine that you want to restore onto

2. Edit Settings of the VM, set the CD/DVD drive to the ShadowProtect ISO image

3. Set Network to 'Connected'

4. Set 'Boot Options' to force to setup screen/BIOS

5. If you're in vCentre, select the VM and go to the CONSOLE tab

6. Power up the VM

7. In the BIOS type screen, select the CD/DVD as first boot priority

8. Press 'F10' to save these settings and reboot

9. Follow ShadowProtect instructions which are clear and easy to read

9.1 If you've got DHCP and DNS, you can set up your network,otherwise assign an IP/Gateway/Subnet manually

9.1 Map a drive to your backup location where the images are stored

9.2 Select the drive partition on the VM you want to restore onto (they may require a reboot of the VM)

9.3 Follow the prompts, go make a cup of coffee/tea

Done, we've backed up, deleted and restored several VM's wihtout any hassles.

Even better,this solution doesn't require any fancy footwork with VMWARE and VSS and Snapshotting, it's all done within the VM, you can recover your ShadowProtect backups onto Bare Metal also.

ImageManager Enterprise (from same company) can even tape your backup incrementals, consolidate them and replicate them off-site!

So we're good to go, all backups are VSS capable for crash consistency and application consistency and damn, it just works.

Couldn't be bothered with VMWAre's conversion tools, this is the way to go as far as i'm concerend

-John

rickardnobel · ‎10-18-2010

I assume that in such a thread most people that responds has bad experiences of the product, but it is really this unstable?

It just seems so strange for a company like Vmware to sell such a important product (backup) with this amount of problems. Is there anything said official about this? Is it likely to improve?

My VMware blog: www.rickardnobel.se

John4567 · ‎10-18-2010

It had better improve, I was really dissapointed.

We went with a completely different solution to our VM backups that seems to have worked well so far, it doesn't have the convenience of being able to click an instance-in-time image of a complete server and say 'restore this', but it gets the job done.

We found that:

If VCenter is running on a Virtual Machine under Windows Server 2008 R2, you can't VDR it, at all.. forget about it, I think this is an acknowledged bug, workaround is to run windows backup or some other 'inside the VM' backup solution, which got us onto using ShadowProtect (heck, if it worked for the VCenter VM, why not all the others?)

If you're running Windows Server 2008 R2 Vm's, you get seemingly random Snapshot failures by VDR.

VDR consumes innordinate amounts of resources and is just plain slow.

Corruption of backup repository warning and messages.

Getting spammed with truck loads of e-mails about VDR resource usage from VCenter is just plain annoying, you also get warnings about hosts being offline or lost connectivity to hosts which is really alarming, one morning I had nearly 30+ e-mails all with 'condition red' type alerts, I spent two hours trying to see if we had some kind of critical failure in our switching, only to find it was this VDR thing screwing around

Deleted VDR appliance, no more issues/warnings about resource utilisation, life is better

I will however look at VDR 1.3 in a test environment, but it will be a loooooong time before I even think of putting this into production again.

-John

craiggunson · ‎10-18-2010

To answer your first question (in my experience) ...

VDR on CIFS datastore (unstable)

VDR on NFS datastore (stable)

VDR on RDM datastore (stable)

VDR integrity check on good datastore (stable)

VDR integrity check on bad datastore (stable - but slow)

...and most importantly VDR restore, both the appliance, and vm guests (stable)

On your second question, I personally hope so coz it solves a bunch of issues for us.

SylvainLanglade · ‎11-08-2010

Going back to this thread, I must add that after upgrading to VDR 1.2 and running the thing during one full month, I didn't notice problems anymore. Maybe I'll regret what I just said tomorrow morning ! With 1.1, there was not a single week without losing part of the backup data or seeing integrity check hanging badly.

But the truth is that in our case, using CIFS share, we don't have problem since 1.2. Except for once, after abruptly killing the VDR appliance during an integrity check. The subsequent backup was awfully slow for some VMs. It lasted over a full day to backup those, manually monitoring and driving the process. Then all went back to normal, and staid normal since.

So, maybe is some stability included in the 1.2 release !

aarondovetail · ‎11-08-2010

I don't suppose anyone has a line on when the new version of VDR will be released? I don't really want to drop 1400$ on the Veeam essentials and then 3 days later VDR 1.3 or whatever comes out and finally works

iainh667 · ‎11-15-2010

Another vote for "No"!

VMware had a number of improvments moving from v1.1 to v1.2. It was great for me, I managed to get 100% backups for all the VMs in the environment.

Then for some reason the appliances crashed, nobody from the backup team was monitoring the nightly backups and I only heard about it 4 weeks later.

By that stage it was taking too long to run the integrity checks and I needed to start from scratch again.

I've needed to rebuild the vDR appliances 3 or 4 times now. Okay, I'm prepared to admit there's a few failings onsite, but these appliances should be resistant to crashes.

nathanw · ‎11-17-2010

Hi John,

I just have a few things I would love to clarify, what storage are you running? what disk underlies this? SS, FC or SATA

How is the disk connected to the ESX servers?

vCenter is not supported on 2008 R2 although some have it working this is not actually a supported configuration. If you have an out sourcer supplying you a solution that is not to specifications of VMware supported configurations I would ask why?

Nathan VCP

rickardnobel · ‎11-18-2010

vCenter is not supported on 2008 R2 although some have it working this is not actually a supported configuration.

It is actually supported on vCenter 4.1.

My VMware blog: www.rickardnobel.se

nathanw · ‎11-18-2010

Sitting corrected I am.

I read an article from vmware this week stating that there was only limited support for it on 2008R2 and I was sure they were talking 4.1. my bad.

Nathan VCP

GenPT · ‎04-15-2011

I'm running 3 vDR instances to 3 different CIFS shares. I logged in today to find that the Integrity Checks were failing and that all restore points are "damaged". We'll considering implementing a daily snap on the SAN side to protect against this, but the lack of a repair function shows that this is not a "fully baked" backup solution.