Anyone else having these VDR issues?

[jarsenea]

I love the fact that VMware is providing VDR as part of the vSphere package. It's definitely a step in the right direction, albeit I'm still inclined to think this software hasn't been put through the ringer in terms of proper QA. I'm just trying to put out a feeler to see how many others have experienced some of the same issues I'm having.

To start, I'm backing up my VMs via a network share on a standalone Windows 2003 server that has a NAS attached to it.

Some of the issues I've noticed:

1) Backups take an inordinate amount of time. I can understand the first backup, but my VMs don't change very much from day to day. Most of the data being manipulated is located on RDMs are these are backed up using Tivoli, not VDR (I use VDR solely for the OS partitions). Each partition is approximately 25GB, there are 15 VMs and my backup window (10pm - 6pm) isn't sufficient to complete the process.

2) Integrity checks for the backups are taking a crazy amount of time and will usually stop due to my window being closed (see point #1)

3) I'm getting inconsistent "failures" for certain VMs (the report will simply state that a VM failed to backup, not much else). It also varies per night and not always the same VMs (not exactly sure if this is related to #1 where the window is closing while VDR is executing)

4) I had the most difficult time setting up the remote share from the VDR appliance in vSphere. The username and password would never be accepted (even though if I tried the same share with the same user/pass on a Windows machine, it would work fine). I finally narrowed down the problem to the simple fact that the VDR appliance can't handle passwords that have special characters in them (this password had an "@" and a ","). Looking at the console while attempting to mount the share would spit out a CIFS error -22. Changing the password to include only numbers and letters was sufficient to work around this issue.

5) Snapshots not being created for no apparent reason and thus failing the VDR process. I'm fully able to do a manual snapshot with or without the memory state, so I'm not sure why VDR can't do it. This issue is very intermittent. I had it often when I first setup VDR, but now it only happens every so often (without any type of consistency).

I think that's all I can think about for now..

[Steve5]

I was happy with VDR 1.1 but after coming to have a look at this week I've deleted it again after finding that it simply wasn't working.

The log files weren't tracking faults, I had a vm which was tagged for backup, was listed as compliant, but didn't appear in the restore list. I'm getting endless quiseing errors with certain server which I appreciate aren't necessarily VDRs fault but the system does nothing to help you with the problem.

The lack of reporting is an endless frustration. To find the volume locked preventing all backups is one thing but to not be informed of a fairly critical issue is quite unacceptable. I'll deploy it once more and if it doesn't work I'll have to take my small deployment of < 25 vm's to a rather expensive solution to ensure the reliability a release product should have at launch.

I understand the supportive mentality of some of the posters accepting the teething problems but this isn't free software.

[KBuchanan]

Let me tell you history story:

Do you remember when Johnson & Johnson recalled the Tylenol product in the early 1980s? They recalled ALL of their product from the shelves because it was linked to the death of several people. Their intent was to prevent further harm to people - and to save the product.

By withdrawing all Tylenol, even though there was little chance of discovering more cyanide laced tablets; Johnson & Johnson showed that they were not willing to take a risk with the public's safety, even if it cost the company millions of dollars.

Johnson & Johnson also used the media, both PR and paid advertising to communicate their strategy during the crisis. Johnson & Johnson used the media to issue a national alert to tell the public not to use the Tylenol product. In the first week of the crisis Johnson & Johnson established a 1-800 hot line for consumers to call. The company used the 1-800 number to respond to inquiries from customers concerning safety of Tylenol. They also establish a toll-free line for news organizations to call and receive pre-taped daily messages with updated statements about the crisis.

Before the crisis Johnson & Johnson had not actively sought press coverage, but as a company in crisis they recognized the benefits of open communications in clearly disseminating warnings to the public as well as the company's stand.

Johnson & Johnson communicated their new triple safety seal packaging- a glued box, a plastic sear over the neck of the bottle, and a foil seal over the mouth of the bottle, with a press conference at the manufacturer's headquarters. Tylenol became the first product in the industry to use the new tamper resistant packaging just 6 months after the crisis occurred. (Ref: http://www.ou.edu/deptcomm/dodjcc/groups/02C2/Johnson%20&%20Johnson.htm)

-

---

>The end result...well, Tylenol is still the one of best selling pain relievers in the market because they were able to regain public trust. In cost them initially - but in the end, it was worth it because they demonstrated they had the courage to "protect their customers". Sure - it was also saved their product and future profits, but it incredible smart because it cemented their commitment in the minds of their customers. THEY DID WHAT WAS RIGHT.

[KBuchanan]

Jjewett:

Yeah - maybe my comparison is a bit extreme, but there is a commonality: Take responsibility for your product. But somewhere "out there"...someone IS relying on VDR - and it is going to let them down.

Kevin

[Stevester]

VMware may simply need some help. Its clear that the backup dedupe arena is new to them and they simply lack experience. They may want to partner up with Veeam, PHD, or some entity to get trained engineers or even resell some of the backup dedupe products out there but that's highly unlikely. WOW, i stated in a seperate thread that time will actually show the truth and it hasn't failed yet. Since the birth of the Universe, time is the only constant.....to sort of speak.

Stevester

[KBuchanan]

You're probably right. ...good point.

The do such an EXCELLENT job with VM technology, and they have the right concept for the VDR product. They just need to get it in a working/reliable product.

Kevin

[jjewett]

Yeah, VMware needs to partner with someone with storage experience. 😛 Did you forget who owns VMware? Actually, what help EMC has been to VMware is questionable. But they did acquire Avamar's dedup technology. I wonder if any of the Avamar tech found it's way into the vDR appliance.

[blautens]

I, too, was happy with VDR 1.1. But it suffers from issues (the same ones others have mentioned - creating snapshots it doesn't delete, failing to take snapshots, locking a volume, and sometimes just plain stopping whatever it was doing for no particularly logged reasons and I cannot end the running task).

Error reporting/logging doesn't appear sufficient (I'm sure there are deeper layers for VMWare staff that aren't readily visible...ummm...right?). Notifications are zip, but I'd be willing to live with that if it didn't just hose up randomly every week or so.

Rebooting the VDR appliance is the usually the only option when it throws some sort of error. I have best practices (correct volume size, etc.) implemented. And no, I don't have connectivity issues. I've installed it a few times now (I'm an old pro at recataloging).

Once I decided to vMotion VDR to another host, and (perhaps unrelated to VDR) I had an issue that caused the ESXi 4.0 host to become unresponsive (no change to VM status, though). Restarting the agents didn't help, etc.

Anyway, VMWare support (very quick and courteous) managed to "command line" me out of the problem, and then when I mentioned how it all started, she wasn't particularly enthusiastic about VDR and just sort of threw the comment "we don't support vMotion of VDR appliances" out there - I dropped the issue, because I was just happy to be back in control of the host.

The features VDR v1.1 lacks (extensive logging, notification, a little picky with volumes) don't bother me so much. It's just the randomly unreliable part that gets me.

[habibalby]

I'm happy to run vDR version 1.0.2.265 with local VMDK as dedup store. But, this all comes after many strugle with the pervious versions of vDR and the bad luck is i was relying on Network Share as dedup store on Windows Server and i backing up the share on tape. after a major failure in my OpenFiler iSCSI, I restored from the tape and when i try to run check integrity on the Network Share, it cannot continue.

My good luck, when i first test it, i tested in to backup on Local VMDK and at that day when i saw it runs fine, i ran a backup on all of the VMs. Then, I was wondering how to export the Dedup VMDK to tape.. hummmmm. an idea came in my mind to backup the Dedup VMDK with vRanger Pro "trail version" and then i export it to tape.

I restored from vRanger to a LUN and then I attached the VMDK to the 1.0.2 vDR and happily, when i run the Check Integrity on the restored Dedup, the check runs fast and successful. So, Restoration of lost VMs again back up and running in total 3 hours.

conclusion, vDR is really good product but, i found it only on VMDK as destination.

Best Regards,

Hussain Al Sayed

If you find this information useful, please award points for "correct" or "helpful".

[schlenz]

Hi,

"conclusion, vDR is really good product but, i found it only on VMDK as destination."

I can confirm that - I had no good expirience when using SMB as a destination. I use VMDKs on a remote NFS server as destination which works quite well.

Bye

[habibalby]

Also, another method of vDR Backup to Tape with StarWind iSCSI.

I have created an iSCSI LUN on StarWind, this lun is presented to the ESX server and a VMDK disk assigned to the vDR for Deduplication.

Great thing in StarWind is the iSCSI Disk is created as .img file format:) So, after backing up all my VMs to this VMDK Store;

Shutdown the vDR

Remove the VMDK disk from the vDR

Unmount the iSCSI Lun from ESX hosts, so the .img file doesn't being used or locked.

Backup the .img file which is contains the VMDK Deduplication Store

So, restoration of VMs that have been backed up with vDR from off-site storage "tape" is possible with StarWind

Restore the .img file StarWind Server

Add new Image File Device, and select Mount Existing Virtual Disk

Browse to the .img file and mount it.

Rescan the ESX vmhba32 adapter, you will see that the old iSCSI LUN is appeared once again.

Mount existing VMFS store without changing the signature, and the same orginal VMFS will be added.

Add the VMDK which is contains the Deduplication Store to the vDR and select existing VMDK.

Once the VMDK is re-added, mount it to the vDR appliance via the Vi-Client and run check integrity of the Deduplication Store.

Backup and Restoration works like a charm, and i found that VMDK reliability is far better than Network Share.

Best Regards,

Hussain Al Sayed

If you find this information useful, please award points for "correct" or "helpful".

[KBuchanan]

Wow...and to think, we could do that every day. I have a better idea: VMWARE COULD BUILD THIS INTO THE "BACKUP" PRODUCT!

I don't like to sound sarcastic - but VMware hasn't listened to customers, or at rather, they should have asked before they tried to develop a backup solution. I have seen enough customers comment on the features that are glaringly absent.

A backup solution with de-dupe isn't necessarily unique, but it is a GREAT idea. I am anxious to see it work reliably in my environment.

BTW...I am not using some free, 2-bit, unsupported destination target for VDR. I am using 15k tier-1 fiber channel disks on my EMC SAN array.

Kevin

[habibalby]

No need to use a free / unsupported product in your enviornment, but as long as the solution work and somewhat reliable than relaying on Network Share for off-site storage which I don't even have a signle success into restoration from Network Share that been backed up into tape. I think this idear is a decent way for off-site storage, and the great thing that vDR is backing up to VMDK which is much much better than Network Share or CIFS.

I don't agree with you that VMware doesn't listen to thier customers, because when i attended my two courses DSA and Install & Configure I personally suggested the backup to be integrated within the vCenter / Vi-Client. I DON'T KNOW, this idea comes from me or from others or may be VMWARE themselves thought about it before I suggest it.

Best Regards,

Hussain Al Sayed

If you find this information useful, please award points for "correct" or "helpful".

[KBuchanan]

Maybe they listened; but they should have asked the community-at-large - they didn't. But - that is a moot point, isn't it?

Kevin

[habibalby]

Agree, but do you think for every product they release they have to ask the community to give thier opnion about it?

Best Regards,

Hussain Al Sayed

If you find this information useful, please award points for "correct" or "helpful".

[KBuchanan]

Good counter point. It is too easy to "Monday-morning-quarterback" and be critical of their decisions.

I think there are many that are very disappointed and have much higher expectation that their VDR product would a well-thought product. It lacks too many features that we (or I) would take for granted - such as alert/notification, management of the backup images, and a product that works reliably.

Before someone beats me up...it may be working reliably for a few...but...it I believe it is only a few, indeed. There are TOO many others in the community forum that have experienced some unfortunate symptoms. My prior posts expand it this well enough.

But, a good lesson they should learn - is to do more/better homework on what their customers need. I would hope they aren't developing a product in complete vacuum. The ESX product is absolutely GREAT and I strongly advocate for it. Maybe that is really frustrating issue for me - I expected greater than what they delivered (with VDR).

I am confident they will fix the problems - but it would have been better to post the product as a "beta" product.

Kevin

[habibalby]

I am confident they will fix the problems - but it would have been better to post the product as a "beta" product.

Kevin

</div>

Absolutely right :smileylaugh:

Best Regards,

Hussain Al Sayed

If you find this information useful, please award points for "correct" or "helpful".

[chriswaltham]

I'm still having all sorts of problems with VDR. Here are my most common:

1. Backup jobs simply don't run. Even though they're scheduled, they don't run at all. So, no snapshots are created.

2. Can't log in to the VDR appliance with the VDR plug-in. I can usually fix this by connecting to the appliance via SSH and then running "/etc/init.d/datarecovery restart"

3. When using the VDR plug-in, the appliance claims there are no restore points. Even though they're clearly sitting on the destination!

Has anyone else run into #1 or #3? If I reboot the appliance, it sometimes fixes #1 (that is, the schedules are executed the next day.)

[admin]

Are you running VDR 1.1 (both the appliance and the plug-in)?

[chriswaltham]

Yes, I am.

[KBuchanan]

Azmir:

I'm glad to see you following this thread - I thought it had been abandoned! I'm sure you are frustrated by the messages. It would be very encouraging to hear some news about what we can anticipate.

You couldn't tell from some of my posts, but I am strong advocate for VMware. I just wish I could sing and dance about VDR, also. VDR 1.1 would lock up, or rather seem to stop processing during the backup. After it happened a few times, I lost faith/confidence and stopped using it. I can't waste time and I don’t have time to open a SR for a product that apparently isn't as ready as it should be.

To be very honest - my backup scripts (all of which is using supported commands) runs perfectly and provides a daily e-mail notification. It doesn't require any babysitting.

I am looking forward to VDR doing the same...honestly.

Kevin