VMware Cloud Community
SCX
Enthusiast
Enthusiast

vDP - Sick of being a beta tester

I am really tired of VMWare releasing half-finished software and expecting end users to be beta testers. First it was SSO (what a disaster that was/is), and now vDP - what is going on with this appliance? It's a complete resource hog that crashes constantly. Why does it need 4 CPUs? What on earth can it be doing that needs so much? Why does it take 10 minutes to boot the appliance? Why is installation such a mess? How come when I try to start backup jobs, half the time, nothing actually happens (apart from a snapshot being taken)? How come you are forcing me to use the webclient to access it?

The whole 5.1 release reeks of being rushed and I'm thoroughly unimpressed.

25 Replies
GSparks
Enthusiast
Enthusiast

I am sorry to hear that you are frustrated with your experience with VDP so far.

We would like to work with you through your issues so that you can see the value that VDP can add as your backup solution.

In regards to your questions, here is some information to provide some insight into how the appliance works.

Why does VDP require four CPUs?   

Several of the activities performed by the software are CPU intensive.  Examples are garbage collection, the HFS check and backup and restore hash calculations.  Without adequate CPU resources, these activities would cause performance on the system to degrade below what we deem acceptable.

Why does it take 10 minutes to boot the appliance? 

During the initial VDP appliance setup, we are performing many of the following steps –starting up core, file system and management services; network reconfiguration; keystore and SSL setup; registering with the vCenter and proxy clients; synchronizing of passwords across all subsystems; and performing an integrity check, which creates a checkpoint and validates the checkpoint.

Why is installation such a mess?

Can you please provide more information as to what was problematic with the installation?  Your feedback is valuable to us.

How come when I try to start backup jobs, half the time, nothing actually happens?

This sounds like a configuration problem.  Would you be interested in opening an SR with support?  I’m certain we can solve this issue for you.

How come you are forcing me to use the web client to access it?

There are a variety of reasons why we are moving to web based management UI.  In our user testing, we had customers who initially were hesitant to move to the web-based client, but after using it for a bit, they felt it really makes sense.  Plus, in multi OS/browser environments, the Flex UI is a real treat.  Any particular feedback you have is welcome here as well.

Thank you,

-Greg Sparks
Reply
0 Kudos
mauricev
Enthusiast
Enthusiast

I would like to give you some feedback on how 5.1 and VDP is a major downgrade in terms of user experience.

First off, the program and the data are one piece. I get fixed sizes, which was not the case in VDR. Worse, do you not expect anyone to implement this on simple mirrored disk arrays? That is, the 1 TB ova requires a 2 TB disk and the 2 TB one requires a 4 TB one because it requires slightly more than the size of a 3 TB disk's capacity. Of course, if I wanted anything larger, I have to have multiple instances of VDP. What was simple has now become inordinately complex.

Second, for my simple installation, I have a private network and I just used a hosts file and it worked. Now VDP is hardcoded to ignore it (to ignore nsswitch.conf in the VDP appliance) and to use DNS exclusively, so I had to set that up from the ground up. Doesn't that violate POSIX or something to ignore nsswitch.conf?

Third, when the installation prompted me for my vCenter username and password, it rejects me. I recall that before (after doing a simple install all of vSphere 5.1), I had to add explicitly add the Administrator (the vCenter Administrator user) as a user to SSO before the web client would see my vCenter. But then I had to at least prepend the domain, that is, system-domain\, to the username before it would be accepted by the VDP installer. I believe I had to add a separate user other than the Administrator for that, but I don't even recall. This should have all be done by the installer automatically.

Fourth, what's up with the crazy password policy for VDP, which is on top of a different crazy password policy for the SSO?

I just played a little with the web interface...

Fifth, I find it more confusing than in VDR. In VDR, there were colors and icons and I could see clearly in events what had taken place. Now the listing in reports is totally generic with all the text the same size and colors. For example, the state of the machine is right there alongside with the backup status. Who cares about the former? I really just want to know which machines failed. They should be in their own list at top along with a detailed reason field to explain why. (I have found that I can manipulate the columns, but it's limited and why not just set them up optimally to begin with?)

Sixth, I may be misunderstanding some things here, but I don't see a way in the backup configuration to select individual VMDKs. When I click a triangle for a particular VM, it shows no further details.

Seventh, in VDR, if I ever had to restore the appliance itself, I could so easily and just point it back to the storage. It's not clear how to do with VDP, which installs only with its storage.

Lastly, I can't speak much for what's under the hood in VDP. VDR would often fail with cryptic errors and one desirable feature was some level of file restore. I think it would have been better to concentrate the effort on just those things and keeping the interface inisde vCenter mostly intact.

Reply
0 Kudos
SCX
Enthusiast
Enthusiast

I agree about what (to me) seems like overly obstructive and confusing security configurations in SSO. Then again, we are a SMB and need nothing more complex than a single "administrator" user that should be given permissions to run anything. Having to explicitly set permissions for SRM, vCenter vDP etc by using the SSO admin account is cumbersome at best.

And yes, what is with the ridiculous password policy in vDP? Exactly 9 characters, no special characters, one uppercase character etc? Fail.

There is no way to backup individual VMDKs - it's by virtual machine.

I don't care how much you try to convince me how much work the machine is doing under the hood, 10+ minutes for a reboot is unacceptable.

Regarding the web interface, it is just "ok". What about showing me the space taken up per VM? I can't seem to find that anywhere.

The "Filter" functionality on the "Reports" tab is unfriendly and buggy.

Anyway, the best thing I can say about it is that it is "free". Hopefully, future updates will make it a bit more user-friendly.

Reply
0 Kudos
Litejk
Contributor
Contributor

I am also evaluation vDP and I have used vDR before.

1. Installation is painful, and not very bullet-proof. In my case, I could not register the appliance with the hostname of the vCenter sever (even if all the records where ok and verified), I had to use the IP address in stead. Same goes for SSO servers hostname, it would only accept IP address.

Second one, in my test, I had to configure vDP 2 times (with 2 reboots), before finally I could log in to the post-setup status and settings pages.

2.  Way to high CPU usage. On the testserver it runs on now, it constantly uses between 4,500 MHz and 6000 MHz on a testbox with single CPU with quad core running at 2,5 GHz, which means that half of the CPU's in average are running at max peak constantly. Very strange patterns in performance tab also. Note that the vDP appliance only have done 1 single backup of a small Linux VM at around 6 GB (using a 1 TB datastore) . I cannot think of any reason for why the appliance needs to do a heck of maintance and garbage recycling constantly for so small amounts of data.

[url=http://bildr.no/view/1302674][img]http://bildr.no/thumb/1302674.jpeg[/img][/url]

[url=http://bildr.no/view/1302675][img]http://bildr.no/thumb/1302675.jpeg[/img][/url]

[url=http://bildr.no/view/1302676][img]http://bildr.no/thumb/1302676.jpeg[/img][/url]

[url=http://bildr.no/view/1302677][img]http://bildr.no/thumb/1302677.jpeg[/img][/url]

I see that there is a lot of java processes going crazy sometimes, with peaks up to 94% of total CPU's on the appliance.

3. The Web Client is way to slow, but that goes for all administration of vSphere when using the Web Client. The old fat client is much more snappier.

Why in the earth rely on flash for creating an admin interface? In my mind, even standard simple HTML (not HTML5) would be enough to create the UI.

o

4. File level restore requires you to do this ONLY from a server that has only been backed up with vDP. That means, if I need to restore files from 2 servers (ie REDHAT1 or Win2K8R2), I have to login to either those 2 servers or other servers that has alreade been backed up. I cannot do this from a client computer (for example my own laptop). That means to use the VM Console, RDP or VNC and then start it. Why in the earth is that a show stopper? The old vDR did not have that requirement and to me it seems that someone who designed this solution at VMWare seriously need to basic system architecture again. Since vDP requires the client to initiate an advanced login to be a VM it self (pushing application logic to the client in stead of keeping it middleware/server layer), I think it makes it very cumbersome to do FLR. After all, who is running GUI on a Linux server anyway nowadays? And on Win servers, I never install any Flash Player and I'm not happy to install the Client Integration extensions either on production servers.
In my mind, the vDP appliance should not care from which client you are coming from, as long as the client have access to vCenter also.

Edit: 5. The Web Client is buggy when it comes to international keyboards. For example the sign '@', which in Norwegian (and other Scandinavian keyboards) to press Alt GR + 2, don't work, it just forces the client to go back on step. Only solution to overcome the limit is to write any text in notepad and copy it (for example when filling out email addresses on Reports tab)

Reply
0 Kudos
cojo71
Contributor
Contributor

1.  I really don't understand why it required almost 1TB for 500GB backup drive.  Putty into VDP, I see you allocated 77GB for /space(include avamar and /home) and it is using 1.4GB.  Most of the remaining space are allocated to /data01, /data02, /data03 and I see backup are stored in these folders.

2.  Flash player is one of the buggies software with endless updates on earth and I have to install it on the server before I can use FLR.

3.  It took 18m21s to restored a single 124MB file using FLR?  CPU utilization hit 100% during the restore process.  I can't imaging what it will do to my server if I have to restore a bunch of files or a very large file.

Reply
0 Kudos
RParker
Immortal
Immortal

I got news for you Greg.. VDP / VDR has been in BETA since it has been out, we have BEEN complaining about this lousy software since DAY 1!.. NOW you are sorry to hear we have a problem?!?!?!?!?!?!?

Uh... you don't pay attention much do you?

VDR has been buggy since inception, apparently VDP isn't any better NOW either. I will not recommend it to ANY vWare customer.

FIX IT!  Tell VMware engineers to ACTUALLY USE it, instead of programming .. and as the OP put it.. stop MAKING USERS BETA TESTERS!

I don't see how it's SO blatanly obvious it's such a worthless, horrid software that VMware refuses to ACTUALLY see for themselves..  It NEEDS to be FIXED NOW!!!

Reply
0 Kudos
RParker
Immortal
Immortal

Let us also not forget the 100 VM limitation per instance of VDP, it will do 8 concurrent backups (not settable) per instance, and it's limited to 1TB EACH datastore attached to VDP instance, and you can only have 2!

VDP is just a completely useless, and utterly buggy.  I am glad you posted what you did... don't let up, keep on them..

Reply
0 Kudos
Josh26
Virtuoso
Virtuoso

Greg Sparks wrote:

Plus, in multi OS/browser environments, the Flex UI is a real treat.  Any particular feedback you have is welcome here as well.

Thank you,

"Flash" is, by definition, the opposite of "a real treat".

Sensible security policies ban it on servers. Being "multi OS" would have actually been reached if the interface was HTML5, but it isn't. Instead, it's a proprietry system, supported on the platforms Adobe chooses to support.

It's clunky and crashes on a regular basis. There's no value in filing an SR because it's usually a safe assumption that the response will be "that's an Adobe crash and we aren't responsible for it".

Reply
0 Kudos
Anonymous
Not applicable

I also upgraded from vdr to vdp and I'm really disappointed with the upgrade. I spent 2 days configuring vdp and experienced the same problems mentioned in this discussion (sso problems, dns problems, configuration problems, ...).

I would be nice if Vmware added a VDP administrative interface in the vSphere Windows client with FLR support. This would solve a lot of annoyances for adminstrators (Flash requirement on a server for FLR, clumsy web interface, ...). When I go to settings - plugins in the vSphere Windows client the VDP plugin is already listed so maybe this is already (partially) implemented ?

Vmware, if you are listening to your (paying) customers you should at least give us a timeframe in which these problems will be resolved.

krismcewan
Enthusiast
Enthusiast

As a consultant having recomended our customers to use VDP we are now in the embarrasing situation of having to tell them its not suitable for production use.

We have NEVER managed to get the Webclient to connect to the appliance. (outstanding SR with VMware on this 6 weeks and counting)

Based on everyone elses experiances We have NO confidence when we do get it working that it will work.

Having a VM with internal storage to back up to is UNNACEPTABLE.  95% of customers have a single SAN that they use for VM's they do not want to backup their VM's to the same SAN.  Not being able to use a CIFS share that is either external or on cheap storage that can be dumped to tape and located offsite is not a backup solution.

The boot time is UNNACEPTABLE.  Being on the phone with VMware Support for 4 Hours half of which is waiting for a VM to Boot up is a waste of my time and the poor Support person at Vmware.

Worst of all Vmware Support do not have the skills to resolve issues as they seem to be Escalated to EMC Avamar team.

This is a serious hit to the Reputation of VMware software.  Stop letting the Sales and Marketing team drive innovation.

A VMware Consultant in Scotland for Taupo Consulting http://www.taupoconsulting.co.uk If you think I'm right or helpful award me some points please
PlanglowLtd
Contributor
Contributor

I couldn't agree more on most of what has already been said in this thread.

Currently I'm re-installing VMware Data Recovery to backup to an external NAS for disaster recovery. Other than VDR the only other options offered to me so far have been £1000+ per host licensed products... forget that!

Reply
0 Kudos
edinburgh1874
Enthusiast
Enthusiast

Are there any fixes for the Java processes using large amounts of CPU?

I've just deployed a VDP appliance, with no backup jobs and Java is instantly using 3 GHz....the VMWare KB just states that a reboot is the fix, the issue just reappears.

Cheers

Reply
0 Kudos
Buschfunker
Contributor
Contributor

Same Problem here.

8Ghz of CPU usage.

Why is there no statement from VMware itself?

This backup solution works very unreliable.

Reply
0 Kudos
edinburgh1874
Enthusiast
Enthusiast

Hi All,

I actually received a call from VMWare which helped us understand the issues with VDP a bit better.

VDP will use as much resources as it possibily can to do backup, maintainance and integrity jobs - we seen it use 10GHz and 32Gb during a job.

To counter this if it's in a DRS cluster, you can limit the CPU/memory usage, the jobs will complete but will take a much longer time.

VMWare don't set any limits out of the box, in order to make the jobs complete as quickly as possible.

The first job will consume a massive amount of resources, but subsequent jobs will be less intensive.

Backup jobs can appear to be running all day, but are not actually touching the source - it's running dedup on the backup data.

VDP is based on EMC Avamar, which is usually deployed on an appliance dedicated to backups so is used to having it's own resources.

So, given this, we deployed it on it on an old HP G1 with it's own storage and it's much happier.

Also, the appliance must be allowed to complete the blackout window - so this may mean rebooting, etc, during a backup window...perhaps some folk are doing maintaince in the blackout window causing integrity issues.

When I first tried out the product, I thought that all maintainance was in the "maintainance" window and that the blackout window was clear...I didn't read the manual well enough.

Reply
0 Kudos
CraigD
Enthusiast
Enthusiast

We setup (EDIT) VDP and have it backing up two servers in a test ESXi environment.  We added the vCenter appliance and the VDP appliance to our existing test ESXi box in order to do this.  I did not want to set up VDP on our production vSphere 5.1 system, and I do not have the free space on our production SAN.

I hate it that I cannot "point to" external storage from the appliance.  Instead I must carve out enough space for the pre-allocated appliance.  Also, if I choose wrong (i.e. a smaller appliance than what I really need), there is no way for me to move to a larger appliance and keep my backups.

The FLR process is very cumbersome and not very practical.  I have tested it and it does work, but wow, what a chore to get a file off a backup!  This must be improved.

Lastly, there really needs to be a way to ship the backups offsite.  I would love to use Amazon or some other cloud provider.  Maybe someone can outline for me how to do this... or maybe there isn't a way.  From what I have read, VDP (and its data) just isn't "portable."

What I really want (30,000 foot view) is a system that will run full VM backups that are both runnable and accessible at the file level (for file-level restores).  I want to be able to store these backups on local storage and also transfer them to offsite (preferably to the cloud).  Lastly, I want to ensure that in the event of a disaster I can connect these backed-up VMs to a completely new VMware environment (if necessary) and crank them up and have them run happily.

Anybody got tips?  Veeam, perhaps?  Can it be done with VDP?  Is it on the VDP road map?

VDP has so much promise, conceptually... it just needs to actually work.

Reply
0 Kudos
edinburgh1874
Enthusiast
Enthusiast

Offsite backups is potentially possible using vSphere replication of the VDP VM...I'm looking to implement this but am still building the DR site.

The other option is hosting VDP on an NFS datastore, then replicating that.

Reply
0 Kudos
jb42
Contributor
Contributor

I agree in principle, I wish it was easier. But after struggling with it for a while, I think it can be done.

I'm hosting VDP on iSCSI zfs volumes. ZFS has snapshots and replication. Replicated appliance is coming on line as I type so I'll report back with results. Here's another DR method utilizing ZFS that doesn't include VDP: http://blog.laspina.ca/ubiquitous/provisioning_disaster_recovery_with_zfs. And here's a blog post showing an example VDP replication technique, though not the one I've used: http://blogs.vmware.com/vsphere/2012/12/recover-replicated-vdp-appliance.html.

If you look into ZFS, OpenIndiana and Napp-it are good searches to start with.

Reply
0 Kudos
gravesg
Enthusiast
Enthusiast

I've have not one, but now TWO VDP appliance unrecoverable crashes.

Seriously, how bad can this product be?

Reply
0 Kudos
edinburgh1874
Enthusiast
Enthusiast

I'm thinking of ditching it, we had a Linux VM that hung during a VDP backup and then reported "Unknown Internal Error" on trying to reboot it.

We weren't able to restore any of the backups.

Such a shame, since VDR was quite reliable.

Reply
0 Kudos