VDP 6.1 freezes while executing backup jobs

[dspeight]

Hi

I've been having trouble with the VDP Appliance in Version 6.1. Its a fresh install, running on a Windows vCenter 6.0.0 Build 3018524 and using a Buffalo TeraStation NAS as backup destination.

The problem starts when I try any kind of backup job. Before that everything works as intended. Creating jobs, managing the appliance etc. Once I start a backup job the appliance kind of freezes. In the web client it wont load any data

Sometimes in runs into the error that the time snyc. isnt working properly. Although I have checked vCenter Server, the hosts and the appliance. They're all configured with the proper ntp server and syncing properly. The VDP appliance is also showing the correct time. Restartet the tools services aswell.

At some point the Web Client looses the connection to the appliance.

Connecting over SSH to the appliance sometimes runs into timeouts aswell. Restarting the VDP appliance takes about 20-30 minutes.

UPDATE:

After literally a hour I was able to connect back into the appliance within the the web client. My test backup job started gradually counting up to 38% (Its 20GB small and really shouldnt take a hour to backup). After a few minutes it disconnected again.. the task stopping at 38%.

One thing to note: The VDPs virtual install disk is running on the same datastore as where its backing up to. Could this be the problem im having? I cant really any other problems within the config.

David

[dspeight]

I think I answered my own question :smileyblush:

I storage migrated the install disk and config files over to my cluster storage wich runs alot faster than the NAS. Just started another Backup Job. Running perfectly fine so far.. everything working as intended.

[dotAndy]

I had this after updating from 5.8 to 6.1 but rebooting the appliance seemed to sort it out for a while.

Similar scenario, the VDP disks (system and backup stores) are located on the same NAS device via iSCSI. Next time the appliance gets grumpy I might try moving the system disk to a local datastore on the host.

We nearly rolled back to 5.8 because the performance of 6.1 seems to deteriorate over a few days - we were starting to get alerts stating "The most recent checkpoint for the VDP appliance is outdated". However since setting up a task to reboot the appliance every 48 hours it seems to be running more smoothly.

[dspeight]

Well .. my success was short-lived.

Starting a set of backups stopped at 92% (although there are TBs of storage left) all while the appliance crashing or rather becoming unreachable for a while. Im getting messages aswell saying that checkpoints are outdated and that "Management Services arent running" or "Backupplaner not running".

Any suggestions? Im this close to reinstalling the appliance. Maybe on another Version?

David

[dotAndy]

Reinstalling from scratch didn't work for me. At first I thought the appliance maybe hadn't upgraded properly so I installed a new instance of VDP and re-created the backup schedules so there was nothing brought across from the upgrade appliance.

So far, rebooting every 48 hours is working for me so I wonder if there is some kind of resource leak. Annoyingly I now get alert emails telling me the services are stopped because vCenter has requested a guest reboot!

If the problems re-occur our plan is to try the earlier VDP 6.0.2 appliance and see if that still has the same issue. I don't really want to go back to 5.8 (which worked OK) as I don't think backing up ESXi 6 hosts is in the compatibility matrix, upgrading to VDP 6.1 was the first step in our vSphere 5.5 to 6 upgrade, so it hasn't gone well so far!

If you're struggling to get through even the first backup window are you able to allocate any additional RAM to the VDP appliance?

[PatricioGB]

This product has not yet mature enough to work optimally, gives many problems, I have worked since version 5.1 and so far they are in version 6 still many problems, lack many more VMware team work.

[jnsvano]

OK, then one can try to add more memory to the instance (default is 4GB), seems to be it's constantly swapping...

[RyanJMN]

What size appliance are you using?  Did you run a performance test on the storage?

I have experience installing VDP on one Buffalo TeraStation.  I got 56 MB/s write and 84 MB/s read with the storage performance test.  Because it was NFS and thin provisioned I couldn't get a good seek test but suspect it performed poorly.  Disk latency was poor if I remember correctly 50ms wasn't all the uncommon.  The 2TB appliance worked alright but not as good as other installs I've done using different storage.  It was the only thing running on the storage.  There are other NAS devices that perform better and are on the VMWare supported list that are not all that expensive.

[AlexM100]

Hi, same problems with the VDP 6.1 positioned in a non-performance FC storage, this is my solution:

- Deploy new appliance latest release 6.1.1.19, in my case from 0.5 TB;
- Increased RAM from 4 GB to 8GB;
- Limit concurrent backups to 1 (https://communities.vmware.com/thread/498026);

In this way I solved the exact same problems experienced by all, the basic appliance VDP always swapped to disk, it is not properly configured. I hope to help you.

Alessandro.

[dotAndy]

Ended up downgrading to 6.0.3 here (well, installing a new appliance from scratch) and haven't had any problems since, running a 0.5TB store with the default 4GB RAM.

I've come to the conclusion that out-of-the-box 6.1.x is a dud release so hopefully whatever the problem with it is will be fixed in a future update. Chucking even more host resources at the VM for what seems to be a minor update isn't very appealing...

[vdp4life]

Never had success using a TeraStation but I did have success with a QNAP appliance.

Did you do a direct connect with Jumbo frames to help?