Tragic VDP backup performance to NAS via iSCSI

[dwma]

Hello,

From few months, we're experiencing tragic VDP backup performance to our Synology NAS DS1513+ datastore (5x WD Black 2TB in RAID5) via iSCSI. From the time, nothing changed in our environment (except new NAS Synology DSM software updates).

We're backing up vCenter server (115GB) in 42 minutes, Windows Server 2012 (100GB) in 12 minutes, which is acceptable for us. We also have another W2012 server machine (file server) which is 900GB big (100GB system disk, 800GB data disk), which takes 9 to 11 hours! New bytes are around 0.3 to 0.5%... Previously it took about half this time to backup this VM.

We've checked our network, and from network side, everything it's fine. Our server has aggregated 8x 1Gbps links, Synology NAS has 2x 1Gbps links. The VDP appliance (6.1.2.19 version) - has 4 vCPUs (E5-2430) & 12 gigs of RAM, and it's not using max 30% CPU when backing up, and average 45% or RAM.

I've also tested the RAID5 performance (with DD) on our synology, and write performance is about 278MB/s, mdadm showed 291,7MB/s (disk read). I tried to do storage performance check via VDP (in vdp-configure panel), but it fails every time (exceeded allowed time to complete).

Probably I've exhausted all my ideas for now. Maybe it's VDP appliance fault (low performance)?

[daphnissov · Accepted solution]

You may want to try resetting CBT on those machines. If you're doing incrementals on a 900 GB file server which experiences a change rate of 0.3-0.5%, it should be taking nowhere near that amount of time, even given poor performance, unless it's rescanning the entire disk. Also, I hate to break it to you, but VDP never was a good performer to begin with so there may not be a whole lot from an appliance perspective you can do. You do get what you pay for...

[dwma]

Great, thank you!

After CBT reset, now backup of our fileserver takes from about 1 to 4 hours max, but the 1st backup (full backup) took 39 hours!

We've set a new VDP appliance for test, and we have separated another DS1513+ NAS, to check the "performance analysis" in the fresh VDP... but it's failing everytime (time exceeded).