What speed and dedup are you all seeing with vDR?

[HenrikElm]

To be able to plat vDR deployments, I'd like to know wome things from you all that have deployed Data Recovery 1.1.

1) What typical speed are you experiencing to backup, say, 500 GB of data? Time for initial backup? Time for incremental backups? Over what links? Does it saturate a 1 Gb Link when doing backup?

2) What kind of deduplication ratios are you seeing? Like above, a 500 Gb backup results in X Gb of space on the destination? Should be easy to check if one uses a Windows SMB share as destination. I know it depends on the similarity of the servers being backed up, but I'm still interested in your average experience.

/Henrik

[admin]

"Does it saturate a 1 Gb Link when doing backup"

I just wanted to highlight the transport mechanisms that VDR can leverage in terms of tranferring data - note that VDR tries hot add first.

HOTADD (short for SCSI Hot-Add) is a protocol used when accessing virtual SCSI hard-disks from within a virtual machine so that disks can be added to/removed from a running virtual machine without shutting down the virtual machine. In this instance, VDR Block-read/block-write requests do not go over the LAN.

SAN (standing for Storage Area Network) is a transport mode to enable backups of virtual disks that are stored on a LUN on a Fibre Channel storage array that is connected to ESX. Using SAN, a client can access virtual disk data directly across the SAN, thereby bypassing the LAN and avoiding to put any additional load on production ESX Servers. In this instance, VDR Block-read/block-write requests do not go over the LAN.

NBD (Network Block Device) is a network-based transport mode. When using NBD, a client connects to the ESX Server hosting the Virtual Machine that the virtual disk it wants to access belongs to. VDR block-read/block-write requests are sent over the LAN to ESX.

NBDSSL: Is the same protocol as NBD, but communication between ESX and the client happens through an encrypted SSL tunnel.

[HenrikElm]

Interesting information. I was thinking more about network speed from the vDR appliance to the SMB destination, but this was also good info.

Still, any estimate about speed to destination?

/Henrik

[Paul11]

to 1)

Full Backup with VDR and 80 GB Disk over 1 Gbit Network to CIFS-Share (84-Disks)

12/11/2009 11:28:42 AM: Task completed successfully

12/11/2009 11:28:42 AM: Completed: 5 files, 80,1 GB

12/11/2009 11:28:42 AM: Performance: 20617 MB/minute

12/11/2009 11:28:42 AM: Duration: 00:40:25 (00:00:41 idle/loading/preparing)

Incremental Backup with VDR and 80 GB Disk over 1 Gbit Network to CIFS-Share (84-Disks)

12/12/2009 10:56:10 AM: Task completed successfully

12/12/2009 10:56:10 AM: Completed: 5 files, 80,1 GB

12/12/2009 10:56:10 AM: Performance: 261446 MB/minute

12/12/2009 10:56:10 AM: Duration: 00:03:36 (00:00:28 idle/loading/preparing)

to 2)

My VCB Backups needs 1.4 TB to Backup 88 VM's to Disk (-F = 0, compaction is on) every day.

VDR needs 400 GB for the first Full-Backup of all VM's, and than over the time it grows to a maximum of 480 GB for all Backups with Retention Policy "More" (which is the default).

So I think it is a very good deduplication ratio, if you mention, that you get 1.4 TB with VCB every day and only 480 GB for the sum of all your daily Backups with VDR!!

I like this product since 1.1. (no comment to the releases before)

Hope this helps!

[HenrikElm]

Yes. Thank you very much for this info. Exactly what I was looking for. If anyone else would be willing to share their same statistics, I'd be most grateful.

The more real world statistics we all get the better.

/H

[jjewett]

Here is the vDR log from our environment. (VM names have been changed) Performance times and data throughput stats will be confusing if you don't take into account the deduplication process. It takes quite a bit of CPU time to dedup the data in a VM and it seems to be where the bottleneck is. Our Full backup times are usually around 2.5GB/min which I think is a reflection of the CPU power more than of network or disk bandwidth. If our appliance ran on a faster CPU, then I think the backup times would improve, but for now 2.5GB/min of data is about all the vDR appliance can process.

12/13/2009 7:24:21 PM: Executing Integrity Check

12/13/2009 7:24:21 PM: To Backup Set /SCSI-0:2/...

12/13/2009 7:24:26 PM: Starting full integrity check

12/13/2009 9:26:54 PM: Task completed successfully

12/13/2009 9:26:54 PM: Remaining: 65 files, 282.5 GB

12/13/2009 9:26:54 PM: Completed: 136 files, 1152.1 GB

12/13/2009 9:26:54 PM: Performance: 9633.7 MB/minute

12/13/2009 9:26:54 PM: Duration: 02:02:32 (00:00:05 idle/loading/preparing)

12/13/2009 9:27:05 PM: Normal backup using vm01 Backup Job

12/13/2009 9:32:22 PM: Copying vm01

12/13/2009 9:32:45 PM: Performing incremental back up of disk "[vm-sas01] vm01/vm01-flat.vmdk" using "Network"

12/13/2009 9:34:27 PM: Performing incremental back up of disk "[vm-sas01] vm01/vm01_1-flat.vmdk" using "Network"

12/13/2009 9:35:16 PM: Task completed successfully

12/13/2009 9:35:16 PM: Completed: 7 files, 34 GB

12/13/2009 9:35:16 PM: Performance: 13449.8 MB/minute

12/13/2009 9:35:16 PM: Duration: 00:08:11 (00:05:36 idle/loading/preparing)

12/13/2009 7:58:56 PM: Normal backup using vm02 Backup Job

12/13/2009 8:03:14 PM: Copying vm02

12/13/2009 8:03:41 PM: Performing incremental back up of disk "[vm-sas01] vm02/vm02-flat.vmdk" using "SCSI Hot-Add"

12/13/2009 9:35:17 PM: Performing incremental back up of disk "[vm-sas01] vm02/vm02_1-flat.vmdk" using "SCSI Hot-Add"

12/13/2009 9:36:29 PM: Task completed successfully

12/13/2009 9:36:29 PM: Completed: 7 files, 20.2 GB

12/13/2009 9:36:29 PM: Performance: 222.1 MB/minute

12/13/2009 9:36:29 PM: Duration: 01:37:32 (00:04:41 idle/loading/preparing)

12/13/2009 8:53:46 PM: Normal backup using vm-winxp Backup Job

12/13/2009 8:58:30 PM: Copying vm-winxp

12/13/2009 8:58:52 PM: Performing incremental back up of disk "[vm-sas01] vm-winxp/vm-winxp-flat.vmdk" using "SCSI Hot-Add"

12/13/2009 9:37:33 PM: Task completed successfully

12/13/2009 9:37:33 PM: Completed: 5 files, 19.1 GB

12/13/2009 9:37:33 PM: Performance: 502.8 MB/minute

12/13/2009 9:37:33 PM: Duration: 00:43:48 (00:05:04 idle/loading/preparing)

12/13/2009 7:33:24 PM: Normal backup using vm03 Backup Job

12/13/2009 7:34:40 PM: Copying vm03

12/13/2009 7:35:50 PM: Performing incremental back up of disk "[vm-sas01] vm03/vm03-flat.vmdk" using "SCSI Hot-Add"

12/13/2009 7:55:53 PM: Performing incremental back up of disk "[vm-sas01] vm03/vm03_1-flat.vmdk" using "SCSI Hot-Add"

12/13/2009 7:56:03 PM: Performing incremental back up of disk "[vm-sas01] vm03/vm03_2-flat.vmdk" using "SCSI Hot-Add"

12/13/2009 8:09:16 PM: Performing incremental back up of disk "[vm-sas01] vm03/vm03_3-flat.vmdk" using "SCSI Hot-Add"

12/13/2009 9:50:51 PM: Task completed successfully

12/13/2009 9:50:51 PM: Completed: 11 files, 301.8 GB

12/13/2009 9:50:51 PM: Performance: 2288.6 MB/minute

12/13/2009 9:50:51 PM: Duration: 02:17:25 (00:02:24 idle/loading/preparing)

12/13/2009 7:34:26 PM: Normal backup using emailvm Backup Job

12/13/2009 7:37:35 PM: Copying emailvm

12/13/2009 7:38:01 PM: Performing full back up of disk "[vm-sas01] emailvm/emailvm-flat.vmdk" using "SCSI Hot-Add"

12/13/2009 8:49:17 PM: Performing full back up of disk "[vm-sas01] emailvm/emailvm_1-flat.vmdk" using "SCSI Hot-Add"

12/13/2009 9:57:24 PM: Task completed successfully

12/13/2009 9:57:24 PM: Completed: 7 files, 64.6 GB

12/13/2009 9:57:24 PM: Performance: 473.8 MB/minute

12/13/2009 9:57:24 PM: Duration: 02:22:57 (00:03:33 idle/loading/preparing)

12/13/2009 8:03:49 PM: Normal backup using vm04 Backup Job

12/13/2009 8:08:10 PM: Copying vm04

12/13/2009 8:08:33 PM: Performing full back up of disk "[vm-sas01] vm04/vm04-flat.vmdk" using "Network"

12/13/2009 8:50:56 PM: Performing full back up of disk "[vm-sas01] vm04/vm04_1-flat.vmdk" using "Network"

12/13/2009 9:59:37 PM: Task completed successfully

12/13/2009 9:59:37 PM: Completed: 7 files, 17 GB

12/13/2009 9:59:37 PM: Performance: 156.0 MB/minute

12/13/2009 9:59:37 PM: Duration: 01:55:47 (00:04:42 idle/loading/preparing)

12/13/2009 7:32:55 PM: Normal backup using dmmnc1 Backup Job

12/13/2009 7:33:41 PM: Copying dgmmnc1

12/13/2009 7:34:14 PM: Performing incremental back up of disk "[vm-sas01] dgmmnc1/dgmmnc1-flat.vmdk" using "SCSI Hot-Add"

12/13/2009 8:56:37 PM: Performing incremental back up of disk "[vm-sas01] dgmmnc1/dgmmnc1_1-flat.vmdk" using "SCSI Hot-Add"

12/13/2009 10:00:26 PM: Performing incremental back up of disk "[vm-sas01] dgmmnc1/dgmmnc1_2-flat.vmdk" using "SCSI Hot-Add"

12/13/2009 10:08:08 PM: Task completed successfully

12/13/2009 10:08:08 PM: Completed: 9 files, 93.4 GB

12/13/2009 10:08:08 PM: Performance: 621.3 MB/minute

12/13/2009 10:08:08 PM: Duration: 02:35:12 (00:01:17 idle/loading/preparing)

12/13/2009 7:49:25 PM: Normal backup using vm05 Backup Job

12/13/2009 7:53:16 PM: Copying vm05

12/13/2009 7:53:50 PM: Performing incremental back up of disk "[vm-sas01] vm05/vm05-flat.vmdk" using "Network"

12/13/2009 9:37:29 PM: Performing incremental back up of disk "[vm-sas01] vm05/vm05_1-flat.vmdk" using "Network"

12/13/2009 10:26:52 PM: Task completed successfully

12/13/2009 10:26:52 PM: Completed: 7 files, 120.2 GB

12/13/2009 10:26:52 PM: Performance: 803.1 MB/minute

12/13/2009 10:26:52 PM: Duration: 02:37:25 (00:04:16 idle/loading/preparing)

12/14/2009 11:30:30 AM: Starting VMware Data Recovery, version 1.1.0.707