Rediculous (VDR) Recatalog / Reclaming times

[RParker]

I started this process last week, on Thursday. First I noticed that VDR was 'reclaiming' not sure what it's 'reclaiming, but I assume space.

So I waited, and waited.. 36 hours later, STILL reclaiming. so I cancelled it. Next it was reindexing, that took another 24 hours, and I tried to stop it.. gracefully. That took 4 hours, never did stop, so I rebooted the appliance.

NOW it's re-cataloging, and that has been going since last night (which almost 24 hours).

I can see disk activity, but this is rediculous to the point of absurd, 5-8 MBS IOPs for over 100hours? What the h-E- doble hockey sticks is TAKING so long?!?!

I ask you, what could it POSSIBLY be doing?

I am ready to delete it and start all over AGAIN (don't care if I lose my backups, I can't run my backups until this thing quits, which at this pace (37%) could be next week...

[whitesj]

Seems a lot of people are having issues with the VDR. I think there are a few other posts here in the forum about it.

[admin]

Can you share (either via this thread or via a private message - see questions below) - we are looking for trends on why we see such a huge performance spectrum in our customer base. Just to give you an example of my test setup running vSphere (2 ESX hosts, iSCSI array with SAS drives, 40GB vmdk as dedupe store with out 18Gb used ) here are my latest log entries - this is the duration we generally expect these operations to take

8/11/2009 3:17:52 PM: Executing Integrity Check

8/11/2009 3:17:52 PM: To Backup Set /SCSI-0:1/...

8/11/2009 4:07:54 PM: Task completed successfully

8/11/2009 4:07:54 PM: Remaining: 1499 files, 15327.2 GB

8/11/2009 4:07:54 PM: Completed: 328 files, 699.7 GB

8/11/2009 4:07:54 PM: Performance: 14329.3 MB/minute

8/11/2009 4:07:54 PM: Duration: 00:50:00

8/12/2009 2:33:15 PM: Executing Reclaiming

8/12/2009 2:33:15 PM: Reclaiming Destination /SCSI-0:1/...

8/12/2009 2:33:25 PM: Deleted restore point 8/5/2009 10:54:32 AM for virtual machine Nostalgia Games - DOS

8/12/2009 2:44:17 PM: Task completed successfully

8/12/2009 2:44:17 PM: Duration: 00:11:02

Inventoty questions

1) What type box is the destination (home-grown desktop, SMB NAS,Enterprise Filer, etc.)?

2) What transport are the destination disks? (SCSI, USB, FC, etc.)?

3) Connection speed from source host to destination?

4) If CIFS, what OS is running on the CIFS serve (Windows or Linux) -- including version. If Linux, what is the Samba version that is running?

5) If CIFS, Is the CIFS destination used for other things (other sharing, any other processes)?

6) Total destination disk size and % used

There is a datarecovery.ini file to override the policy engine defaults around frequency and actions of the dedupe store access. These are the options that you can use to override the policy engine but it does not solve the base performance problem. that is what the above questions are trying to determine - and ways we can further optimize VDR.

RetentionPolicyInterval the amount of days before doing an reclaim on the dedupe store, values 1 - 7.

DedupeCheckOnRecatalog if an integrity check should be run after a recatalog.

So if you want to the reclaim to run only once a week and not run an integrity check after a recatalog, then the datarecovery.inin options would look like the following

RetentionPolicyInterval=7

DedupeCheckOnRecatalog=0

[RParker]

1) What type box is the destination (home-grown desktop, SMB NAS,Enterprise Filer, etc.)?

NAS box attached via Fiber, specifically Jetstor: http://www.jetstor.com/02_01_jetstor_sata_516f.html

SATA Array (16 * 1TB Disks)

2) What transport are the destination disks? (SCSI, USB, FC, etc.)?

SCSI (SATA)

3) Connection speed from source host to destination?

4GB Fiber connection, via an ESX datastore. The vmdk is attached directly to the VDR Appliance.

6) Total destination disk size and % used

2TB Datastore

There is a datarecovery.ini file to override the policy engine defaultsCan you give us ALL available options for this file? I have:

Those you mentioned plus integrity check. That's 3 options I have, any more?

[admin]

Thanks - One single 2TB store or 2 * 1TB store?

From the Admin Guide (http://www.vmware.com/pdf/vdr_10_admin.pdf)

Data Recovery is designed to support deduplication stores that are up to one terabyte in size and each backup

appliance is designed to support the use of two deduplication stores. Data Recovery does not impose limits

on the size of deduplication stores or number of deduplication stores, but if more than two stores are used or

as the size of a store exceeds one terabyte, performance may be affected.

[RParker]

but if more than two stores are used or as the size of a store exceeds one terabyte, performance may be affected.

DOH!

Thanks - One single 2TB store or 2 * 1TB store?

Uh a single 2TB store

[D1Q4]

hey..

im sorry to continue this issuee.. im using VDR 1.2.1 1616, and destination backup pointing to 900GB vdisk using LSI logic in my environment.

i have problem when they running reclaiming after restarting vdr appliance.. is it normal?? because no one backup job running when reclaiming still in progress.. what if i manually cancel the reclaiming??

i'm sorry because this such as fool's question.

[DSTAVERT]

You should always create a new discussion and explain your situation as completely as possible.

I would let it run. Read the documentation in the reclaim section for issues that might warrant leaving it run.

Consider using two smaller destination disks.