Help with my VDP, disk full issue

[hiscal]

my VDP version is 6.0.2, when I first ran my backup jobs it got problem.

Appliance status stopped at "admin" status and refused to get back to "normal".

"gsan" service was degraded. From the logs I can see it might caused by disk full, but even I cleared all my restore points, disk space won't be reclaimed because appliance status is "admin".

Here are some of my logs, anyone can help? Thanks a lot.

df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2             7.9G  4.8G  2.8G  64% /
udev                  3.9G  136K  3.9G   1% /dev
tmpfs                 3.9G     0  3.9G   0% /dev/shm
/dev/sda1             130M   38M   86M  31% /boot
/dev/sda7             1.5G  121M  1.3G   9% /var
/dev/sda9              77G  4.9G   68G   7% /space
/dev/sdb1             512G  498G   15G  98% /data01
/dev/sdc1             512G  502G   11G  98% /data02
/dev/sdd1             512G  501G   12G  98% /data03

cplist
cp.20150808160314 Sat Aug  8 09:03:14 2015   valid rol ---  nodes   1/1 stripes   3253
cp.20150808165046 Sat Aug  8 09:50:46 2015   valid rol ---  nodes   1/1 stripes   3253

status.dpn
Sun Aug 16 19:05:51 PDT 2015  [servername] Mon Aug 17 02:05:51 2015 UTC (Initialized Tue Aug  4 05:13:53 2015 UTC)
Node   IP Address     Version   State   Runlevel  Srvr+Root+User Dis Suspend Load UsedMB Errlen  %Full   Percent Full and Stripe Status by Disk
0.0   10.192.14.192 7.1.81-107  ONLINE fullaccess mhpu+0hpu+0000   1 false   0.32 7590  2374797  12.1%  12%(onl:1086) 12%(onl:1084) 12%(onl:1083)
Srvr+Root+User Modes = migrate + hfswriteable + persistwriteable + useraccntwriteable

System ID: 1438665233@00:50:56:AB:4E:38

All reported states=(ONLINE), runlevels=(fullaccess), modes=(mhpu+0hpu+0000)
System-Status: ok
Access-Status: admin

Checkpoint failed with result MSG_ERR_DISKFULL : cp.20150816150216 started Sun Aug 16 08:02:46 2015 ended Sun Aug 16 08:02:46 2015, completed 0 of 3253 stripes
Last GC: finished Sun Aug 16 08:01:46 2015 after 00m 30s >> recovered 0.00 KB (MSG_ERR_DISKFULL)
No hfscheck yet

Maintenance windows scheduler capacity profile is active.
  The maintenance window is currently running.
  Next backup window start time: Sun Aug 16 20:00:00 2015 PDT
  Next maintenance window start time: Mon Aug 17 08:00:00 2015 PDT

dpnclt status
Identity added: /home/dpn/.ssh/dpnid (/home/dpn/.ssh/dpnid)
dpnctl: INFO: gsan status: degraded
dpnctl: INFO: MCS status: up.
dpnctl: INFO: Backup scheduler status: up.
dpnctl: INFO: axionfs status: up.
dpnctl: INFO: Maintenance windows scheduler status: enabled.
dpnctl: INFO: Unattended startup status: enabled.

[RyanJMN · Accepted solution]

I looks like the checkpoints are taking up all of the space allocated to them.  Before VDP deletes a checkpoinit it goes through a process of creating a checkpoint and validating the integrity of the previous checkpoint.  I believe you will need to contact support to resolve the issue.

With that said it appears like you deleted all the restore points.  If so it would be easier to delete and start over instead of trying to resolve the issue.

[hiscal]

Thank you Ryan. It has been fixed with the help of support.

[gameto]

Hi, hiscal

I need your help.

My VDP also has the same problem as you.

Please let me know how you solved this problem.

Thank you so much.

Best Regards,

Neal.Choi

E-Mail : sanha.choi.ext@nokia.com