I have shell script which executes the file based native backup via cron job on a daily basis.
It gets triggered by making the POST call to the endpoint /rest/appliance/recovery/backup/job.
Backup shipping is done via SCP protocol.
Log::
The /var/log/vmware/applmgmt/backup.log shows the following trace
2019-10-17 02:00:38,291 [ConfigFilesBackup:PID-38685] INFO: tarCmd = tar -cz -C / --ignore-failed-read -T "/var/log/vmware/applmgmt/cfg_incl_pq_ggq.lst" -X "/var/log/vmware/applmgmt/cfg_excl_eB6DWP.lst" --warning="no-file-ignored"
2019-10-17 02:00:38,340 [LotusBackup:PID-38686] INFO: stdout: VdcBackupDB: Setting vmdir state to VMDIRD_READ_ONLY
VdcBackupDB: Backing up: /storage/db/vmware-vmdir//data.mdb
VdcBackupDB: Backing up: /storage/db/vmware-vmdir//lock.mdb
VdcBackupDB: Setting vmdir state to (3)
2019-10-17 02:00:38,340 [LotusBackup:PID-38686] INFO: BackupLotus: Dispatching files ['lock.mdb', 'data.mdb', 'partnerlist']
2019-10-17 02:00:38,341 [LotusBackup:PID-38686] INFO: tarCmd = tar -cz -C /tmp/backup_lotus/ --ignore-failed-read --warning="no-file-ignored" "lock.mdb" "data.mdb" "partnerlist"
2019-10-17 02:00:40,595 [LotusBackup:PID-38686] ERROR: Failed to backup Lotus DB
2019-10-17 02:00:40,595 [LotusBackup:PID-38686] ERROR: Failed to backup Lotus DB
Underlying process status. rc: 255
stdout:
stderr:
Traceback (most recent call last):
File "/usr/lib/applmgmt/backup_restore/py/vmware/appliance/backup_restore/components/Lotus.py", line 93, in BackupLotus
raise BackupRestoreError(err_msg, status)
BackupRestoreError: Failed to backup Lotus DB
Underlying process status. rc: 255
stdout:
stderr:
2019-10-17 02:00:40,696 [LotusBackup:PID-38686] INFO: Successfully completed Lotus cleanup.
2019-10-17 02:00:41,113 [MainProcess:PID-38117] ERROR: Error at process LotusBackup; rc:255.
2019-10-17 02:00:41,113 [MainProcess:PID-38117] ERROR: stderr:Failed to backup Lotus DB
2019-10-17 02:00:41,113 [MainProcess:PID-38117] INFO: Following error message isn't localized:
stderr:Failed to backup Lotus DB
2019-10-17 02:00:41,114 [MainProcess:PID-38117] ERROR: BackupManager encountered an exception: Hit exception inside process LotusBackup:
2019-10-17 02:00:41,115 [MainProcess:PID-38117] INFO: Cleaning up the failed backup job.
2019-10-17 02:00:41,116 [MainProcess:PID-38117] INFO: Cleaning up all running backup child processes.
2019-10-17 02:00:41,181 [MainProcess:PID-38117] INFO: Cleaning up all sizeFiles.
2019-10-17 02:00:41,181 [MainProcess:PID-38117] INFO: Cleaning up ConfigFiles.
2019-10-17 02:00:41,181 [MainProcess:PID-38117] INFO: Cleaning up Lotus.
2019-10-17 02:00:41,181 [MainProcess:PID-38117] INFO: Successfully completed Lotus cleanup.
Solution I found is simply executing the script again to fire the backup, and it works.
Please help me understand,
1. What is Lotus DB and what information it holds w.r.t vCenter Appliance.
2. How can debug further to find the root cause for this issue.?
Thanks.
Is it a windows or linux server you are backing up to?
It basically holdsyour sso information.
And what is the size of data.mdb file?
/storage/db/vmware-vmdir
It's a Linux based appliance.
I don't have that path,
under /storage/db,
I have
journal
lost+found
vmware-imagebuilder
vmware-vmafd
vpostgres
External PSC?
yes. this is external PSC.
also, 3 more vCenters use the same PSC.
Can you check the size of the data.mdb file from psc ?
/storage/db/vmware-vmdir
16K lost+found
172K vmware-vmafd
1.9G vmware-vmdir
data.mdb -- 1.8GB
Check your DM pls..