Upgraded to 7.0.2 and since then my backups are failing with an error...BackupManager encountered an exception. See logs for details. i have to admit i am not well versed at reading the log bundles. Where in that bundle might i find the backup info to troubleshoot? Any help is appreciated. thanks
Perry
What backup r u using and does it support 7.0 U2 ?
Hello Perry,
Please follow certain steps for me to understand this issue better.
1. ssh to the vcsa - in the folder /var/log/vmware/applmgmt you see backup.log, backupscheduler.log, backupschedulercron.log.
2. "grep" for error, fail etc.
NOTE - Which protocol you are using ?
Few debugging to start on your own
1. Make sure you have proper permission to file servers
2. Check space in your file servers
3. Try to mount fileserver(nfs/smb) locally in vCSA, if you are using those protocols to check if mounting is fine.
-Anil
Using FTP. Ran teh command you ask and got no return with grep.
Retry the backup for FTP and attach the backup.log file from VC here.
Hi
Can you please share the error code.
The odd part about this is that it's ben working fine up until a couple weeks ago. Nothing has changed at that point but all of the sudden it stopped working. Does it matter if i am running version 7
backup.log
2021-04-23T18:49:16.572 [20210423-184810-17694817] [LotusBackup:PID-21365] [Lotus::BackupLotus:Lotus.py:61] INFO: stdout: VdcBackupDB: Server in state (8) in which it cannot perform backup. error(9125)
2021-04-23T18:49:16.573 [20210423-184810-17694817] [LotusBackup:PID-21365] [Lotus::BackupLotus:Lotus.py:68] ERROR: BackupLotus: command "/usr/lib/vmware-vmdir/bin/vdcbackup /storage/db/vmware-vmdir/ /tmp/backup_lotus/" failed. rc: 165.
Please follow my blog on the same issue here.
https://virtual-power.in/f/noticed-few-users-on-6770-hitting-issues-with-vami-backups
Ajay,
No luck with your suggestion but thanks for the reply. Got to a point I had to finally open a ticket with support. Tech was on my vcenter for an hour adn was still stumped. I had to send them a log bundle. Still haven't heard back.
Hmm. Interesting. Does it fail with same error ?
SR number ?
It did. SR 21215999704
Same error. Doesn't look like you followed the complete blog. Your vmdird is still in standalone mode .
Point 4 from blog . https://virtual-power.in/f/noticed-few-users-on-6770-hitting-issues-with-vami-backups
backup.log
2021-04-26T18:15:57.319 [20210426-181512-17694817] [LotusBackup:PID-62583] [Lotus::BackupLotus:Lotus.py:61] INFO: stdout: VdcBackupDB: Server in state (8) in which it cannot perform backup. error(9125)
2021-04-26T18:15:57.320 [20210426-181512-17694817] [LotusBackup:PID-62583] [Lotus::BackupLotus:Lotus.py:68] ERROR: BackupLotus: command "/usr/lib/vmware-vmdir/bin/vdcbackup /storage/db/vmware-vmdir/ /tmp/backup_lotus/" failed. rc: 165.
vmdird-syslog.log
2021-04-26T15:28:33.724256+00:00 info vmdird t@139678593369920: VmDir State (8)
2021-04-26T15:28:33.724305+00:00 info vmdird t@139678593369920: Lotus Vmdird: running... state (8)
[\\services\vmdir]
+ "Arguments" REG_SZ "/usr/lib/vmware-vmdir/sbin/vmdird -s -l 0 -f /usr/lib/vmware-vmdir/share/config/vmdirschema.ldif -m standalone"
+ "dcAccount" REG_SZ "mwnvcs01.neuro.com"
Ran the command /usr/lib/vmware-vmdir/bin/vdcadmintool. Option 6 shows VmDir State is - (null). choose option 5 then type NORMAL. comes back with State of Vmdir set to NORMAL. Ran command /opt/likewise/bin/lwsm restart vmdir and the service restarts. type /usr/lib/vmware-vmdir/bin/vdcadmintool choose option 6 and it comes back with VmDir State is - (null)
vmdird null is bad. Let support handle it then .
Last shot:-
Make vmdird state normal first. Don't restart service.
Run /opt/likewise/bin/lwregshell delete_value "[HKEY_THIS_MACHINE\Services\vmdir]" Arguments
Then /opt/likewise/bin/lwsm refresh and /opt/likewise/bin/lwsm restart vmdir
Run /opt/likewise/bin/lwregshell and then cd HKEY_THIS_MACHINE\services\vmdir\.
Then type ls and check "Arguments" information . Grab the output.
Re-check the vmdird state again. If normal ; re-run backup.
Ajay
Just wanted to update you. vmware support has still yet to figure out what is going on. At one point we thought we had it working by doing a manual backup but after that it was right back to bad again.
Ok. I see . SO the moment vmdir was set to "NORMAL" it worked. But after reboot it again changed to "standalone".
I would still like to check point 4 and 5 from https://virtual-power.in/f/noticed-few-users-on-6770-hitting-issues-with-vami-backups.
May be ask the Support Guy to help with this and see if the vmdird goes back to standalone after reboot
Ajay,
Here is teh scoop. Seems like there was some corruption in the Directory Service in vCenter. The tech ended up trying to recreate the issue in his lab and wound up copying some files to my vcenter. Long story short, When we locked at the status of the DS it was set to STANDALONE (/usr/lib/vmware-vmafd/bin/dir-cli state get). Once we et it to NORMAL (/usr/lib/vmware-vmafd/bin/dir-cli state set --state NORMAL --server-name localhost) we were able to take a manual backup. I have to monitor the behavior for a bit but we think it is going to be fine. All caused by teh upgrade