Hi All,
Quick background, VDP stopped working around a week ago, restarted and it started up again, then stopped again next day, same again. Got looking at logs and saw a load of disk full errors.
Ran df -u and found the below
df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 7.9G 7.9G 0 100% /
udev 3.9G 180K 3.9G 1% /dev
tmpfs 3.9G 0 3.9G 0% /dev/shm
/dev/sda1 130M 38M 86M 31% /boot
/dev/sda7 1.5G 571M 825M 41% /var
/dev/sda9 77G 57G 16G 79% /space
/dev/sdb1 1.0T 112G 912G 11% /data01
/dev/sde1 1.0T 113G 911G 12% /data02
/dev/sdf1 1.0T 111G 913G 11% /data03
/dev/sdg1 1.0T 111G 913G 11% /data04
/dev/sdh1 1.0T 109G 916G 11% /data05
/dev/sdi1 1.0T 110G 914G 11% /data06
So I read some articles etc... they all pointed towards proxy logs in /usr/local/avarmaclient/ however I saw no such files there, I carried on found some large tomcat logs in /usr/local/avamar-tomcat-7.0.54/webapps/ROOT/logBundle/
I moved (to /space) 1 to clear 2.5g and then restarted emwebapp.sh and was showing as 60% fullish. Back ups worked fine the next day but then died the day after, checked the disk and full again!
So I moved the other file and cleared 3g this time, restarted everything and it worked fine the night.
Came in today and guess what, full again! Checked the /usr/local/avamar-tomcat-7.0.54/webapps/ROOT/logBundle/ location and it went from 1mb of log files to 3gb over night!
Checked some log files and seeing a lot of these errors -
org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/vdp-configure] appears to have started a thread named [pool-12-thread-2] but has failed to stop it. This is very likely to create a memory leak.
Can anyone help with pinpointing why the logs are filling up so quickly?
When I run into space issues in the system volumes I follow the instructions on the 6.0 admin guide on how to free up space before upgrade (page 37). Although enough changed in VDP 6.1 that the instructions don't really work for that version. 6.1 also as twice the space for the sytem vmdk so your less likely to have root fill up.
Instructions includes stopping enwebapp.sh and etc/init.d/apache2 before deleting log files. You only mention enwebapp.sh. Possibly thats related to your issue. Or you just need to delete the other logs mentioned in the admin guide.
Thanks for the suggestion, I followed the guide but removed 0% space.
I have gotten one of the 3g tomcat files off the server, had to repair it as it was corrupt, but looking at it, it just looks like a copy of load of data, including the /var /usr and /data partitions.
Seems a bit weird to copy stuff off.