VMware Cloud Community
jslarouche
Enthusiast
Enthusiast

Vcops /data 100% full getting HTTP Status 404 - error

Hi All,

Our vCops instances stopped working the other day and we are unable to purge the /data on the vCops analytics VM.. Does anyone have a process on clearing this /data folder? We do not want to add more disk to the VM's.. We logged a support call with VMware and here are his intructions.. We tried the following with no luck.. Any input would be great thanks.

_________________________________________________________________________

admin@s11vcop02:~> df -h

Filesystem Size  Used Avail Use% Mounted on

/dev/sda3 7.8G  1.9G  5.6G 25% /

udev 4.5G  100K  4.5G 1% /dev

tmpfs 4.5G     0  4.5G 0% /dev/shm

/dev/sda1 130M   18M  105M 15% /boot

/dev/mapper/data_vg-data

197G  187G     0 100% /data

The best way to accomplish this is to change your retention period to a lower number. Please see the guide here: http://kb.vmware.com/kb/2008223

Once this is done, it may take a couple cycles or the services to get this to run properly. I am not sure how long you have been collecting, or how much back-data you have. I would suggest putting this to 90 days if you do not need data older than 3 months.

Also it is best to delete items that are not collecting:

Adjust your controller.properties to auto-delete  non-collecting resources after 48 hours. The procedure to do this:

- SSH into the Analytics virtual machine as root.

- Run the following command:

- vim /usr/lib/vmware-vcops/user/conf/controller/controller.properties

- Modify the following line to state true instead of false:

- deleteNotExisting = true

- You can also modify the delete schedule. It is measured in hours with the following line:

- deletionSchedulePeriod = 24

- Modify the following line to the how many days a deleted object will be cleaned up (default is 15):

- deletionPeriodInHours = 48

- Save and quit from the editor.

- Login to the user interface virtual machine as the Admin user and run the following command:

- vcops-admin restart

-After this command has completed, you will need to wait the appropriate number of days for non-existent objects to get cleaned up.

0 Kudos
2 Replies
TammieD
Enthusiast
Enthusiast

When you did this, were you able to get the analytics service to restart?

Did you change the oldDataCleanerExecutionFrequency and daysoldDataCleanerDateRange in the analytics advanced.properties file?  If your oldDataCleanerExecutionFrequency is set to 7, for example, it will not do the data cleanup until 7 days after the last time you restarted the analytics service.  I needed to clean up data due to running out of space, so I changed this to 1, so that it would only wait one day.  I did not find anything that would tell it to clean up old data immediately.

I figured out the structure of the FSDB database, but I suspect manual deletion of files / folders is probably not recommended.

jslarouche
Enthusiast
Enthusiast

I ended up abandoning the original deployment and scrapped it.. It was easier to just re-deploy a new appliance to get us up and running again.. Looks like VMware has some work ahead of them to get the product dialled in a little more.. Pretty painful..

0 Kudos