ESXi 5 and The ramdisk 'root' is full

[lifeisepic]

Hi,

I'm new to ESXi and have recently installed version 5 onto a server. Everything seems to be working OK, but the "The ramdisk 'root' is full" message is appearing frequently and I don't know why.

We store all our VMs on a NAS leaving the local disk purely for the ESXi server.

The ramdisk 'root' is full.  As a result, the file /var
/log/ipmi/0/.sensor_threshold.raw could not be
written.
error
26/04/2012 01:10:13
localhost.SSG-140

This is just one example, there are a lot but mention different files that can't be written.

Space seems OK.

Filesystem   Size   Used Available Use% Mounted on
NFS          1.8T  93.2G      1.7T   5% /vmfs/volumes/VM_NAS
VMFS-5     131.8G 971.0M    130.8G   1% /vmfs/volumes/local_disk
vfat         4.0G  11.4M      4.0G   0% /vmfs/volumes/4f291fe9-1a755017-25db-00237da16c8c
vfat       249.7M 127.4M    122.3M  51% /vmfs/volumes/d3b88a7e-71755e09-586a-2610ee101e71
vfat       249.7M   8.0K    249.7M   0% /vmfs/volumes/8fe72d7f-abce1db1-539b-c6d47e9f8369
vfat       285.8M 176.2M    109.6M  62% /vmfs/volumes/4f291fe2-fb7e77ce-d9e1-00237da16c8c

Any ideas?

[VTsukanov]

Take a look at topic The ramdisk 'root' is full.

This is not your case?

[lifeisepic]

I had a look at that link but couldn't find any thing to suggest the file system is full.

-rw-r--r--    1 root     root               3840 Apr 26 11:21 wtmp

[VTsukanov]

Based on output df command, you have enough free space.

Try to investigate other reasons for ipmi error

[lifeisepic]

What would you recommend I look into?

[VTsukanov]

try to execute over ssh

cd /var/log/ipmi/0

and

ls -la

check permissions for files & directories (should be 755 for directory 0, owner root, for files owner root)

try to delete

rm .sensor_threshold.raw 

after this restart

/etc/init.d/sfcbd-watchdog restart 

[lifeisepic]

Permissions appear to be 644.

/var/log/ipmi/0 # ls -l
-rw-r--r--    1 root     root                  0 Apr 27 11:33 fru
-rw-r--r--    1 root     root               2867 Apr 27 11:33 sdr_content.raw
-rw-r--r--    1 root     root                  0 Apr 27 11:33 sdr_header.raw
-rw-r--r--    1 root     root                 35 Apr 27 11:52 sel_header.raw
-rw-r--r--    1 root     root                  0 Apr 27 11:52 sensor_hysteresis.raw
-rw-r--r--    1 root     root                400 Apr 27 11:52 sensor_readings.raw
-rw-r--r--    1 root     root                  0 Apr 27 11:52 sensor_threshold.raw

Ownership looks fine and sensor_readings.raw has been wrote to recently.

I removed the file and ran the restart command.

[VTsukanov]

> Permissions appear to be 644.

for files in directory /var/log/ipmi/0 it looks as normal

wait for the attempts to write to a file /log/ipmi/0/.sensor_threshold.raw

[lifeisepic]

The message is wide spread. Here are some more messages.

The ramdisk 'root' is full.  As a result, the file /var
/log/ipmi/0/.sel_header.raw could not be written.
error
27/04/2012 12:57:34
localhost.SSG-140

The ramdisk 'root' is full.  As a result, the file /var
/log/ipmi/0/.sdr_content.raw could not be written
.
error
27/04/2012 12:57:34
localhost.SSG-140

The ramdisk 'root' is full.  As a result, the file /var
/log/ipmi/0/.sensor_threshold.raw could not be
written.
error
27/04/2012 12:52:22
localhost.SSG-140

The ramdisk 'root' is full.  As a result, the file /var
/log/ipmi/0/.sel_header.raw could not be written.
error
27/04/2012 12:51:34
localhost.SSG-140

The ramdisk 'root' is full.  As a result, the file /var
/log/ipmi/0/.sensor_threshold.raw could not be
written.
error
27/04/2012 12:43:50
localhost.SSG-140

The ramdisk 'root' is full.  As a result, the file /var
/run/vmware/tickets/vmtck-52607fec-c504-dd
could not be written.
error
27/04/2012 12:43:33
localhost.SSG-140

[lifeisepic]

I think I've cracked it.

When we were debugging and internet connection problem someone downloaded a test file to the root folder.

-rw-r--r--    1 root     root           33177600 Apr 24 14:54 ubuntu-12.04-beta2-desktop-i386.iso

I removed this file from the root directory and haven't seen the message in a while. Will keep an eye on it.

[lifeisepic]

Issue was caused by large file in / directory.

[gajuambi]

If you are using HP Servers then please use this to resolve the issue

http://ambitech.blogspot.com/2013/04/ram-disk-full-for-esxi-on-hp-servers.html

To avoid filling the RAM disk, AMS is now creating its log file  “hpHelper.log” in the configured scratch location (default of  /scratch/log directory).  In addition, a soft link to the log file is  created in the path /var/log.  The hpHelper.log file may grow to a  maximum size of 10MB and a file rotation algorithm has been implemented  with support of up to 10 archived hpHelper.log files.