Ramdisk /etc is full ?

[mzac23]

I've seen a lot of people having their /root ramdisk full, but I'm running across an issue that my /etc ramdisk is full.  I don't know what all the files are so I'm not sure what I need to delete.  Since it is full, hostd won't start so I can't get into the GUI of my esxi...

[root@massif:/etc] vmware -v

VMware ESXi 6.7.0 build-11675023

-----

Ramdisk                   Size      Used Available Use% Mounted on

root                       32M        2M       29M   7% --

etc                        28M       28M        0B 100% --

opt                        32M        0B       32M   0% --

var                        48M      436K       47M   0% --

tmp                       256M      112K      255M   0% --

iofilters                  32M        0B       32M   0% --

shm                      1024M        0B     1024M   0% --

hostdstats               1553M       10M     1542M   0% --

snmptraps                   1M        0B        1M   0% --

2019-02-19T01:26:57.269Z cpu19:2099427 opID=a2b69545)WARNING: VisorFSRam: 206: Cannot extend visorfs file /etc/vmware/esx.conf.LOCK.2099398 because its ramdisk (etc) is full.

2019-02-19T01:26:57.269Z cpu19:2099427 opID=a2b69545)MemSchedAdmit: 470: Admission failure in path: etc/etc

2019-02-19T01:26:57.269Z cpu19:2099427 opID=a2b69545)MemSchedAdmit: 477: etc (228) extraMin/extraFromParent: 1/1, etc (227) childEmin/eMinLimit: 7168/7168

2019-02-19T01:26:57.269Z cpu19:2099427 opID=a2b69545)WARNING: VisorFSRam: 206: Cannot extend visorfs file /etc/vmware/esx.conf.LOCK.2099398 because its ramdisk (etc) is full.

Any ideas how to fix this?

Thanks

[ThompsG]

Hi there and welcome!!

If you could PUTTY into your server and capture the output of this command when run from the /etc folder:

ls -RlhaS

This will give you a listing of all the files in this folder structure sorted by size. Hopefully something leaps off the screen at you then.

If you want, you could also capture the output and attach here as a file for people to peruse and maybe spot something for you as well!

Kind regards.

[mzac23]

Hi, please see the attached file of the listing of my /etc dir...

I thought it could be a swap issue and I noticed that swap is not on the datastore, and i cannot change it since /etc is full:

[root@massif:/etc] esxcli sched swap system get

   Datastore Active: false

   Datastore Enabled: false

   Datastore Name:

   Datastore Order: 0

   Hostcache Active: false

   Hostcache Enabled: true

   Hostcache Order: 1

   Hostlocalswap Active: false

   Hostlocalswap Enabled: true

   Hostlocalswap Order: 2

[root@massif:/etc] esxcli sched swap system set -n datastore0

Unable to set system swap configuration: /etc/vmware/esx.conf.tmp.2099398: write failed

[ThompsG · Accepted solution]

Hi there,

Looks like this ESXi host is added to an AD domain and the "Likewise" provider that VMware are using to provide this functionality is causing this:

./likewise/db:
total 28388
-rw-------    1 root     root       27.6M Feb 19 01:12 lsass-adcache.filedb.CAMPUS.MCGILL.CA.new
-r--r--r-T    1 root     root       60.0K Jan  8 03:13 .#registry.db
-rwx------    1 root     root       44.0K Feb 19 00:57 registry.db
-rw-------    1 root     root        3.1K Feb 19 01:03 lsass-adcache.filedb.CAMPUS.MCGILL.CA
drwx------    1 root     root         512 Feb 19 01:12 .
drwxr-xr-x    1 root     root         512 Feb 19 00:56 ..
-rw-r--r-T    1 root     root           0 Jan  8 03:13 pstore.filedb

The file "lsass-adcache.filedb.CAMPUS.MCGILL.CA.new" is 27.6MB in size and this will be consuming all the /etc space.

Your next question will be can I remove this file - this files seems to be an AD cache file so while I have not been able to find an official VMware document related to newer versions of ESXi however this article seems to imply the answer is yes:

https://kb.vmware.com/s/article/1026012?lang=en_US

Here is another implying the same thing: Not able to login to ESXi with Domain Account. Error: Cannot login to domain\user | IT Support Rocke...

After completing and restarting ESXi, just check that the ESXi server is still connected to AD. If not then reconnect.

Kind regards.

[mzac23]

Hi, I had also noticed this big file.  I have removed it and re-started hostd and now my system is back to normal.

This is a single ESXi managed by a vsphere in our lab and I had completely forgotten that I joined the ESXi server to AD.  I have no need for it to join the domain so I'll just leave that part disabled for now!

[ThompsG]

Haha - excellent work and glad you got it solved.

Have a great day!