5 Replies Latest reply on Feb 19, 2019 5:38 PM by ThompsG

    Ramdisk /etc is full ?

    mzac23 Lurker

      I've seen a lot of people having their /root ramdisk full, but I'm running across an issue that my /etc ramdisk is full.  I don't know what all the files are so I'm not sure what I need to delete.  Since it is full, hostd won't start so I can't get into the GUI of my esxi...

       

      [root@massif:/etc] vmware -v

      VMware ESXi 6.7.0 build-11675023

       

      -----

      Ramdisk                   Size      Used Available Use% Mounted on

      root                       32M        2M       29M   7% --

      etc                        28M       28M        0B 100% --

      opt                        32M        0B       32M   0% --

      var                        48M      436K       47M   0% --

      tmp                       256M      112K      255M   0% --

      iofilters                  32M        0B       32M   0% --

      shm                      1024M        0B     1024M   0% --

      hostdstats               1553M       10M     1542M   0% --

      snmptraps                   1M        0B        1M   0% --

       

      2019-02-19T01:26:57.269Z cpu19:2099427 opID=a2b69545)WARNING: VisorFSRam: 206: Cannot extend visorfs file /etc/vmware/esx.conf.LOCK.2099398 because its ramdisk (etc) is full.

      2019-02-19T01:26:57.269Z cpu19:2099427 opID=a2b69545)MemSchedAdmit: 470: Admission failure in path: etc/etc

      2019-02-19T01:26:57.269Z cpu19:2099427 opID=a2b69545)MemSchedAdmit: 477: etc (228) extraMin/extraFromParent: 1/1, etc (227) childEmin/eMinLimit: 7168/7168

      2019-02-19T01:26:57.269Z cpu19:2099427 opID=a2b69545)WARNING: VisorFSRam: 206: Cannot extend visorfs file /etc/vmware/esx.conf.LOCK.2099398 because its ramdisk (etc) is full.

       

      Any ideas how to fix this?

       

      Thanks

        • 1. Re: Ramdisk /etc is full ?
          ThompsG Master

          Hi there and welcome!!

           

          If you could PUTTY into your server and capture the output of this command when run from the /etc folder:

           

          ls -RlhaS
          

           

          This will give you a listing of all the files in this folder structure sorted by size. Hopefully something leaps off the screen at you then.

           

          If you want, you could also capture the output and attach here as a file for people to peruse and maybe spot something for you as well!

           

          Kind regards.

          • 2. Re: Ramdisk /etc is full ?
            mzac23 Lurker

            Hi, please see the attached file of the listing of my /etc dir...

             

            I thought it could be a swap issue and I noticed that swap is not on the datastore, and i cannot change it since /etc is full:

             

            [root@massif:/etc] esxcli sched swap system get

               Datastore Active: false

               Datastore Enabled: false

               Datastore Name:

               Datastore Order: 0

               Hostcache Active: false

               Hostcache Enabled: true

               Hostcache Order: 1

               Hostlocalswap Active: false

               Hostlocalswap Enabled: true

               Hostlocalswap Order: 2

             

            [root@massif:/etc] esxcli sched swap system set -n datastore0

            Unable to set system swap configuration: /etc/vmware/esx.conf.tmp.2099398: write failed

            • 3. Re: Ramdisk /etc is full ?
              ThompsG Master

              Hi there,

               

              Looks like this ESXi host is added to an AD domain and the "Likewise" provider that VMware are using to provide this functionality is causing this:

               

              ./likewise/db:
              total 28388
              -rw-------    1 root     root       27.6M Feb 19 01:12 lsass-adcache.filedb.CAMPUS.MCGILL.CA.new
              -r--r--r-T    1 root     root       60.0K Jan  8 03:13 .#registry.db
              -rwx------    1 root     root       44.0K Feb 19 00:57 registry.db
              -rw-------    1 root     root        3.1K Feb 19 01:03 lsass-adcache.filedb.CAMPUS.MCGILL.CA
              drwx------    1 root     root         512 Feb 19 01:12 .
              drwxr-xr-x    1 root     root         512 Feb 19 00:56 ..
              -rw-r--r-T    1 root     root           0 Jan  8 03:13 pstore.filedb
              

               

              The file "lsass-adcache.filedb.CAMPUS.MCGILL.CA.new" is 27.6MB in size and this will be consuming all the /etc space.

               

              Your next question will be can I remove this file - this files seems to be an AD cache file so while I have not been able to find an official VMware document related to newer versions of ESXi however this article seems to imply the answer is yes:

               

              https://kb.vmware.com/s/article/1026012?lang=en_US

               

              Here is another implying the same thing: Not able to login to ESXi with Domain Account. Error: Cannot login to domain\user | IT Support Rocket

               

              After completing and restarting ESXi, just check that the ESXi server is still connected to AD. If not then reconnect.

               

              Kind regards.

              • 4. Re: Ramdisk /etc is full ?
                mzac23 Lurker

                Hi, I had also noticed this big file.  I have removed it and re-started hostd and now my system is back to normal.

                 

                This is a single ESXi managed by a vsphere in our lab and I had completely forgotten that I joined the ESXi server to AD.  I have no need for it to join the domain so I'll just leave that part disabled for now!

                • 5. Re: Ramdisk /etc is full ?
                  ThompsG Master

                  Haha - excellent work and glad you got it solved.

                   

                  Have a great day!