1 2 Previous Next 24 Replies Latest reply on Aug 27, 2018 10:51 AM by tasikas

    Vmware Esxi Update No Space Left On Device Error

    Mustibh Novice

      Hello,

      I'm trying to install updates from a zip file but always gives the same error. "No Space Left On Device"

      But there is a plenty of space on Device.

      When i try to run

      esxcli software vib update -d /vmfs/volumes/datastore1/ESXi600-201703001.zip

      it gives that error:

      [InstallationError]

      [Errno 28] No space left on device

             vibs = VMware_bootbank_misc-drivers_6.0.0-3.57.5050593

      Please refer to the log file for more details.

       

      And in the log file:

      2017-04-02T00:13:50Z esxupdate: vmware.runcommand: INFO: runcommand called with: args = 'localcli system visorfs ramdisk list |            grep /stagebootbank &&            localcli system visorfs ramdisk remove -t /tmp/stagebootbank', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

      2017-04-02T00:13:50Z esxupdate: BootBankInstaller.pyc: WARNING: Cannot remove staging directory: [Errno 16] Device or resource busy: '/tmp/stagebootbank'

      2017-04-02T00:13:50Z esxupdate: vmware.runcommand: INFO: runcommand called with: args = 'localcli system visorfs ramdisk list |            grep /stagebootbank &&            localcli system visorfs ramdisk remove -t /tmp/stagebootbank', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

      2017-04-02T00:13:51Z esxupdate: root: ERROR: Traceback (most recent call last):

      2017-04-02T00:13:51Z esxupdate: root: ERROR:   File "/usr/lib/vmware/esxcli-software", line 462, in <module>

      2017-04-02T00:13:51Z esxupdate: root: ERROR:     main()

      2017-04-02T00:13:51Z esxupdate: root: ERROR:   File "/usr/lib/vmware/esxcli-software", line 453, in main

      2017-04-02T00:13:51Z esxupdate: root: ERROR:     ret = CMDTABLE[command](options)

      2017-04-02T00:13:51Z esxupdate: root: ERROR:   File "/usr/lib/vmware/esxcli-software", line 362, in VibInstallCmd

      2017-04-02T00:13:51Z esxupdate: root: ERROR:     checkacceptance=checkacceptance)

      2017-04-02T00:13:51Z esxupdate: root: ERROR:   File "/build/mts/release/bora-3620759/bora/build/esx/release/vmvisor/sys-boot/lib/python2.7/site-packages/vmware/esximage/Transaction.py", line 250, in InstallVibsFromSources

      2017-04-02T00:13:51Z esxupdate: root: ERROR:   File "/build/mts/release/bora-3620759/bora/build/esx/release/vmvisor/sys-boot/lib/python2.7/site-packages/vmware/esximage/Transaction.py", line 356, in _installVibs

      2017-04-02T00:13:51Z esxupdate: root: ERROR:   File "/build/mts/release/bora-3620759/bora/build/esx/release/vmvisor/sys-boot/lib/python2.7/site-packages/vmware/esximage/Transaction.py", line 399, in _validateAndInstallProfile

      2017-04-02T00:13:51Z esxupdate: root: ERROR:   File "/build/mts/release/bora-3620759/bora/build/esx/release/vmvisor/sys-boot/lib/python2.7/site-packages/vmware/esximage/HostImage.py", line 710, in Stage

      2017-04-02T00:13:51Z esxupdate: root: ERROR:   File "/build/mts/release/bora-3620759/bora/build/esx/release/vmvisor/sys-boot/lib/python2.7/site-packages/vmware/esximage/HostImage.py", line 483, in _download_and_stage

      2017-04-02T00:13:51Z esxupdate: root: ERROR: InstallationError: ('VMware_bootbank_vsanhealth_6.0.0-3000000.3.0.3.58.5224738', '[Errno 28] No space left on device')

       

      "vdf -h" Output

      Ramdisk                   Size      Used Available Use% Mounted on

      root                       32M      244K       31M   0% --

      etc                        28M      204K       27M   0% --

      opt                        32M        0B       32M   0% --

      var                        48M      428K       47M   0% --

      tmp                       256M       20K      255M   0% --

      iofilters                  32M        0B       32M   0% --

      hostdstats                303M        2M      300M   0% --

      As you can see %0 is used on /tmp i dont know why gives that error.

       

      How can i fix this problem ? I cant update my esxi 3620759 to 5224934.

        • 2. Re: Vmware Esxi Update No Space Left On Device Error
          Mustibh Novice

          Thanks for your reply,

          I tried what said in this KB but it did not helped. All it shows these files.

          [root@myserver:/tmp] find / -path "/vmfs" -prune -o -type f -size +50000k -exec ls -lh '{}' \;

          -r--r--r--    1 root     root      139.5M Apr  1 23:53 /tardisks/sb.v00

          -r--r--r--    1 root     root      315.7M Apr  1 23:53 /tardisks/s.v00

          -rw-------    1 root     root      439.0G Apr  2 08:38 /dev/lvm/5878b981-2bb79012-ec3d-0cc47ac67b50

          -rw-------    1 root     root        1.8T Apr  2 08:38 /dev/lvm/587641e3-53d93890-7ae2-0cc47ac67b50

          -rw-------    1 root     root        1.8T Apr  2 08:38 /dev/disks/naa.600605b000a029f02008fd5307f25822

          -rw-------    1 root     root        1.8T Apr  2 08:38 /dev/disks/naa.600605b000a029f02008fd5307f25822:1

          -rw-------    1 root     root      446.6G Apr  2 08:38 /dev/disks/naa.600605b000a029f02000e7f407fec1fb

          -rw-------    1 root     root        2.5G Apr  2 08:38 /dev/disks/naa.600605b000a029f02000e7f407fec1fb:9

          -rw-------    1 root     root      286.0M Apr  2 08:38 /dev/disks/naa.600605b000a029f02000e7f407fec1fb:8

          -rw-------    1 root     root      110.0M Apr  2 08:38 /dev/disks/naa.600605b000a029f02000e7f407fec1fb:7

          -rw-------    1 root     root      250.0M Apr  2 08:38 /dev/disks/naa.600605b000a029f02000e7f407fec1fb:6

          -rw-------    1 root     root      250.0M Apr  2 08:38 /dev/disks/naa.600605b000a029f02000e7f407fec1fb:5

          -rw-------    1 root     root      439.2G Apr  2 08:38 /dev/disks/naa.600605b000a029f02000e7f407fec1fb:3

          -rw-------    1 root     root        4.0G Apr  2 08:38 /dev/disks/naa.600605b000a029f02000e7f407fec1fb:2

          Here is the stat -f output too;

           

          [root@myserver:/tmp] stat -f /

            File: "/"

              ID: 100000000 Namelen: 127     Type: visorfs

          Block size: 4096

          Blocks: Total: 398288     Free: 249397     Available: 249397

          Inodes: Total: 524288     Free: 518408

          Please help how can i fix this problem ?

          • 3. Re: Vmware Esxi Update No Space Left On Device Error
            a.p. Guru
            User ModeratorsvExpertCommunity Warriors

            To rule out issues with the downloaded "ESXi600-201703001.zip", please ensure that the .zip file isn't corrupted by verifying it's checksum.

             

            André

            • 4. Re: Vmware Esxi Update No Space Left On Device Error
              Mustibh Novice

              Thanks for your reply,

              I already checked the file but it does not seems with the file issue.

              Sometimes on log shows "cannot allocate memory" and i event cant write ls -la command always says '-sh cant fork'

               

              I didnt understand whats going on with this server :/

              • 5. Re: Vmware Esxi Update No Space Left On Device Error
                Mustibh Novice

                Nobody knows any fix about this ?

                Problem still exists

                • 6. Re: Vmware Esxi Update No Space Left On Device Error
                  Mustibh Novice

                  Problem still exists ? No one had this problem ?

                  • 7. Re: Vmware Esxi Update No Space Left On Device Error
                    vijayrana968 Master
                    vExpert

                    Did you check space on the /locker directory ? Have a look on this : ESXi 6 Update error – No Space left on device /locker » vPRH

                    • 8. Re: Vmware Esxi Update No Space Left On Device Error
                      YaZi Lurker

                      I have the same issue on 2 different ESXI servers, both running ESXi-6.5.0-20170104001-standard.

                       

                      Have you made any progress? Like you I have tried pretty much all KB's that seems related and made no progress.

                       

                       

                      File: "/"

                      ID: 100000000 Namelen: 127     Type: visorfs

                      Block size: 4096     

                      Blocks: Total: 356432     Free: 164084     Available: 164084

                      Inodes: Total: 655360     Free: 648922

                      Ramdisk                    Size           Used      Available      Use% Mounted on

                      root                            32M            24K       31M             0% --

                      etc                             28M           164K       27M             0% --

                      opt                             32M             0B       32M             0% --

                      var                             48M           320K       47M             0% --

                      tmp                            256M             8K      255M             0% --

                      iofilters                       32M             0B       32M                  0% --

                      hostdstats                 112M             1M      110M             1% --

                      stagebootbank          250M           146M      103M            58% --

                      Filesystem        Size        Used           Available      Use%      Mounted on

                      VMFS-5          225.2G      158.9G          66.3G       71%      /vmfs/volumes/datastore1

                      vfat                 285.8M      128.0K         285.7M        0%      /vmfs/volumes/58de5208-936557fc-18e3-f44d30665179

                      vfat                   4.0G         640.0K           4.0G           0%      /vmfs/volumes/58de520e-00e0262a-3724-f44d30665179

                      vfat                 249.7M      143.7M         106.0M       58%      /vmfs/volumes/8ce13377-7f367bc5-634a-9b0ea248875f

                      vfat                 249.7M      146.5M         103.2M       59%      /vmfs/volumes/b2a84977-623ad144-c31a-02c96d03d2ed

                      • 9. Re: Vmware Esxi Update No Space Left On Device Error
                        msripada Expert
                        vExpert

                        If it is cannot allocate memory, then can you try restarting the ESXi host after keeping it in maintenance mode and then try to update it after reboot.

                         

                        Thanks,

                        MS

                        • 10. Re: Vmware Esxi Update No Space Left On Device Error
                          YaZi Lurker

                          Thanks, tried your suggestion but still get the same ...

                           

                          [InstallationError]

                          [Errno 28] No space left on device

                                 vibs = VMware_locker_tools-light_6.5.0-0.23.5969300

                          Tried using script from here, worked for previous updates perfectly, but not this time.

                           

                          https://esxi-patches.v-front.de/ESXi-6.5.0.html

                           

                          esxcli network firewall ruleset set -e true -r httpClient

                          esxcli software profile update -p ESXi-6.5.0-20170701001s-standard \

                          -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

                          esxcli network firewall ruleset set -e false -r httpClient

                          Not sure what else to check or try?

                          • 11. Re: Vmware Esxi Update No Space Left On Device Error
                            FirstS0ul Novice

                            Hey!

                             

                            i have the same issue and tested all the suggestions...

                            Any update?

                             

                            UPDATE:

                            I've solution!

                             

                            1) Install the locker vib manually:

                            cd /tmp

                            wget http://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/tools-light/VMware_locker_tools-light_6.5.0-0.23.5969300.vib

                            esxcli software vib install -f -v /tmp/VMware_locker_tools-light_6.5.0-0.23.5969300.vib

                             

                            2)

                            Update:

                            esxcli software profile update -f -p ESXi-6.5.0-20170702001-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-

                            depot-index.xml

                             

                            Reboot and youre done :-)

                            3 people found this helpful
                            • 12. Re: Vmware Esxi Update No Space Left On Device Error
                              FreeAgent Novice

                              @FirstS0ul

                              Thanks for your solution.  We tried this but encountered memory errors. The solution was to enter the host webui and Go to Host > System > Swap and activate swap on our datastore vmfs. Once the setting was enabled the esxcli software profile update command worked normally.

                              Thanks for posting very helpful.

                              11 people found this helpful
                              • 14. Re: Vmware Esxi Update No Space Left On Device Error
                                YaZi Lurker

                                Thanks, this worked for me also.


                                I tested just enabling SWAP and skipping all the other steps, which also worked, so I guess only that is needed.

                                1 2 Previous Next