1 2 3 Previous Next 39 Replies Latest reply on Sep 26, 2019 12:08 PM by stephenyu05 Go to original post
      • 30. Re: Vmware Esxi Update No Space Left On Device Error
        pbraren Hot Shot
        VMware EmployeesvExpert

        Wow, harrypnyce, I really appreciate the kind words, thank you!

        • 31. Re: Vmware Esxi Update No Space Left On Device Error
          harrypnyce Novice

          Finally resolved this after taking a (somewhat lengthy) break and managed to find some time this afternoon to poke around at it again, and appears all is now resolved.  Was running ESXi host, version: 6.7.0 Update 1 (Build 11675023) -- update to v6.7 U2 fails with the following error message:

          [OSError]

          [Errno 28] No space left on device

          Please refer to the log file for more details.

           

          Unfortunately adjustments to the swap file / location didn't seem to resolve my issue(s), even after placing the host in Maintenance Mode and even rebooting between adjustments.  I could see the swap file moving from one datastore to the next as I tried both SSDs and a larger 1TB spinning rust with gobs of free space, but ultimately the fix was to manually apply the specific VIB in question, at which point I was able to rerun the original update command and things seem to have been applied successfully.  Waiting on the host to come back up now.

           

          > [root@esxi2:/tmp] wget http://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/tools-light/VMware_locker_tools-light_10.3.5.10430147-12986307.vib

          Connecting to hostupdate.vmware.com (184.29.84.35:80)

          VMware_locker_tools- 100% |*****************************************************************************************************************************************************|   153M  0:00:00 ETA

           

          > [root@esxi2:/tmp] esxcli software vib install -f -v /tmp/VMware_locker_tools-light_10.3.5.10430147-12986307.vib

          Installation Result

             Message: Operation finished successfully.

             Reboot Required: false

             VIBs Installed: VMware_locker_tools-light_10.3.5.10430147-12986307

             VIBs Removed: VMware_locker_tools-light_10.3.2.9925305-10302608

             VIBs Skipped:

           

          And here is where I re-ran the original update command, which applied successfully:
          > [root@esxi2:/tmp] esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p  ESXi-6.7.0-20190504001-standard

          Update Result

             Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.

             Reboot Required: true

           

          After failing with moving swap file(s) around to various datastores and checking inode values & free space, eventually my error message morphed to display the following, at which point (see above) i manually installed the mentioned VIB:

          [InstallationError]

          [Errno 28] No space left on device

                 vibs = VMware_locker_tools-light_10.3.5.10430147-12986307

          Please refer to the log file for more details.

           

          Voila, now on: v6.7.0 Update 2 (Build 13644319)

           

          As always, your site is an immensely valuable resource, Mr. Braren, sir!~  My apologies for the lack of chronology contained within this post.  Hope someone else is spared the research efforts once they've stumbled upon thi

          (cc: pbraren)

          1 person found this helpful
          • 32. Re: Vmware Esxi Update No Space Left On Device Error
            Neutro Novice

            Thank you i had the same problem and your solution worked perfectly

            • 33. Re: Vmware Esxi Update No Space Left On Device Error
              psed Novice

              Hi, the same problem

               

              update vmware 6.7up1 -> up2

               

              [root@olvmware01:/tmp] esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p  ESXi-6.7.0-20190504001-standard

              [OSError]

              [Errno 28] No space left on device

              Please refer to the log file for more details.

               

               

              [root@olvmware01:/tmp] df -h

              Filesystem   Size   Used Available Use% Mounted on

              VMFS-6     924.0G  13.3G    910.7G   1% /vmfs/volumes/datastore01_SATA_1TB_Seagate

              VMFS-6     931.2G 701.4G    229.8G  75% /vmfs/volumes/datastore02_SATA_1TB_Seagate

              vfat       285.8M 172.9M    112.9M  60% /vmfs/volumes/5c6bd88a-81d56d1c-4eed-4c5262099845

              vfat       249.7M 146.7M    103.0M  59% /vmfs/volumes/970d7e7f-244c6e5c-de39-3887b52dde70

              vfat       249.7M 146.7M    103.0M  59% /vmfs/volumes/21f8f578-b9d58c62-3141-5767f4693b98

              vfat         4.0G  17.2M      4.0G   0% /vmfs/volumes/5c6bd892-9a69c4fe-0692-4c5262099845

               

              Navigator > Host > Manage > System > Swap > Edit Settings >

              Enable - Yes

              Datastore - "datastore01_SATA_1TB_Seagate"

              Local swap - Yes

              Host Cache -Yes

              +reboot

               

              [root@olvmware01:/tmp] tail -30 /var/log/esxupdate.log

              2019-07-03T13:06:55Z esxupdate: 2099827: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/esxcfg-advcfg', '-U', 'host-acceptance-level', '-G']', outfile = 'None', returnoutput = 'True', timeout = '30.0'.

              2019-07-03T13:06:55Z esxupdate: 2099827: Transaction: INFO: Downloading esx-update vib from https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/esx-update/VMware_bootbank_esx-update_6.7.0-2.55.13644319.vib

              2019-07-03T13:06:55Z esxupdate: 2099827: vmware.runcommand: INFO: runcommand called with: args = 'localcli system visorfs ramdisk add -m 0 -M 20 -n esx-update-2099827 -p 755 -t /tmp/esx-update-2099827', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

              2019-07-03T13:06:55Z esxupdate: 2099827: downloader: INFO: Downloading https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/esx-update/VMware_bootbank_esx-update_6.7.0-2.55.13644319.vib to /tmp/esx-update-2099827/VMware_bootbank_esx-update_6.7.0-2.55.13644319

              2019-07-03T13:06:56Z esxupdate: 2099827: root: DEBUG: Verifying VIB VMware_bootbank_esx-update_6.7.0-2.55.13644319 signature #2

              2019-07-03T13:06:56Z esxupdate: 2099827: Ramdisk: INFO: Mount tardisk /tmp/esx-update-2099827/esxupdt-2099827 to /tardisks.noauto/

              2019-07-03T13:06:56Z esxupdate: 2099827: Ramdisk: INFO: Attach tardisk esxupdt-2099827 to ramdisk esx-update-2099827

              2019-07-03T13:06:56Z esxupdate: 2099827: Ramdisk: INFO: Mount tardisk /tmp/esx-update-2099827/weaselin-2099827 to /tardisks.noauto/

              2019-07-03T13:06:56Z esxupdate: 2099827: Ramdisk: INFO: Attach tardisk weaselin-2099827 to ramdisk esx-update-2099827

              2019-07-03T13:06:56Z esxupdate: 2099827: Transaction: DEBUG: Added /tmp/esx-update-2099827/usr/lib/vmware and /tmp/esx-update-2099827/lib64/python3.5/site-packages/vmware to sys.path

              2019-07-03T13:06:56Z esxupdate: 2099827: vmware.runcommand: INFO: runcommand called with: args = 'localcli system visorfs ramdisk list | grep /esx-update-2099827 && localcli system visorfs ramdisk remove -t /tmp/esx-update-2099827', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

              2019-07-03T13:06:56Z esxupdate: 2099827: Ramdisk: INFO: Unmounting manual tardisk /tardisks.noauto/esxupdt-2099827

              2019-07-03T13:06:56Z esxupdate: 2099827: Ramdisk: INFO: Unmounting manual tardisk /tardisks.noauto/weaselin-2099827

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR: Traceback (most recent call last):

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:   File "/usr/lib/vmware/esxcli-software", line 470, in <module>

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:     main()

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:   File "/usr/lib/vmware/esxcli-software", line 461, in main

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:     ret = CMDTABLE[command](options)

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:   File "/usr/lib/vmware/esxcli-software", line 213, in ProfileUpdateCmd

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:     nohwwarning=opts.nohwwarning)

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:   File "/build/mts/release/bora-10764712/bora/build/esx/release/vmvisor/esxupdate/lib64/python3.5/site-packages/vmware/esximage/Transaction.py", line 375, in UpdateProfileFromDepot

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:   File "/tmp/esx-update-2099827/usr/lib/vmware/weasel/util/upgrade_precheck.py", line 2161, in cliUpgradeAction

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:   File "/tmp/esx-update-2099827/usr/lib/vmware/weasel/util/upgrade_precheck.py", line 997, in _parseVmwareVersion

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:   File "/build/mts/release/bora-10302608/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 514, in getoutput

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:   File "/build/mts/release/bora-10302608/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 495, in getstatusoutput

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:   File "/build/mts/release/bora-10302608/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 316, in check_output

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:   File "/build/mts/release/bora-10302608/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 383, in run

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:   File "/build/mts/release/bora-10302608/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 676, in __init__

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR:   File "/build/mts/release/bora-10302608/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 1228, in _execute_child

              2019-07-03T13:06:56Z esxupdate: 2099827: root: ERROR: OSError: [Errno 28] No space left on device

               

               

              [root@olvmware01:/tmp] stat -f /

                File: "/"

                  ID: 100000000 Namelen: 127     Type: visorfs

              Block size: 4096

              Blocks: Total: 580901     Free: 383584     Available: 383584

              Inodes: Total: 655360     Free: 648814

               

              Install VMware_locker_tools-ligh ok

              [root@olvmware01:/tmp] esxcli software vib install -f -v /tmp/VMware_locker_tools-light_10.3.5.10430147-12986307.vib

              Installation Result

                 Message: Operation finished successfully.

                 Reboot Required: false

                 VIBs Installed: VMware_locker_tools-light_10.3.5.10430147-12986307

                 VIBs Removed: VMware_locker_tools-light_10.3.2.9925305-10176879

                 VIBs Skipped:

               

              How can i fix this problem ?

              Thank you
              Pavel

              1 person found this helpful
              • 34. Re: Vmware Esxi Update No Space Left On Device Error
                ivanerben Enthusiast

                Same here, unable to update. And hitting bug with OCFlush cpu load

                High CPU usage by system process after 49 days of uptime (OCFlush)

                high OCFlush %USED with no VMs running. Esxtop %USED sort not working once in world 

                which is maybe solved...

                 

                [root@esxi:~] esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-6.7.0-20190604001-standard

                [OSError]

                [Errno 28] No space left on device

                Please refer to the log file for more details.

                 

                VMware ESXi 6.7.0 build-10302608

                VMware ESXi 6.7.0 Update 1

                • 35. Re: Vmware Esxi Update No Space Left On Device Error
                  GeraldVM Lurker

                  Very frustrating. I just spent more than an hour trying all kinds of ways to update a fresh install of

                  "VMware-VMvisor-Installer-6.7.0.update02-13006603.x86_64.iso"

                  to the latest version using

                  "esxcli software profile update -p ESXi-6.7.0-20190604001-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml"

                   

                  and it always failed with: [Errno 28] No space left on device

                   

                  I tried everything, except manually installing the VIB. Nothing which usually helps (setting swap space, changing the locker directory, removing files from the locker directory, installing the -no-tools instead of standard profile) helped. I might have been able to fix this by installing the vib manually but...

                   

                  I just wiped the machine and installed VMware ESXi 6.5u2 (VMware-VMvisor-Installer-6.5.0.update02-8294253.x86_64.iso), and the update to ESXi-6.7.0-20190604001-standard just worked without any hassle. (I did set the swap space before trying, but did not even reboot after doing so. It just worked)

                   

                  Something is seriously wrong with the 6.7u2 installation media. Guess quality control was too expensive and so is fixing the bug.

                  • 36. Re: Vmware Esxi Update No Space Left On Device Error
                    Neutro Novice

                    And you're bitching about a free software.

                     

                    If you were using the paid version of ESXi you would have updated through vcenter without having to update through SSH.

                     

                    Not saying that you should pay for it but since most VMware users are entreprises using Vcenter only a few people actually have to update through SSH like you and me. So it's understandable that since only a few people use it, when bugs are present they are detected and fixed less faster.

                     

                    In any case, crying like you do wont change anything... Maybe it could make VMware stop giving free licences because too much people like you complain after.

                    • 37. Re: Vmware Esxi Update No Space Left On Device Error
                      GeraldVM Lurker

                      >And you're bitching about a free software.

                       

                      This will come as a shock to you, but VMware is not giving away the free version out of the goodness in their hearts. I'm sorry to burst your bubble.

                      This is a business decision, where giving away a somewhat limited edition draws people into their ecosystem and then upsells them into a paid version once they eventually require a feature beyond what is offered in the free version.

                       

                      What does make sense in this context (contrary to what you are thinking) is making the free version bad and buggy.

                      Users of VMware which are not yet heavily invested will not say

                      "Oh, the update is not working AGAIN, I guess everything will be fine in the paid version."

                      instead they will say

                      "Oh, the update is not working AGAIN, I guess I need to have a closer look at Proxmox."

                      • 38. Re: Vmware Esxi Update No Space Left On Device Error
                        tymchenko Lurker

                        Thanks it helps me update 6.7 U2 -> 6.7 U3

                        • 39. Re: Vmware Esxi Update No Space Left On Device Error
                          stephenyu05 Novice

                          i cannot upgrade from 6.7u2 to u3/

                          no space left

                          1 2 3 Previous Next