6 Replies Latest reply on Jun 22, 2019 4:12 AM by Kheru

    ESXi 6.7 upgrade - No space left on device

    mzac23 Lurker

      I am trying to upgrade a small 'free' esxi 6.7 server I have to the latest 6.7 patch and I'm getting an error during the upgrade:

       

      [root@esx-desk:~] esxcli network firewall ruleset set -e true -r httpClient

      [root@esx-desk:~] esxcli software profile update -p ESXi-6.7.0-20180804001-standard \

      > -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

      [InstallationError]

      [Errno 28] No space left on device

             vibs = VMware_locker_tools-light_10.2.1.8267844-8941472

      Please refer to the log file for more details.

       

      Disk space looks good:

       

      [root@esx-desk:~] df

      Filesystem        Bytes        Used    Available Use% Mounted on

      VMFS-6     241860345856 59807629312 182052716544  25% /vmfs/volumes/datastore1

      vfat          299712512      204800    299507712   0% /vmfs/volumes/5b2d038c-ebae3704-d2b0-180373bd7e50

      vfat          261853184        8192    261844992   0% /vmfs/volumes/0e1aef7a-3250faf2-fde1-bed9900a6cc6

      vfat         4293591040    13434880   4280156160   0% /vmfs/volumes/5b2d0393-f1876dd8-bdc4-180373bd7e50

      vfat          261853184   148025344    113827840  57% /vmfs/volumes/44f5b073-0894e133-905d-37f5f70931ed

       

      I had read other forms with a similar issue and I tried to find the issue but can't find it, any ideas?

        • 1. Re: ESXi 6.7 upgrade - No space left on device
          Sureshkumar M Expert
          vExpert

          Can you try this

           

          enter the host webui and Go to Host > System > Swap and activate swap on our datastore vmfs.

          1 person found this helpful
          • 2. Re: ESXi 6.7 upgrade - No space left on device
            mzac23 Lurker

            Thanks that did it!

             

            [root@esx-desk:~] esxcli software profile update -p ESXi-6.7.0-20180804001-standard \

            > -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

            Update Result

               Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.

               Reboot Required: true

               VIBs Installed: VMW_bootbank_qedentv_2.0.6.4-10vmw.670.0.17.9214924, VMware_bootbank_cpu-microcode_6.7.0-0.20.9484548, VMware_bootbank_esx-base_6.7.0-0.20.9484548, VMware_bootbank_esx-ui_1.25.1-9210161, VMware_bootbank_vsan_6.7.0-0.20.9484548, VMware_bootbank_vsanhealth_6.7.0-0.20.9484548, VMware_locker_tools-light_10.2.1.8267844-8941472

               VIBs Removed: VMW_bootbank_qedentv_2.0.6.4-8vmw.670.0.0.8169922, VMware_bootbank_cpu-microcode_6.7.0-0.0.8169922, VMware_bootbank_esx-base_6.7.0-0.0.8169922, VMware_bootbank_esx-ui_1.25.0-7872652, VMware_bootbank_vsan_6.7.0-0.0.8169922, VMware_bootbank_vsanhealth_6.7.0-0.0.8169922, VMware_locker_tools-light_10.2.0.7253323-8169922

               VIBs Skipped: VMW_bootbank_ata-libata-92_3.00.9.2-16vmw.670.0.0.8169922, VMW_bootbank_ata-pata-amd_0.3.10-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-atiixp_0.4.6-4vmw.670.0.0.8169922, VMW_bootbank_ata-pata-cmd64x_0.2.5-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-hpt3x2n_0.3.4-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-pdc2027x_1.0-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-serverworks_0.4.3-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-sil680_0.4.8-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-via_0.3.3-2vmw.670.0.0.8169922, VMW_bootbank_block-cciss_3.6.14-10vmw.670.0.0.8169922, VMW_bootbank_bnxtnet_20.6.101.7-11vmw.670.0.0.8169922, VMW_bootbank_brcmfcoe_11.4.1078.0-8vmw.670.0.0.8169922, VMW_bootbank_char-random_1.0-3vmw.670.0.0.8169922, VMW_bootbank_ehci-ehci-hcd_1.0-4vmw.670.0.0.8169922, VMW_bootbank_elxiscsi_11.4.1174.0-2vmw.670.0.0.8169922, VMW_bootbank_elxnet_11.4.1094.0-5vmw.670.0.0.8169922, VMW_bootbank_hid-hid_1.0-3vmw.670.0.0.8169922, VMW_bootbank_i40en_1.3.1-18vmw.670.0.0.8169922, VMW_bootbank_iavmd_1.2.0.1011-2vmw.670.0.0.8169922, VMW_bootbank_igbn_0.1.0.0-15vmw.670.0.0.8169922, VMW_bootbank_ima-qla4xxx_2.02.18-1vmw.670.0.0.8169922, VMW_bootbank_ipmi-ipmi-devintf_39.1-4vmw.670.0.0.8169922, VMW_bootbank_ipmi-ipmi-msghandler_39.1-4vmw.670.0.0.8169922, VMW_bootbank_ipmi-ipmi-si-drv_39.1-4vmw.670.0.0.8169922, VMW_bootbank_iser_1.0.0.0-1vmw.670.0.0.8169922, VMW_bootbank_ixgben_1.4.1-11vmw.670.0.0.8169922, VMW_bootbank_lpfc_11.4.33.1-6vmw.670.0.0.8169922, VMW_bootbank_lpnic_11.4.59.0-1vmw.670.0.0.8169922, VMW_bootbank_lsi-mr3_7.702.13.00-4vmw.670.0.0.8169922, VMW_bootbank_lsi-msgpt2_20.00.04.00-4vmw.670.0.0.8169922, VMW_bootbank_lsi-msgpt35_03.00.01.00-10vmw.670.0.0.8169922, VMW_bootbank_lsi-msgpt3_16.00.01.00-1vmw.670.0.0.8169922, VMW_bootbank_misc-cnic-register_1.78.75.v60.7-1vmw.670.0.0.8169922, VMW_bootbank_misc-drivers_6.7.0-0.0.8169922, VMW_bootbank_mtip32xx-native_3.9.6-1vmw.670.0.0.8169922, VMW_bootbank_ne1000_0.8.3-4vmw.670.0.0.8169922, VMW_bootbank_nenic_1.0.11.0-1vmw.670.0.0.8169922, VMW_bootbank_net-bnx2_2.2.4f.v60.10-2vmw.670.0.0.8169922, VMW_bootbank_net-bnx2x_1.78.80.v60.12-2vmw.670.0.0.8169922, VMW_bootbank_net-cdc-ether_1.0-3vmw.670.0.0.8169922, VMW_bootbank_net-cnic_1.78.76.v60.13-2vmw.670.0.0.8169922, VMW_bootbank_net-e1000_8.0.3.1-5vmw.670.0.0.8169922, VMW_bootbank_net-e1000e_3.2.2.1-2vmw.670.0.0.8169922, VMW_bootbank_net-enic_2.1.2.38-2vmw.670.0.0.8169922, VMW_bootbank_net-fcoe_1.0.29.9.3-7vmw.670.0.0.8169922, VMW_bootbank_net-forcedeth_0.61-2vmw.670.0.0.8169922, VMW_bootbank_net-igb_5.0.5.1.1-5vmw.670.0.0.8169922, VMW_bootbank_net-ixgbe_3.7.13.7.14iov-20vmw.670.0.0.8169922, VMW_bootbank_net-libfcoe-92_1.0.24.9.4-8vmw.670.0.0.8169922, VMW_bootbank_net-mlx4-core_1.9.7.0-1vmw.670.0.0.8169922, VMW_bootbank_net-mlx4-en_1.9.7.0-1vmw.670.0.0.8169922, VMW_bootbank_net-nx-nic_5.0.621-5vmw.670.0.0.8169922, VMW_bootbank_net-tg3_3.131d.v60.4-2vmw.670.0.0.8169922, VMW_bootbank_net-usbnet_1.0-3vmw.670.0.0.8169922, VMW_bootbank_net-vmxnet3_1.1.3.0-3vmw.670.0.0.8169922, VMW_bootbank_nhpsa_2.0.22-1vmw.670.0.0.8169922, VMW_bootbank_nmlx4-core_3.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_nmlx4-en_3.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_nmlx4-rdma_3.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_nmlx5-core_4.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_nmlx5-rdma_4.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_ntg3_4.1.3.0-1vmw.670.0.0.8169922, VMW_bootbank_nvme_1.2.1.34-1vmw.670.0.0.8169922, VMW_bootbank_nvmxnet3-ens_2.0.0.21-1vmw.670.0.0.8169922, VMW_bootbank_nvmxnet3_2.0.0.27-1vmw.670.0.0.8169922, VMW_bootbank_ohci-usb-ohci_1.0-3vmw.670.0.0.8169922, VMW_bootbank_pvscsi_0.1-2vmw.670.0.0.8169922, VMW_bootbank_qcnic_1.0.2.0.4-1vmw.670.0.0.8169922, VMW_bootbank_qfle3_1.0.50.11-9vmw.670.0.0.8169922, VMW_bootbank_qfle3f_1.0.25.0.2-14vmw.670.0.0.8169922, VMW_bootbank_qfle3i_1.0.2.3.9-3vmw.670.0.0.8169922, VMW_bootbank_qflge_1.1.0.11-1vmw.670.0.0.8169922, VMW_bootbank_sata-ahci_3.0-26vmw.670.0.0.8169922, VMW_bootbank_sata-ata-piix_2.12-10vmw.670.0.0.8169922, VMW_bootbank_sata-sata-nv_3.5-4vmw.670.0.0.8169922, VMW_bootbank_sata-sata-promise_2.12-3vmw.670.0.0.8169922, VMW_bootbank_sata-sata-sil24_1.1-1vmw.670.0.0.8169922, VMW_bootbank_sata-sata-sil_2.3-4vmw.670.0.0.8169922, VMW_bootbank_sata-sata-svw_2.3-3vmw.670.0.0.8169922, VMW_bootbank_scsi-aacraid_1.1.5.1-9vmw.670.0.0.8169922, VMW_bootbank_scsi-adp94xx_1.0.8.12-6vmw.670.0.0.8169922, VMW_bootbank_scsi-aic79xx_3.1-6vmw.670.0.0.8169922, VMW_bootbank_scsi-bnx2fc_1.78.78.v60.8-1vmw.670.0.0.8169922, VMW_bootbank_scsi-bnx2i_2.78.76.v60.8-1vmw.670.0.0.8169922, VMW_bootbank_scsi-fnic_1.5.0.45-3vmw.670.0.0.8169922, VMW_bootbank_scsi-hpsa_6.0.0.84-3vmw.670.0.0.8169922, VMW_bootbank_scsi-ips_7.12.05-4vmw.670.0.0.8169922, VMW_bootbank_scsi-iscsi-linux-92_1.0.0.2-3vmw.670.0.0.8169922, VMW_bootbank_scsi-libfc-92_1.0.40.9.3-5vmw.670.0.0.8169922, VMW_bootbank_scsi-megaraid-mbox_2.20.5.1-6vmw.670.0.0.8169922, VMW_bootbank_scsi-megaraid-sas_6.603.55.00-2vmw.670.0.0.8169922, VMW_bootbank_scsi-megaraid2_2.00.4-9vmw.670.0.0.8169922, VMW_bootbank_scsi-mpt2sas_19.00.00.00-2vmw.670.0.0.8169922, VMW_bootbank_scsi-mptsas_4.23.01.00-10vmw.670.0.0.8169922, VMW_bootbank_scsi-mptspi_4.23.01.00-10vmw.670.0.0.8169922, VMW_bootbank_scsi-qla4xxx_5.01.03.2-7vmw.670.0.0.8169922, VMW_bootbank_shim-iscsi-linux-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-iscsi-linux-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libata-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libata-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libfc-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libfc-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libfcoe-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libfcoe-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-vmklinux-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-vmklinux-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-vmklinux-9-2-3-0_6.7.0-0.0.8169922, VMW_bootbank_smartpqi_1.0.1.553-10vmw.670.0.0.8169922, VMW_bootbank_uhci-usb-uhci_1.0-3vmw.670.0.0.8169922, VMW_bootbank_usb-storage-usb-storage_1.0-3vmw.670.0.0.8169922, VMW_bootbank_usbcore-usb_1.0-3vmw.670.0.0.8169922, VMW_bootbank_vmkata_0.1-1vmw.670.0.0.8169922, VMW_bootbank_vmkfcoe_1.0.0.0-1vmw.670.0.0.8169922, VMW_bootbank_vmkplexer-vmkplexer_6.7.0-0.0.8169922, VMW_bootbank_vmkusb_0.1-1vmw.670.0.0.8169922, VMW_bootbank_vmw-ahci_1.2.0-6vmw.670.0.0.8169922, VMW_bootbank_xhci-xhci_1.0-3vmw.670.0.0.8169922, VMware_bootbank_elx-esx-libelxima.so_11.4.1184.0-0.0.8169922, VMware_bootbank_esx-dvfilter-generic-fastpath_6.7.0-0.0.8169922, VMware_bootbank_esx-xserver_6.7.0-0.0.8169922, VMware_bootbank_lsu-hp-hpsa-plugin_2.0.0-13vmw.670.0.0.8169922, VMware_bootbank_lsu-lsi-lsi-mr3-plugin_1.0.0-12vmw.670.0.0.8169922, VMware_bootbank_lsu-lsi-lsi-msgpt3-plugin_1.0.0-8vmw.670.0.0.8169922, VMware_bootbank_lsu-lsi-megaraid-sas-plugin_1.0.0-9vmw.670.0.0.8169922, VMware_bootbank_lsu-lsi-mpt2sas-plugin_2.0.0-7vmw.670.0.0.8169922, VMware_bootbank_native-misc-drivers_6.7.0-0.0.8169922, VMware_bootbank_qlnativefc_3.0.1.0-5vmw.670.0.0.8169922, VMware_bootbank_rste_2.0.2.0088-7vmw.670.0.0.8169922, VMware_bootbank_vmware-esx-esxcli-nvme-plugin_1.2.0.32-0.0.8169922

            • 3. Re: ESXi 6.7 upgrade - No space left on device
              strauch Lurker

              Thank you. This Helps me too.

              • 4. Re: ESXi 6.7 upgrade - No space left on device
                unmesh59 Lurker

                I'm having a similar problem but no vib file is called out in the error message. I have swap enabled.

                 

                The end of esxupdate.log is

                 

                2019-01-21T06:42:42Z esxupdate: 2099595: root: ERROR: File "/build/mts/release/bora-10302608/bora/build/esx/release/vmvisor/esxupdate/lib64/python3.5/site-packages/vmware/esximage/Transaction.py", line 375, in UpdateProfileFromDepot

                2019-01-21T06:42:42Z esxupdate: 2099595: root: ERROR: File "/tmp/esx-update-2099595/usr/lib/vmware/weasel/util/upgrade_precheck.py", line 2161, in cliUpgradeAction

                2019-01-21T06:42:42Z esxupdate: 2099595: root: ERROR: File "/tmp/esx-update-2099595/usr/lib/vmware/weasel/util/upgrade_precheck.py", line 997, in _parseVmwareVersion

                2019-01-21T06:42:42Z esxupdate: 2099595: root: ERROR: File "/build/mts/release/bora-10302608/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 514, in getoutput

                2019-01-21T06:42:42Z esxupdate: 2099595: root: ERROR: File "/build/mts/release/bora-10302608/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 495, in getstatusoutput

                2019-01-21T06:42:42Z esxupdate: 2099595: root: ERROR: File "/build/mts/release/bora-10302608/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 316, in check_output

                2019-01-21T06:42:42Z esxupdate: 2099595: root: ERROR: File "/build/mts/release/bora-10302608/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 383, in run

                2019-01-21T06:42:42Z esxupdate: 2099595: root: ERROR: File "/build/mts/release/bora-10302608/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 676, in __init__

                2019-01-21T06:42:42Z esxupdate: 2099595: root: ERROR: File "/build/mts/release/bora-10302608/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 1228, in _execute_child

                2019-01-21T06:42:42Z esxupdate: 2099595: root: ERROR: OSError: [Errno 28] No space left on device

                 

                 

                • 5. Re: ESXi 6.7 upgrade - No space left on device
                  harrypnyce Novice

                  I'm experiencing similar difficulties.  I've successfully upgraded my primary ESXi host to 6.7 U2, but struggling with this machine.  I appear to have plenty of free space and have also enabled the /swap file and pointed it at a number of different datastores (SSDs, 10k rpm, et cetera), however all attempts have come up empty... still failing.  What else am I doing wrong?

                  [root@esxi2:~] stat -f /

                    File: "/"

                      ID: 100000000 Namelen: 127     Type: visorfs

                  Block size: 4096

                  Blocks: Total: 641306     Free: 447964     Available: 447964

                  Inodes: Total: 655360     Free: 648570

                  [root@esxi2:~] vdf -h

                  -----

                  Ramdisk                   Size      Used Available Use% Mounted on

                  root                       32M        2M       29M   7% --

                  etc                        28M      224K       27M   0% --

                  opt                        32M       40K       31M   0% --

                  var                        48M      480K       47M   0% --

                  tmp                       256M       24K      255M   0% --

                  iofilters                  32M        0B       32M   0% --

                  shm                      1024M        0B     1024M   0% --

                  hostdstats                303M        2M      300M   0% --

                  The specific error I am experiencing is:

                  [root@esxi2:~] esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-6.7.0-20190402001-standard

                  [InstallationError]

                  [Errno 28] No space left on device

                         vibs = VMware_locker_tools-light_10.3.5.10430147-12986307

                  Please refer to the log file for more details.

                  Originally installed using the custom HPE ISO, currently stuck on --> Version: 6.7.0 Update 1 (Build 11675023)

                   

                  I've downloaded the most recent ISO i could find <VMware-ESXi-6.7.0-Update2-13006603-HPE-Gen9plus-670.U2.10.4.1.8-Apr2019.iso> but would rather not completely rebuild my bootable USB flash drive, however I'm not sure what else to try at this point.  As I've been poking around the web, trying to research my issue(s), I saw someone eventually just went through this list: <https://esxi-patches.v-front.de/vm-6.7.0.html> and applied every available update manually, but I'm also not sure that's really a solution I'm after.  My other host (Cisco server) is usually the one that gives me fits, this HP Z420 workstation has been a breeze to work with, comparatively.  Thanks for anyone who may be able to shed some light on my difficulties.  Obviously I have a LOT to learn.

                  • 6. Re: ESXi 6.7 upgrade - No space left on device
                    Kheru Novice

                    I don't know if it helps, but trying using the profile (esxcli software profile update) kept giving me out of space.

                     

                    I switched to downloading the various packages with wget in / tmp, taking them from https://esxi-patches.v-front.de/ESXi-6.7.0.html .

                      wget https:// seemed to be stucked.  Using wget http:// i downloaded them and then esxcli software vib install   was flawlessy.