sz3jdii
Contributor
Contributor

[OSError] [Errno 28] No space left on device even SWAP is enabled

Hello,

I have 6.7.0 Update 3 (Build 14320388) ESXi without vSphere which I want to update to the newest version. The problem is I get the below error:

Spoiler
[root@s0:~] esxcli software profile update -p ESXi-6.7.0-20201103001-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-de
pot-index.xml
[OSError]
[Errno 28] No space left on device
Please refer to the log file for more details.
[root@s0:~]

 

I have swap enabled.

I've tried manual VMware_locker_tools-light_11.1.1.16303738-16701467.vib extracted from the newest update and uploaded to /tmp instalation with

Spoiler
esxcli software vib install -f -v /tmp/VMware_locker_tools-light_11.1.1.16303738-16701467.vib

but it says it's skipped.

df -h

Spoiler
Filesystem Size Used Available Use% Mounted on
VMFS-6 1.8T 1.0T 791.3G 58% /vmfs/volumes/STRG_1
VMFS-6 1.8T 752.4G 1.1T 40% /vmfs/volumes/STRG_2
VMFS-6 1.8T 1.8G 1.8T 0% /vmfs/volumes/BACKUP
vfat 4.0G 25.6M 4.0G 1% /vmfs/volumes/5ede6198-f3e30ba2-28c3-0cc47a7ae075
vfat 249.7M 156.1M 93.7M 62% /vmfs/volumes/57828670-5e7ec7d2-181d-4c63851fea26
vfat 249.7M 156.2M 93.5M 63% /vmfs/volumes/6bd2607f-b602f803-4517-5a374ab17978
vfat 285.8M 197.5M 88.4M 69% /vmfs/volumes/5ede6190-0b945224-b893-0cc47a7ae075
[root@s0:~]

esxupdate.log

Spoiler
2020-11-08T00:27:23Z esxupdate: 2100550: root: ERROR: File "/build/mts/release/bora-17098360/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 1228, in _execute_child
2020-11-08T00:27:23Z esxupdate: 2100550: root: ERROR: OSError: [Errno 28] No space left on device

 

All VMs are stopped, host in maintenance mode.

 

vdf -h

 

Spoiler
[root@s0:~] vdf -h
Tardisk Space Used
vmx.v00 102M 102M
vim.v00 112M 112M
sb.v00 188M 188M
s.v00 58M 58M
i40en.v00 680K 678K
intel_nv.v00 80K 78K
net_ixgb.v00 568K 565K
lsiprovi.v00 27M 27M
ata_liba.v00 296K 294K
ata_pata.v00 48K 44K
ata_pata.v01 36K 35K
ata_pata.v02 40K 36K
ata_pata.v03 40K 37K
ata_pata.v04 44K 41K
ata_pata.v05 40K 38K
ata_pata.v06 36K 35K
ata_pata.v07 40K 39K
block_cc.v00 88K 84K
bnxtnet.v00 560K 557K
bnxtroce.v00 304K 303K
brcmfcoe.v00 2M 2M
char_ran.v00 52K 51K
ehci_ehc.v00 96K 95K
elxiscsi.v00 516K 514K
elxnet.v00 644K 641K
hid_hid.v00 64K 60K
iavmd.v00 180K 176K
igbn.v00 332K 329K
ima_qla4.v00 1M 1M
ipmi_ipm.v00 44K 40K
ipmi_ipm.v01 84K 82K
ipmi_ipm.v02 104K 102K
iser.v00 236K 234K
ixgben.v00 536K 532K
lpfc.v00 2M 2M
lpnic.v00 644K 641K
lsi_mr3.v00 368K 366K
lsi_msgp.v00 516K 515K
lsi_msgp.v01 560K 560K
lsi_msgp.v02 548K 544K
misc_cni.v00 24K 22K
misc_dri.v00 1M 1M
mtip32xx.v00 256K 254K
ne1000.v00 640K 639K
nenic.v00 272K 270K
net_bnx2.v00 288K 286K
net_bnx2.v01 2M 2M
net_cdc_.v00 28K 25K
net_cnic.v00 148K 145K
net_e100.v00 312K 311K
net_e100.v01 356K 353K
net_enic.v00 176K 175K
net_fcoe.v00 80K 79K
net_forc.v00 128K 125K
net_igb.v00 324K 321K
net_libf.v00 76K 75K
net_mlx4.v00 356K 355K
net_mlx4.v01 240K 236K
net_nx_n.v00 1M 1M
net_tg3.v00 316K 312K
net_usbn.v00 56K 53K
net_vmxn.v00 112K 108K
nfnic.v00 548K 544K
nhpsa.v00 568K 565K
nmlx4_co.v00 728K 726K
nmlx4_en.v00 756K 755K
nmlx4_rd.v00 328K 326K
nmlx5_co.v00 1M 1M
nmlx5_rd.v00 260K 257K
ntg3.v00 120K 119K
nvme.v00 296K 292K
nvmxnet3.v00 172K 171K
nvmxnet3.v01 172K 170K
ohci_usb.v00 64K 63K
pvscsi.v00 120K 118K
qcnic.v00 284K 281K
qedentv.v00 2M 2M
qfle3.v00 2M 2M
qfle3f.v00 1M 1M
qfle3i.v00 336K 332K
qflge.v00 504K 500K
sata_ahc.v00 88K 87K
sata_ata.v00 60K 56K
sata_sat.v00 68K 66K
sata_sat.v01 48K 46K
sata_sat.v02 52K 48K
sata_sat.v03 40K 39K
sata_sat.v04 36K 34K
scsi_aac.v00 180K 178K
scsi_adp.v00 452K 450K
scsi_aic.v00 296K 292K
scsi_bnx.v00 284K 280K
scsi_bnx.v01 204K 203K
scsi_fni.v00 244K 242K
scsi_hps.v00 212K 211K
scsi_ips.v00 108K 106K
scsi_isc.v00 44K 42K
scsi_lib.v00 212K 211K
scsi_meg.v00 100K 99K
scsi_meg.v01 176K 173K
scsi_meg.v02 96K 93K
scsi_mpt.v00 460K 459K
scsi_mpt.v01 504K 501K
scsi_mpt.v02 432K 429K
scsi_qla.v00 292K 289K
sfvmk.v00 604K 602K
shim_isc.v00 16K 15K
shim_isc.v01 16K 15K
shim_lib.v00 44K 43K
shim_lib.v01 44K 43K
shim_lib.v02 24K 20K
shim_lib.v03 24K 20K
shim_lib.v04 12K 9K
shim_lib.v05 12K 9K
shim_vmk.v00 220K 217K
shim_vmk.v01 232K 228K
shim_vmk.v02 236K 232K
smartpqi.v00 284K 281K
uhci_usb.v00 64K 63K
usb_stor.v00 164K 162K
usbcore_.v00 320K 318K
vmkata.v00 204K 201K
vmkfcoe.v00 936K 935K
vmkplexe.v00 48K 46K
vmkusb.v00 980K 979K
vmw_ahci.v00 240K 236K
xhci_xhc.v00 236K 235K
elx_esx_.v00 4M 4M
btldr.t00 588K 584K
esx_dvfi.v00 492K 488K
esx_ui.v00 14M 14M
esxupdt.v00 576K 572K
weaselin.t00 4M 4M
lsu_hp_h.v00 152K 149K
lsu_inte.v00 44K 41K
lsu_lsi_.v00 420K 417K
lsu_lsi_.v01 240K 237K
lsu_lsi_.v02 496K 493K
lsu_lsi_.v03 264K 260K
lsu_lsi_.v04 560K 557K
lsu_smar.v00 96K 93K
native_m.v00 820K 818K
qlnative.v00 2M 2M
rste.v00 828K 825K
vmware_e.v00 188K 186K
vsan.v00 43M 43M
vsanheal.v00 7M 7M
vsanmgmt.v00 25M 25M
xorg.v00 3M 3M
imgdb.tgz 1M 1M
state.tgz 16K 16K
-----
Ramdisk Size Used Available Use% Mounted on
root 32M 2M 29M 7% --
etc 28M 196K 27M 0% --
opt 32M 0B 32M 0% --
var 48M 372K 47M 0% --
tmp 256M 20K 255M 0% --
iofilters 32M 0B 32M 0% --
shm 1024M 0B 1024M 0% --
hostdstats 303M 1M 301M 0% --

Thank you for any help

Labels (1)
0 Kudos
1 Reply
Neutro
Enthusiast
Enthusiast

Hey,

Oh yes this problem also caused me to go nuts before!

Even if you enable swap the ESXi busybox space which is used for cli is limited to around 200MB space, which is not enough for running the update.

The solution is simple:

Manually download the update package from https://my.vmware.com/group/vmware/patch#search on your computer then upload it on your server datastore, let's say in /datastore1/update/ESXi670-202011001.zip for example

Then on the SSH cli use these commands:

esxcli software profile update --dry-run -p ESXi-6.7.0-20201103001-standard -d /vmfs/volumes/datastore1/update/ESXi670-202011001.zip

If the dry run is successful, do the update with

esxcli software profile update -p ESXi-6.7.0-20201103001-standard -d /vmfs/volumes/datastore1/update/ESXi670-202011001.zip

Reboot and the update is done 🙂