Error while updating from 6.5 to 6.7 no space left

[scolombo]

We're updating from esxi 6.5 to 6.7 with offline update.

The update fails with the error 

[InstallationError]
[Errno 28] No space left on device
vibs = VMware_bootbank_esx-base_6.7.0-3.120.16773714
Please refer to the log file for more details.

we already enable the swap on data store and rebooted but still getting the error

Can anyone provide a fix for this ?

[kastlr]

Hi,

you should check the following KB article.

ESXi/ESX error: No free space left on device

Hth,

Ralf

[scolombo]

Hello 

I already had a look at it but Inodes are fine 

Following is the output of the command

[root@sg-bu-svt-01:/tmp] find / -path "/vmfs" -prune -o -type f -size +50000k -exec ls -lh '{}' \;
-r--r--r-- 1 root root 174.6M Nov 27 10:33 /tardisks/sb.v00
-r--r--r-- 1 root root 311.6M Nov 27 10:33 /tardisks/s.v00
-r--r--r-- 1 root root 114.6M Nov 27 10:33 /tardisks/kdriver.t00
-rw------- 1 root root 271.8G Nov 27 11:06 /dev/lvm/5fd21f48-be222f02-1f89-b47af13f8ce4
-rw------- 1 root root 279.4G Nov 27 11:06 /dev/disks/naa.600508b1001c7e3a9f863ed2438265d8
-rw------- 1 root root 2.5G Nov 27 11:06 /dev/disks/naa.600508b1001c7e3a9f863ed2438265d8:9
-rw------- 1 root root 286.0M Nov 27 11:06 /dev/disks/naa.600508b1001c7e3a9f863ed2438265d8:8
-rw------- 1 root root 110.0M Nov 27 11:06 /dev/disks/naa.600508b1001c7e3a9f863ed2438265d8:7
-rw------- 1 root root 250.0M Nov 27 11:06 /dev/disks/naa.600508b1001c7e3a9f863ed2438265d8:6
-rw------- 1 root root 250.0M Nov 27 11:06 /dev/disks/naa.600508b1001c7e3a9f863ed2438265d8:5
-rw------- 1 root root 272.0G Nov 27 11:06 /dev/disks/naa.600508b1001c7e3a9f863ed2438265d8:3
-rw------- 1 root root 4.0G Nov 27 11:06 /dev/disks/naa.600508b1001c7e3a9f863ed2438265d8:2

[a_p_]

Some questions:

1. What's the hosts current ESXi version/build?
2. Do you use the VMware default, or vendor customized ESXi version?
3. What's  the exact command line that you use for the offline upgrade?

Likely not related to the issue, but why don't you upgrade to the current ESXi 6.7 build?

André

[scolombo]

The host was running 6.5 and I've been able to install the HPE customized version VMware ESXi, 6.7.0, 15160138

However if I try, even now, to install the HPE Customized version  VMware ESXi, 6.7.0,16773714 I got the error "no space left"

the command I'm running is 

esxcli software profile update -p HPE-ESXi-6.7.0-Update3-Gen9plus-670.U3.10.6.0.83-customized -d /vmfs/volumes/SG-BU-SVT-DS01/Upgrade/Q8A57-11113_hpe-esxi6.7u3ep16-16773714-670.U3.10.6.0.83-offline-bundle.zip

[a_p_]

How did you run the upgrade? Did you run the commend using "profile install", or "profile update". "Update" may leave old - now unused - vibs installed. Unless you use additional custom drivers, you may try running the command using "profile install", which will end up with just the new vibs which are included in the profile, i.e. requiring less disk space.

Also verify the offline bundle's checksum to ensure that it's healthy.

André

[scolombo]

I ran the "update" not the "install" since on the host there might be vibs not from HPE customized patch bundle

[a_p_]

Unless you know which individual vibs are installed, you could try to manually uninstall stale vibs.

To get an overview you may run the following two "--dry-run" commands, and compare the "vibs removed" section to find out what can be uninstalled.

esxcli software profile install -p HPE-ESXi-6.7.0-Update3-Gen9plus-670.U3.10.6.0.83-customized -d /vmfs/volumes/SG-BU-SVT-DS01/Upgrade/Q8A57-11113_hpe-esxi6.7u3ep16-16773714-670.U3.10.6.0.83-offline-bundle.zip --ok-to-remove --dry-run

esxcli software profile update -p HPE-ESXi-6.7.0-Update3-Gen9plus-670.U3.10.6.0.83-customized -d /vmfs/volumes/SG-BU-SVT-DS01/Upgrade/Q8A57-11113_hpe-esxi6.7u3ep16-16773714-670.U3.10.6.0.83-offline-bundle.zip --dry-run

André

[scolombo]

I ran the dry-run install command and it reports a lot of VIBS in the "removed" section, I could not say which would be ok to remove actually

[a_p_]

I can't really tell you which ones are individual vibs, since I don't know your setup.

However, the ones which show up in the "update" command, are ones that are replaced with new ones in the bundle, so what you may do is to take a closer look at those from the install command which are not listed by the update command.

André