VMware Cloud Community
Packee
Contributor
Contributor
Jump to solution

Failed updating the bootloader Errno 5 when updating from ESXi 6.5U1 to ESXi 6.7.0-8169922

Hi!

I've been searching hard for an answer to this and was hoping to get some help here.

Yesterday I tried to upgrade my ESXi server from 6.5.0 Update 1 (Build 5969303) to 6.7.0-8169922.

After some research I decided to do an offline update (my firewall is virtual so it made sense):

Tried to run this command:

esxcli software vib update -d /vmfs/volumes/DATASTORE/VMware-ESXi-6.7.0-8169922-depot.zip

However, the update failed with the error Failed updating the bootloader: [Errno 5] Input/output error: '/altbootbank/btldr.t00'

Anyone that has any advice on how to proceed with the upgrade?

Attaching the last part of the log below in case it contains any useful information.

Kind regards,

Patrik

2018-11-13T22:00:50Z esxupdate: 68410: BootBankInstaller.pyc: INFO: /tmp/stagebootbank/boot.cfg: bootstate changed from 3 to 4

2018-11-13T22:00:50Z esxupdate: 68410: HostImage: INFO: Nothing for LockerInstaller to do, skipping.

2018-11-13T22:00:50Z esxupdate: 68410: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/vmkfstools', '-P', '/bootbank']', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

2018-11-13T22:00:50Z esxupdate: 68410: HostInfo: DEBUG: Checking file system on altbootbank...

2018-11-13T22:00:50Z esxupdate: 68410: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/dosfsck', '-a', '-v', '/dev/disks/mpx.vmhba32:C0:T0:L0:5']', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

2018-11-13T22:00:50Z esxupdate: 68410: HostInfo: DEBUG: dosfsck exits with 0 Output:dosfsck 2.11 (12 Mar 2005) dosfsck 2.11, 12 Mar 2005, FAT32, LFN Checking we can access the last sector of the filesystem Boot sector contents: System ID "MSDOS5.0" Media byte 0xf8 (hard disk)        512 bytes per logical sector       4096 bytes per cluster          2 reserved sectors First FAT starts at byte 1024 (sector 2)          2 FATs, 16 bit entries     128000 bytes per FAT (= 250 sectors) Root directory starts at byte 257024 (sector 502)        512 root directory entries Data area starts at byte 273408 (sector 534)      63929 data clusters (261853184 bytes) 32 sectors/track, 64 heads          0 hidden sectors     511968 sectors total Reclaiming unconnected clusters. /dev/disks/mpx.vmhba32:C0:T0:L0:5: 132 files, 25717/63929 clusters

2018-11-13T22:00:50Z esxupdate: 68410: vmware.runcommand: INFO: runcommand called with: args = 'rm -rf /altbootbank/*', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

2018-11-13T22:00:53Z esxupdate: 68410: vmware.runcommand: INFO: runcommand called with: args = 'cp /tmp/stagebootbank/* /altbootbank/', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

2018-11-13T22:04:20Z esxupdate: 68410: vmware.runcommand: INFO: runcommand called with: args = 'localcli system visorfs ramdisk list | grep /stagebootbank && localcli system visorfs ramdisk remove -t /tmp/stagebootbank', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

2018-11-13T22:04:21Z esxupdate: 68410: HostImage: DEBUG: installer BootBankInstaller failed: ('VMware_bootbank_esx-base_6.7.0-0.0.8169922', "Failed updating the bootloader: [Errno 5] Input/output error: '/altbootbank/btldr.t00'"). Clean up the installation.

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR: Traceback (most recent call last):

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:   File "/build/mts/release/bora-5969303/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/site-packages/vmware/esximage/Installer/BootBankInstaller.py", line 1130, in _UpdateBootloader

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR: OSError: [Errno 5] Input/output error: '/altbootbank/btldr.t00'

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR: During handling of the above exception, another exception occurred:

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR: Traceback (most recent call last):

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:   File "/usr/lib/vmware/esxcli-software", line 468, in <module>

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:     main()

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:   File "/usr/lib/vmware/esxcli-software", line 459, in main

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:     ret = CMDTABLE[command](options)

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:   File "/usr/lib/vmware/esxcli-software", line 355, in VibInstallCmd

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:     nosigcheck=opts.nosigcheck)

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:   File "/build/mts/release/bora-5969303/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/site-packages/vmware/esximage/Transaction.py", line 262, in InstallVibsFromSources

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:   File "/build/mts/release/bora-5969303/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/site-packages/vmware/esximage/Transaction.py", line 382, in _installVibs

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:   File "/build/mts/release/bora-5969303/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/site-packages/vmware/esximage/Transaction.py", line 445, in _validateAndInstallProfile

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:   File "/build/mts/release/bora-5969303/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/site-packages/vmware/esximage/HostImage.py", line 893, in Remediate

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:   File "/build/mts/release/bora-5969303/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/site-packages/vmware/esximage/Installer/BootBankInstaller.py", line 1225, in Remediate

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR:   File "/build/mts/release/bora-5969303/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/site-packages/vmware/esximage/Installer/BootBankInstaller.py", line 1180, in _UpdateBootloader

2018-11-13T22:04:21Z esxupdate: 68410: root: ERROR: vmware.esximage.Errors.InstallationError: ('VMware_bootbank_esx-base_6.7.0-0.0.8169922', "Failed updating the bootloader: [Errno 5] Input/output error: '/altbootbank/btldr.t00'")

2018-11-13T22:08:40Z mark: storage-path-claim-completed

2018-11-13T22:08:40Z esxupdate: root: INFO: Command = profile.setacceptance

2018-11-13T22:08:40Z esxupdate: root: INFO: Options = {}

2018-11-13T22:08:40Z esxupdate: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/bootOption', '-rp']', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

2018-11-13T22:08:40Z esxupdate: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/bootOption', '-ro']', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

2018-11-13T22:08:40Z esxupdate: HostImage: INFO: Installers initiated are {'locker': <vmware.esximage.Installer.LockerInstaller.LockerInstaller object at 0x764263e908>, 'live': <vmware.esximage.Installer.LiveImageInstaller.LiveImageInstaller object at 0x7641776b00>, 'boot': <vmware.esximage.Installer.BootBankInstaller.BootBankInstaller object at 0x7641776e48>}

2018-11-13T22:08:40Z esxupdate: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/esxcfg-advcfg', '-U', 'host-acceptance-level', '-G']', outfile = 'None', returnoutput = 'True', timeout = '30.0'.

2018-11-13T22:08:40Z esxupdate: root: DEBUG: Finished execution of command = profile.setacceptance

2018-11-13T22:08:40Z esxupdate: root: DEBUG: Completed esxcli output, going to exit esxcli-softwareinternal

2018-11-13T22:08:53Z esxupdate: 67628: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/esxcfg-advcfg', '-q', '-g', '/UserVars/EsximageNetTimeout']', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

2018-11-13T22:08:53Z esxupdate: 67628: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/esxcfg-advcfg', '-q', '-g', '/UserVars/EsximageNetRetries']', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

2018-11-13T22:08:53Z esxupdate: 67628: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/esxcfg-advcfg', '-q', '-g', '/UserVars/EsximageNetRateLimit']', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

2018-11-13T22:08:53Z esxupdate: 67628: root: INFO: Command = acceptance.get

2018-11-13T22:08:53Z esxupdate: 67628: root: INFO: Options = {'pending': None, 'depot': None, 'dryrun': False, 'proxy': None, 'nomaintmode': False, 'oktoremove': False, 'downgrade': None, 'nameid': None, 'viburl': None, 'nosigcheck': False, 'force': False, 'level': None, 'noliveinstall': False, 'profile': None, 'updateonly': False}

2018-11-13T22:08:53Z esxupdate: 67628: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/bootOption', '-rp']', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

2018-11-13T22:08:53Z esxupdate: 67628: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/bootOption', '-ro']', outfile = 'None', returnoutput = 'True', timeout = '0.0'.

2018-11-13T22:08:53Z esxupdate: 67628: HostImage: INFO: Installers initiated are {'boot': <vmware.esximage.Installer.BootBankInstaller.BootBankInstaller object at 0xf39881d748>, 'locker': <vmware.esximage.Installer.LockerInstaller.LockerInstaller object at 0xf398cc6390>, 'live': <vmware.esximage.Installer.LiveImageInstaller.LiveImageInstaller object at 0xf39881d518>}

2018-11-13T22:08:53Z esxupdate: 67628: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/esxcfg-advcfg', '-U', 'host-acceptance-level', '-G']', outfile = 'None', returnoutput = 'True', timeout = '30.0'.

2018-11-13T22:08:53Z esxupdate: 67628: root: DEBUG: Finished execution of command = acceptance.get

2018-11-13T22:08:53Z esxupdate: 67628: root: DEBUG: Completed esxcli output, going to exit esxcli-software

Reply
0 Kudos
1 Solution

Accepted Solutions
Packee
Contributor
Contributor
Jump to solution

I tried a few things:

a.p's solution failed with an error message saying "Broken pipe".

Booting on an ISO and using it to upgraded failed.

Did not dare to try --force in case I am not able to get the server up again.

In the end I decided to try to re-install on another USB drive and restore the backup.

Lucky for me, since the backup did not work which would be a real head ache to fix.

In case someone runs into the same problem. Here's what worked for me:

  1. Bought a new USB drive.
  2. Made a new backup.
  3. Downloaded an ISO with the same build as my 6.5 U1 installation.
  4. Wrote the ISO to a new, separate USB stick using Rufus.
  5. Booted the ESX server on the new stick.
  6. Restored the backup using vim-cmd hostsvc/firmware/restore_config /tmp/configBundle.tgz
  7. After the ESX server reboot I upgraded it with esxcli software profile update -d /vmfs/volumes/SamsungSSD2/Upgrade/VMware-ESXi-6.7.0-8169922-depot.zip
  8. Rebooted.
  9. Ugraded with the update file using esxcli software vib install -d /vmfs/volumes/SamsungSSD2/Upgrade/update-from-esxi6.7-6.7_update01.zip
  10. Running ESX 6.7 U1 without any errors.

Will run like this for a few weeks until I am sure the system is stable. Then I'll scan the old USB drive for inconsistencies and then I'll re-install on it.

Thanks for all the suggestions!

Kind regards,

Patrik

View solution in original post

Reply
0 Kudos
7 Replies
palloveverma
Contributor
Contributor
Jump to solution

Hey Packee,

I hope the hardware where you are trying to install is compatible.

Try downloading a fresh image again and check it's MD5SUM to make sure what is downloaded is correct.

Once downloaded, try to run this command with below options:

esxcli software vib update -d /vmfs/volumes/DATASTORE/VMware-ESXi-6.7.0-8169922-depot.zip  --force --no-sig-check

Cheers,

Pallove

Reply
0 Kudos
Packee
Contributor
Contributor
Jump to solution

Hi

Thank your for your answer.

The MD5 sum is correct.

--force looks quite dangerous to me. Force seems to bypass the dependency checks. Is there no other suggestion before trying that?

/Patrik

Reply
0 Kudos
a_p_
Leadership
Leadership
Jump to solution

Assuming that the offline bundle covers your hardware, i.e. contains all required drivers, you may try to upgrade the host by running the following commands:

esxcli software sources profile list -d /vmfs/volumes/DATASTORE/VMware-ESXi-6.7.0-8169922-depot.zip

This will return the profile name that's required for the next command (likely ESXi-6.7.0-8169922-standard).

esxcli software update install -d /vmfs/volumes/DATASTORE/VMware-ESXi-6.7.0-8169922-depot.zip -p <profilename> --ok-to-remove --dry-run

If the command's output is what you expect, run it again without the --dry-run option.

If the upgrade completes successfully, enter reboot to restart the host.


André

PS: Depending on your hardware you may want to check whether the hardware vendor offers a customized ESXi image!

Reply
0 Kudos
palloveverma
Contributor
Contributor
Jump to solution

I have tried --force option and it works well without any issues.

How did you go with the upgrade if have tried with another attempt?

Cheers,

Pallove

Reply
0 Kudos
Packee
Contributor
Contributor
Jump to solution

I tried a few things:

a.p's solution failed with an error message saying "Broken pipe".

Booting on an ISO and using it to upgraded failed.

Did not dare to try --force in case I am not able to get the server up again.

In the end I decided to try to re-install on another USB drive and restore the backup.

Lucky for me, since the backup did not work which would be a real head ache to fix.

In case someone runs into the same problem. Here's what worked for me:

  1. Bought a new USB drive.
  2. Made a new backup.
  3. Downloaded an ISO with the same build as my 6.5 U1 installation.
  4. Wrote the ISO to a new, separate USB stick using Rufus.
  5. Booted the ESX server on the new stick.
  6. Restored the backup using vim-cmd hostsvc/firmware/restore_config /tmp/configBundle.tgz
  7. After the ESX server reboot I upgraded it with esxcli software profile update -d /vmfs/volumes/SamsungSSD2/Upgrade/VMware-ESXi-6.7.0-8169922-depot.zip
  8. Rebooted.
  9. Ugraded with the update file using esxcli software vib install -d /vmfs/volumes/SamsungSSD2/Upgrade/update-from-esxi6.7-6.7_update01.zip
  10. Running ESX 6.7 U1 without any errors.

Will run like this for a few weeks until I am sure the system is stable. Then I'll scan the old USB drive for inconsistencies and then I'll re-install on it.

Thanks for all the suggestions!

Kind regards,

Patrik

Reply
0 Kudos
palloveverma
Contributor
Contributor
Jump to solution

Well, that s a good news and glad to know that it worked eventually.

Cheers

Pallove

Reply
0 Kudos
BrianCunnie
Enthusiast
Enthusiast
Jump to solution

I tried my command a second time & it worked. Whew!

Reply
0 Kudos