VMware Cloud Community
SunF1re
Contributor
Contributor
Jump to solution

Unable to upgrade embedded ESXi 3 to U2 on IBM BladeCenter HS21 XM

Hi all,

we just purchased an IBM BladeCenter and for a start loaded with 3x HS21 XM Blades with the embedded ESX3i. In general everything seems to work fine but we are unable to upgrade to the latest U2 patch. The Blades are shipped with embedded ESXi 3.5 build 71173.

When we run the VMware Infrastructure Update client does show the updates and when we choose to install the firmware update progresses to about 98% and then displays and error message "Failed to install Firmware update.: Unable to Install". I attached a sceenshot of what is showing in the ESX server event log after the failed upgrade. We also tried the vihostupdate from the remote CLI package with the very same result.

Did anybody run into similiar issues? This for all 3 Blades so I guess this may be a more general problem with the embedded ESX3i running off a USB Flash drive on the IBM's?

Any help is appreciated.

Thanks, Thorsten

Reply
0 Kudos
46 Replies
benny_hauk
Enthusiast
Enthusiast
Jump to solution

There's probably better explanations of the unsupported shell but here's one (see step 1).

I'll say I don't know much at all about it (I've never had Tech Support send me there) but based on the warnings when you use it, it seems that using it may require you to reboot the host when you are finished using it. Someone chime in if that's wrong, but that's how I read it so I was sure to use a non-production ESXi host when testing dilpreet's fix.

Benny Hauk Systems Admin, VCP3/VCP4 LifeWay Chrstian Resources
Reply
0 Kudos
dilpreet
VMware Employee
VMware Employee
Jump to solution

You should not need to reboot the machine but remember the shell is "unsupported" so writing scripts for it or expecting a particular behavior is ill advised.

Reply
0 Kudos
dilpreet
VMware Employee
VMware Employee
Jump to solution

Can you try the following and see if you get the newer build number:

- reboot ESXi

- while the white bar is displayed across the screen during the boot process, hold down the SHIFT key and the R key.

This should cause you to boot to the altbootbank which should contain the updated image (we will need to figure out why this wasn't done automatically following the update, if this works)

thanks.

Reply
0 Kudos
benny_hauk
Enthusiast
Enthusiast
Jump to solution

Looks like the Update 2 build isn't here that I could tell. Here are the results of the +R during the Hypervisor loading:

I see screen that looks like:

I hit Enter to get rid of the Warning message. Then I hit ESC to see the 2 log messages. Here's what the screen then showed (don't know if these lines are the 2 log messages or not):

boot.cfg: No Kernel argument found.

Warning: Bank 2 is corrupt or invalid. Ignoring.

Thanks again.

Benny Hauk Systems Admin, VCP3/VCP4 LifeWay Chrstian Resources
Reply
0 Kudos
dilpreet
VMware Employee
VMware Employee
Jump to solution

Here is something that should work:

I believe that is because the boot.cfg in /altbootbank is corrupted. Need to add the following step after file system is fixed.

1.cp /bootbank/boot.cfg /tmp

2. modify updated, and bootstate of /tmp/boot.cfg

kernel=vmkernel.gz

kernelopt=

modules=binmod.tgz --- environ.tgz --- cim.tgz --- oem.tgz --- license.tgz[http:// --- state.tgz|http:// --- state.tgz] ====> remove content in bracket if it is there.

build=3.5.0-82664

updated=2------> increase this value by 1

bootstate=0----


>replace the old value with 3

3. cp /tmp/boot.cfg /altbootbank

4. do update

Reply
0 Kudos
benny_hauk
Enthusiast
Enthusiast
Jump to solution

That doesn't seem to make a difference except there are no longer 2 lines in the log when doing the Shift+R command. Still just one valid fallback Hypervisor here though (the build 71173). Here is a list of the steps I followed (to verify I'm following the right steps in the right order):

Installed the 3.5.0 Hypervisor from the VMware ESX Server 3i Recovery Tools CD from IBM (IBM P/N 46M1600; the updated recovery CD I received in the mail that fixes past issues with past recovery CDs)

At this point we're at 3.5.0, build 71173 (that build never seems to change)

From Console:

Alt+F1

Type: unsupported / Password

Type: vmkfstools -P /altbootbank/

Returned an HBA name of vmhba32:0:0:6

Type: dosfsck -v /dev/disks/vmhba32:0:0:6

Type: dosfsck -a /dev/disks/vmhba32:0:0:6

Type: cp /bootbank/boot.cfg /tmp

Type: vi /tmp/boot.cfg (made changes below)

updated=1------> increase this value by 1 - I changed it from '1' to '2'

bootstate=0----


>replace the old value with 3 - I changed it from '0' to '3'

Type: cp /tmp/boot.cfg /altbootbank

Successfully ran Update 2 install from the VMware Infrastrucure Update tool on my desktop

Rebooted

Same result (build 71173).

Ran the dosfsck -a command again to make sure the update didn't corrupt something and it seemed okay

Rebooted and did the Shift+R at the white bar and saw the same results (only one option) as I wrote about above. Only difference here was that there is no more lines in the log, it just says, "Warning: Only one Valid Fallback Hypervisor found. Press Enter to continue."

Observations:

/vmfs/volumes/Hypervisor1

Contains original boot.cfg and various .tgz files dated 1/9/2008.

/vmfs/volumes/Hypervisor2

Contains the boot.cfg file I modified and various .tgz files from the Update 2 package (the boot.cfg still says build=71173) along with a FSCK0000.REC file and an md5 checksum file.

/vmfs/volumes/Hypervisor3

Contains handfull of folders (opt var vmupgrade packages ...)

Benny Hauk Systems Admin, VCP3/VCP4 LifeWay Chrstian Resources
Reply
0 Kudos
benny_hauk
Enthusiast
Enthusiast
Jump to solution

Finally figured it out! (Okay, so Dilpreet figured it out with the 2 of us working offline). Here's the fix that worked regardless of which IBM Recovery CD we used (tested and worked with both the IBM P/N 46M1600 and 44W4140). Dilpreet recommends going ahead and doing this fix in our production environment in order to get to Update 2 instead of waiting on IBM to come out with another Recovery CD. Here's what I did immediately after using the Recovery CD to put a fresh image on the USB and rebooting (I guess I assigned an IP addr to the Mgmt Network, etc, first and then followed these steps):

1. Go to the unsupported shell (from the Console hit Alt+F1 then type 'unsupported' and press Enter - then type root password)

2. Find out the HBA name

vmkfstools -P /altbootbank/

vfat-0.03 file system spanning 1 partitions.

File system label (if any): Hypervisor2

Mode: private

Capacity 50101248 (48927 file blocks * 1024),

50100224 (48926 blocks) avail

UUID: 76ec2b74-a7965bdd-bf7d-a26c90143d09

Partitions spanned (on "disks"):

vmhba32:0:0:6 <---- HBA name

4. repair file system problem (use the same HBA name discovered from previous command):

dosfsck -a /dev/disks/vmhba32:0:0:6

5. rm -rf /altbootbank/*

6. cp /bootbank/boot.cfg /tmp

7. modify updated, and bootstate of /tmp/boot.cfg

kernel=vmkernel.gz

kernelopt=

modules=binmod.tgz --- environ.tgz --- cim.tgz ---

oem.tgz --- license.tgz --- state.tgz ====> remove content in bracket if it is there.

build=3.5.0-82664

updated=2_------> increase this value by 1_

bootstate=0_----


>replace the old value with 3_

8. cp /tmp/boot.cfg /altbootbank

9. do update (I used the VMware Infrastructure Update tool from my desktop)

10. when finished reboot the host and it should come up as VMware ESX Server 3i, 3.5.0, 110271

If you are considering the installable vs. the embedded version of ESXi take into consideration that embedded seems inherently more complex because the VMware updates essentially must have the correct IBM/Dell/HP/etc CIM modules. We just got 2 new blades, these with a single SSD instead of an embedded USB chip. We'll see which is easier. I'm suspecting these newer ones with SSD drives (using the installable version) will be better than there embedded counterparts at least for BladeCenter since I don't really have to have any agents, etc installed at all (all hardware management can be handled out-of-band with RSA, completely avoiding ESX altogether). We'll see. I get the impression that the same may not follow for HP, that they rely more heavily on the CIM modules... can't say.

Good luck and thanks a ton Dilpreet!

Benny Hauk Systems Admin, VCP3/VCP4 LifeWay Chrstian Resources
Reply
0 Kudos