Install Patch/Upgrade on vCenter 6.5 Appliance in vCenter HA mode

[Chnoili]

I try to upgrade my vCenter 6.5 appliance from 6.5.0.5100 Build Number 4602587 to 6.5.0.5200 Build Number 4944578. I downloaded the ISO with the patch, mounted it on the vm (vcenter appliance) and logged in via the vCenter Management Website. The update is found and i can click install updates - but the i get following message:

vCenter HA should be in the maintenance mode to apply an update

But vCenter HA is in maintenance mode - so i have no idea why the update is not working.

Has somebody some experiance with the update of the vCenter appliance? Any ideas?

Thanks

[buddybergman]

This may be a late reply, but for posterity, I wanted to add the step-by-step process I went through to update the vCenter 6.5 nodes to 6.0.5d (version 6.5.0.5500), as I could not find another one out there.

I ran into the same pitfall as the OP with no apparent reason given when trying to update in the vCenter server appliance web console, outside of "vCenter HA should be in maintenance mode", when in fact it already was.

Following the VMware update sequence steps for vSphere 6.5 - KB2147289, it states the PSC needs to be updated first, then vCenter.  It does not blatantly state (that I could see), what sequence to update the vCenter HA cluster nodes. I couldn't find that in any other document either.

If you try to update the active vCenter node via the web interface while vCenter HA is in Maintenance Mode, you will get a maintenance mode error and cannot continue.  No additional information is given.  However, if you try to do the update via command line on the active vCenter node, you get additional, more useful information, like so:

     : Installing patch update is allowed only on Passive or Witness node when the cluster is running in maintenance mode. Err: , Rc: 1

So basiclally, here are the high level update steps that I've determined for the best success:

1. Update the External Platform Services Controller first via web portal
2. Update the vCenter Witness node via CLI, with the ISO
3. Update the vCenter Passive (peer) node via CLI, with the ISO
4. Manually initiate vCenter HA failover
5. Update the (now) Passive vCenter node via CLI, with the ISO

Here are the more granular steps:

1) Put vCenter HA into Maintenance Mode.

2)  Web into the External Platform Services Controller

• Click on the Update button
• Click the Check Updates button -> Check Repository
• Click the Install Update button -> Install All Updates
• Accept the EULA and click the Install button

On the PSC, after the update to 6.5.0d there was an error on boot that stated "Failed to start LSB: Lightning fast webserver with light system requirements.  See 'systemctl status vami-lighttp.service' for details."

From a shell prompt, when you run the command "systemctl status vami-lighttp.service -l" you see a few errors. One of them is leading you to the cause of the problem: "Duplicate config variable in conditional 0 global: server.use-ipv6". This means in the config file there is a duplicate line.  Long story short, thanks to this community post, by editing the /opt/vmware/etc/lighttpd/lighttpd.conf file, and commenting out the last line "server.use-ipv6 = "disable"" (the duplicate in question), the failed service will now start by typing "systemctl start vami-lighttp.service".

3) SSH to the active vCenter node public IP

• type:  Shell
• change to Appliance shell "appliancesh":   /chsh -s /bin/appliancesh root
• verify you are in the Appliance shell:  echo $SHELL

4) From the active vCenter server:

• SSH to the witness node
  • type: ssh root@<Witness node HA IP>
    • you may get a notification that the authenticity can't be established.  I believe this is because the node you are trying to access (passive) is not yet updated and doesn't match the ECDSA key the node you are coming from (active).  Go ahead and continue.
    • once connected and you've entered your password, you should be at a Command> prompt.
• type: software-packages stage --iso --acceptEulas
  • the ISO will mount and check how many packages exist
  • the ISO will unmount
• type: software-packages install --iso --acceptEulas
  • the ISO will mount and the copy process will begin
  • ISO will unmount
  • pre-install script will run
  • VMware services will stop
  • package upgrade will begin
  • reboot when prompted the upgrade is complete
    • type: shutdown reboot -r <reason>
    • SSH connection to witness node will close as the server reboots

5) From the active vCenter node:

• SSH to the passive node
  • type: ssh root@<Passive node HA IP>
    • once connected and you've entered your password, you should be at a Command> prompt.
• type: software-packages stage --iso --acceptEulas
  • the ISO will mount and check how many packages exist
  • the ISO will unmount
• type: software-packages install --iso --acceptEulas
  • the ISO will mount and the copy process will begin
  • ISO will unmount
  • pre-install script will run
  • VMware services will stop
  • package upgrade will begin
  • reboot when prompted the upgrade is complete
    • type: shutdown reboot -r <reason>
    • SSH connection to passive node will close as the server reboots

6) In the vSphere web client, manually initiate a failover

• Side note: I noticed after the failover (to the vCenter node with the updated version) that in the vSphere Web Client, when the vCenter top-level name at the top-left is clicked, the center pane would be blank with a blue background. This was the case in both Firefox and Chrome. This was due to me having an older plug-in for the vCenter Operations Manager enabled in vCenter administration. I went to Home > Administration > Solutions / Client Plug-ins > and disabled the VCOps Client Plugin.
  

7) When the failover is complete, SSH to the (now) active vCenter node public IP, then:

• SSH to the passive node
  • type: ssh root@<Passive node HA IP>
    • once connected and you've entered your password, you should be at a Command> prompt.
• type: software-packages stage --iso --acceptEulas
  • the ISO will mount and check how many packages exist
  • the ISO will unmount
• type: software-packages install --iso --acceptEulas
  • the ISO will mount and the copy process will begin
  • ISO will unmount
  • pre-install script will run
  • VMware services will stop
  • package upgrade will begin
  • reboot when prompted the upgrade is complete
    • type: shutdown reboot -r <reason>
    • SSH connection to passive node will close as the server reboots

exit from the active vCenter SSH session.

I hope this helps someone out.

-Buddy

[Udayendu]

Hi,

Thanks for the update. I followed exactly the same steps and it didn't work through I was putting the HA into maintenance mode and trying to upgrade every node from CLI.

After checking with VMware they informed me that I am using a single PSC and no load balancer in between which is causing the issue.

So they have recommend me two options:

    - Either I need to disable the whole HA and do the upgrade then enable it back.

    - Or Put the loadbalancer between the vcenter and psc nodes as mentioned in

I am going to try it today and will update you soon the experience.

thanks,

--uday