11 Replies Latest reply on Nov 25, 2019 10:54 AM by jgullo

    Update Manager completely gone after updating vCenter 6.7 to 6.7-update01

    perceptiv Lurker

      All,

       

      Weird thing happening.

       

      I do an upgrade from 6.7 and all seems well, except the fact that there is no trace of the Update Manager tool after vCenter restarts.

       

      In an attempt to solve this I did:

      • Re-register the Update Manager with  /usr/lib/vmware-updatemgr/bin/updatemgr-utility.py register_vc
      • Resetting the database using /usr/lib/vmware-updatemgr/bin/updatemgr-utility.py reset-db and cleaning out the patchstore folder
      • Restarted vCenter

       

      None of it helps. Rolling back, Update Manager re-appears without an issue and works fine.

       

      So, after investigating a little more, I discovered one weird thing. After rolling back to 6.7.0-9433931 and checking that everything works fine, I go to the VCSA using SSH.

      After I stage update 1 ISO using the VCSA console and run software-packages list --staged, this is the output I get:

       

      VMware vCenter Server Appliance 6.7.0.14000

      Type: vCenter Server with an embedded Platform Services Controller

      Last login: Sat Oct 27 16:44:21 2018 from 10.x.y.z
      Connected to service

          * List APIs: "help api list"
          * List Plugins: "help pi list"
          * Launch BASH: "shell"

      Command> software-packages stage --iso --acceptEulas
      [2018-10-27T17:12:40.300] : ISO mounted successfully
      [2018-10-27T17:12:41.300] : Evaluating packages to stage...
      [2018-10-27T17:12:41.300] : Verifying staging area
      [2018-10-27T17:12:41.300] : ISO unmounted successfully
      [2018-10-27T17:12:41.300] : Staging process completed successfully
      Command>
      Command>
      Command> software-packages list --staged
      [2018-10-27T17:12:54.300] :
              leaf_services: ['vmware-pod']
              thirdPartyInstallation: False
              rebootrequired: True
              buildnumber: 10244745
              productname: VMware vCenter Server Appliance
              summary: Update for VMware vCenter Server Appliance 6.7.0
              version_supported: ['6.7.0.10000', '6.7.0.11000', '6.7.0.12000']
              updateversion: True
              name: VC-6.7.0U1-Appliance-FP
              severity: Critical
              TPP_ISO: False
              category: Bugfix
              eulaAcceptTime: 2018-10-27 17:12:41 CEST
              kb: https://docs.vmware.com/en/VMware-vSphere/6.7/rn/vsphere-vcenter-server-671-release-notes.html#full_patch
              releasedate: October 16, 2018
              vendor: VMware, Inc.
              version: 6.7.0.20000
              size in MB: 1829
              tags: []
      Command>

       

      So, it seems this update is not supported on my version, although upgrading itself works without any issue other than Update Manager.

       

      I haven't been able to find any article on this issue anywhere just yet, so any help in this regards is very much appreciated.

        • 1. Re: Update Manager completely gone after updating vCenter 6.7 to 6.7-update01
          dbalcaraz Expert
          vExpert

          Hi,

           

          UM is fully supported on vSphere 6.7U1: VMware vSphere Update Manager 6.7 Update 1 Release Notes

          Did you check if there is another update available?

          VUM is integrated with vSphere so you shouldn't have a problem like that...

          Did you check the logs from the vCenter when upgrading?

          • 2. Re: Update Manager completely gone after updating vCenter 6.7 to 6.7-update01
            perceptiv Lurker

            Yes, I have seen the document. And yes, I know it is fully integrated in 6.7U1, that's the main reason to go for it.

            There is no update to the one I have.

            I'm a bit puzzled as to what log file to check for the upgrade results.

             

            But from what I can see everything looks normal. I even see update manager events such as checking for notifications.

            It's just that the shortcut to Update Manager and the Update Manager tabs are missing (both in HTML5 and FLEX) so there is no way of managing the updates.

            • 4. Re: Update Manager completely gone after updating vCenter 6.7 to 6.7-update01
              perceptiv Lurker

              Exactly. I can see it in 6.7, but not in 6.7 Update 1

              • 5. Re: Update Manager completely gone after updating vCenter 6.7 to 6.7-update01
                perceptiv Lurker

                VC67U1_Home.png

                VC67U1_Shortcuts.png

                As you can see the Update Manager does not show up on the Home screen (at the left), nor at the Shortcuts menu.

                It also does not appear in the various tabs.

                • 6. Re: Update Manager completely gone after updating vCenter 6.7 to 6.7-update01
                  dbalcaraz Expert
                  vExpert

                  All right, did you check if the Extension service is started ?

                  • 7. Re: Update Manager completely gone after updating vCenter 6.7 to 6.7-update01
                    IRIX201110141 Master

                    There was a posting about the exactly same issue here last week. Not not if a solution was found. I suggest to open a support request.

                     

                    Regards,

                    Joerg

                    • 8. Re: Update Manager completely gone after updating vCenter 6.7 to 6.7-update01
                      mc1903cae Enthusiast

                      Hi,

                       

                      Did you ever get a solution to this? I am having the same issue after upgrading from vCenter 6.7U2 to 6.7U3, via the VAMI download update.

                       

                      Cheers,

                      Martin

                      • 9. Re: Update Manager completely gone after updating vCenter 6.7 to 6.7-update01
                        mc1903cae Enthusiast

                        After many hours of google-looping searches and several restored vCenter appliances, I found a fix (that works for me at least).

                         

                        Credit to Shrikant @ https://virtuallyvtrue.com/2019/07/10/post-upgrading-to-vcenter-server-appliance-6-7-update-1-update-manager-plug-in-mis…

                         

                        First step is to spin up a new temp vCenter server, on a different IP/FQDN, to the exact same version/build as your broken vCenter server and follow the steps in the link or those below for step-by-step guide.

                         

                        IMPORTANT: Take a backup of your 'broken' vCenter Server via the VAMI before starting, just in case! :-)

                         

                        Then, on the New Temp vCenter Server:

                         

                        Enable SSH

                        Connect via SSH (Use PuTTY or similar) as root

                        Enable the shell

                        Execute the command: chsh -s /bin/bash

                        Connect via SCP (Use WinSCP or similar) as root

                        Browse to the following directory: /etc/vmware/vsphere-ui/cm-service-packages/com.vmware.cis.vsphereclient.plugin

                        Copy the 3 sub-directories below (your version numbers may differ as these are from a v6.7u3 VCSA) to a temp location on your local PC.

                         

                        com.vmware.fallback.mixed-6.5.0

                        com.vmware.vcIntegrity.vcIntegrity-6.7.0.41905

                        com.vmware.vum.client-6.7.0.41905

                         

                        Next, on your 'broken' vCenter Server:

                         

                        Enable SSH

                        Connect via SSH as root

                        Enable the shell

                        Execute the command: chsh -s /bin/bash

                         

                        Connect via SCP as root

                        Browse to the following directory: /etc/vmware/vsphere-ui/cm-service-packages/com.vmware.cis.vsphereclient.plugin

                        Rename the 3 existing directories to .bak (Again, your versions numbers may differ. There may be additional directories - do not touch these)

                         

                        com.vmware.fallback.mixed-6.5.0.bak

                        com.vmware.vcIntegrity.vcIntegrity-6.7.0.41703.bak

                        com.vmware.vum.client-6.7.0.41703.bak

                         

                        Copy the 3 directories from the temp location on your local PC into the same directory on the vCenter Server

                        Execute the command: cd /etc/vmware/vsphere-ui/cm-service-packages/com.vmware.cis.vsphereclient.plugin

                        Execute the command: chown vsphere-ui:users *

                        Execute the command: ls -l

                        Confirm the user:group permissions have been updated to vsphere-ui:users for each subdirectory

                         

                        root@mc-vcsa-v-101 [ /etc/vmware/vsphere-ui/cm-service-packages/com.vmware.cis.vsphereclient.plugin ]# ls -l

                        total 28

                        drwxr-xr-x 3 vsphere-ui users 4096 Oct  3 23:10 com.vmware.fallback.mixed-6.5.0

                        drwxrwx--- 3 vsphere-ui users 4096 Oct  3 23:01 com.vmware.fallback.mixed-6.5.0.bak

                        drwxrwx--- 3 vsphere-ui users 4096 Oct  3 23:10 com.vmware.imagebuilder.imagebuilder-6.7

                        drwxrwx--- 3 vsphere-ui users 4096 Sep 28 11:11 com.vmware.vcIntegrity.vcIntegrity-6.7.0.41703.bak

                        drwxr-xr-x 3 vsphere-ui users 4096 Oct  3 23:10 com.vmware.vcIntegrity.vcIntegrity-6.7.0.41905

                        drwxrwx--- 3 vsphere-ui users 4096 Sep 28 10:49 com.vmware.vum.client-6.7.0.41703.bak

                        drwxr-xr-x 3 vsphere-ui users 4096 Oct  3 23:09 com.vmware.vum.client-6.7.0.41905

                        root@mc-vcsa-v-101 [ /etc/vmware/vsphere-ui/cm-service-packages/com.vmware.cis.vsphereclient.plugin ]#

                         

                        Reboot the vCenter Server.

                        Once it has fully booted, login and you should see the Update Manager plugin once more.

                         

                        Hope this helps someone.

                        Martin

                        • 10. Re: Update Manager completely gone after updating vCenter 6.7 to 6.7-update01
                          jgullo Lurker

                          I believe it may be more nuanced than this, following an upgrade a single file in each folder was modified.  It's an esa file which the "file" utility lists as a java archive data file.  See the file modification dates (which is following a VCSA upgrade where my update plugin is still missing)  If I could get the original esa files, I'd try just replacing those and seeing if it works:

                           

                          root@vcenter [ /etc/vmware/vsphere-ui/cm-service-packages/com.vmware.cis.vsphereclient.plugin ]# ls -as1lR
                          .:
                          total 32
                          4 drwxrwx--- 8 vsphere-ui users 4096 Nov 22 23:26 .
                          4 drwxrwx--- 3 vsphere-ui root  4096 Jul 18 12:24 ..
                          4 drwxrwx--- 3 vsphere-ui users 4096 Nov 22 20:47 com.vmware.fallback.mixed-6.5.0
                          4 drwxrwx--- 3 vsphere-ui users 4096 Nov 20 23:31 com.vmware.vcIntegrity.vcIntegrity-6.7.0.41422
                          4 drwxrwx--- 3 vsphere-ui users 4096 Nov 20 23:31 com.vmware.vum.client-6.7.0.41422
                          
                          
                          ./com.vmware.fallback.mixed-6.5.0:
                          total 64
                           4 drwxrwx--- 3 vsphere-ui users  4096 Nov 22 20:47 .
                           4 drwxrwx--- 8 vsphere-ui users  4096 Nov 22 23:26 ..
                          24 -rw-r--r-- 1 vsphere-ui users 20892 Nov 22 20:47 com.vmware.fallback.mixed-6.5.0.esa
                          24 -rwxrwx--- 1 vsphere-ui users 21046 Dec 19  2018 com.vmware.fallback.mixed-6.5.0.zip
                           4 -rwxrwx--- 1 vsphere-ui users   490 Dec 19  2018 plugin-package.xml
                           4 drwxrwx--- 2 vsphere-ui users  4096 Dec 19  2018 plugins
                          
                          
                          ./com.vmware.fallback.mixed-6.5.0/plugins:
                          total 32
                           4 drwxrwx--- 2 vsphere-ui users  4096 Dec 19  2018 .
                           4 drwxrwx--- 3 vsphere-ui users  4096 Nov 22 20:47 ..
                          24 -rwxrwx--- 1 vsphere-ui users 23498 Dec 19  2018 mixed-mode-support-service-6.1.0.jar
                          
                          
                          ./com.vmware.vcIntegrity.vcIntegrity-6.7.0.41422:
                          total 9128
                             4 drwxrwx--- 3 vsphere-ui users    4096 Nov 20 23:31 .
                             4 drwxrwx--- 8 vsphere-ui users    4096 Nov 22 23:26 ..
                          4556 -rw-r--r-- 1 vsphere-ui users 4664214 Nov 20 23:31 com.vmware.vcIntegrity.vcIntegrity-6.7.0.41422.esa
                          4556 -rwxrwx--- 1 vsphere-ui users 4664151 Dec 19  2018 com.vmware.vcIntegrity.vcIntegrity-6.7.0.41422.zip
                             4 -rwxrwx--- 1 vsphere-ui users     665 Dec 19  2018 plugin-package.xml
                             4 drwxrwx--- 2 vsphere-ui users    4096 Nov 20 23:31 plugins
                          
                          
                          ./com.vmware.vcIntegrity.vcIntegrity-6.7.0.41422/plugins:
                          total 4672
                             4 drwxrwx--- 2 vsphere-ui users    4096 Nov 20 23:31 .
                             4 drwxrwx--- 3 vsphere-ui users    4096 Nov 20 23:31 ..
                           396 -rwxrwx--- 1 vsphere-ui users  403798 Dec 19  2018 updatemanager-6.1.0.jar
                            40 -rwxrwx--- 1 vsphere-ui users   37403 Dec 19  2018 updatemanager-adapter-6.1.0.jar
                          3676 -rwxrwx--- 1 vsphere-ui users 3763854 Dec 19  2018 updatemanager-ui-war-6.1.0.war
                            12 -rwxrwx--- 1 vsphere-ui users    8916 Dec 19  2018 vc-adapter-6.1.0.jar
                           540 -rwxrwx--- 1 vsphere-ui users  549959 Dec 19  2018 vum-vmodl-6.1.0.jar
                          
                          
                          ./com.vmware.vum.client-6.7.0.41422:
                          total 3732
                             4 drwxrwx--- 3 vsphere-ui users    4096 Nov 20 23:31 .
                             4 drwxrwx--- 8 vsphere-ui users    4096 Nov 22 23:26 ..
                          1860 -rw-r--r-- 1 vsphere-ui users 1901228 Nov 20 23:31 com.vmware.vum.client-6.7.0.41422.esa
                          1856 -rwxrwx--- 1 vsphere-ui users 1900213 Dec 19  2018 com.vmware.vum.client-6.7.0.41422.zip
                             4 -rwxrwx--- 1 vsphere-ui users     666 Dec 19  2018 plugin-package.xml
                             4 drwxrwx--- 2 vsphere-ui users    4096 Dec 19  2018 plugins
                          
                          
                          ./com.vmware.vum.client-6.7.0.41422/plugins:
                          total 1992
                             4 drwxrwx--- 2 vsphere-ui users    4096 Dec 19  2018 .
                             4 drwxrwx--- 3 vsphere-ui users    4096 Nov 20 23:31 ..
                           216 -rwxrwx--- 1 vsphere-ui users  220014 Dec 19  2018 vum-api-6.7.0.20000-SNAPSHOT.jar
                            44 -rwxrwx--- 1 vsphere-ui users   43235 Dec 19  2018 vum-connectivity-6.7.0.20000-SNAPSHOT.jar
                           152 -rwxrwx--- 1 vsphere-ui users  152124 Dec 19  2018 vum-service-6.7.0.20000-SNAPSHOT.jar
                          1032 -rwxrwx--- 1 vsphere-ui users 1053808 Dec 19  2018 vum-ui-6.7.0.20000-SNAPSHOT.war
                           540 -rwxrwx--- 1 vsphere-ui users  550480 Dec 19  2018 vum-vmodl-6.7.0.20000-SNAPSHOT.jar
                          
                          • 11. Re: Update Manager completely gone after updating vCenter 6.7 to 6.7-update01
                            jgullo Lurker

                            We had to replace the entire directory to make this work.  As an update, the chown command must be recursive:

                             

                            chown -R vsphere-ui:users *