Reply to Message

View discussion in a popup

Replying to:
rajeshd1508
Contributor
Contributor
‎03-20-2023 12:33 AM

Hi @Denis_Chorbadzh , thanks for clarifying our queries.

To add to @ShivamVerma380's comment, we are NOT putting anything into /usr/lib/vmware-vsphere-ui/plugin-packages/ directory. The contents of this directory are same before and after our plugin deployment.

root@vcsa8 [ ~ ]# ll /usr/lib/vmware-vsphere-ui/plugin-packages/
total 32K
drwxr-xr-x 3 root root 4.0K Mar 14 16:30 cis-data-service-plugin
drwxr-xr-x 3 root root 4.0K Mar 14 16:30 license-client
drwxr-xr-x 3 root root 4.0K Mar 14 16:30 opsmgmt-client
drwxr-xr-x 3 root root 4.0K Mar 14 16:30 root-app
drwxr-xr-x 3 root root 4.0K Mar 14 16:30 sr-upload
drwxr-xr-x 3 root root 4.0K Mar 14 16:30 sso-admin-ui
drwxr-xr-x 3 root root 4.0K Mar 14 16:30 telemetry
drwxr-xr-x 3 root root 4.0K Mar 14 16:30 vsphere-client
root@vcsa8 [ ~ ]#
root@vcsa8 [ ~ ]# ll /usr/lib/vmware-vsphere-ui/plugin-packages/vsphere-client/
total 8.0K
-rwxr-xr-x 1 root root 2.9K Jan 11 12:39 plugin-package.xml
drwxr-xr-x 2 root root 4.0K Mar 14 16:30 plugins
root@vcsa8 [ ~ ]#

As you've mentioned, post installation, the plugin.zip is getting unpacked at /etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity by VCSA which has /libraries and /plugins directories with respective contents. 

root@vcsa8 [ ~ ]# ll /etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity
total 28K
drwxr-x--- 3 vsphere-ui users 4.0K Jan  6 00:44 com.vmware.cloud.provider.services.plugin-1.0.0.0
drwxr-x--- 3 vsphere-ui users 4.0K Jan  6 00:56 com.vmware.vsan.client-8.0.200.10000
drwxr-xr-x 3 vsphere-ui users 4.0K Mar 14 16:47 com.vmware.lcm.client-8.0.0.10200
drwxr-xr-x 3 vsphere-ui users 4.0K Mar 14 16:47 com.vmware.vlcm.client-8.0.0.21216066
drwxr-xr-x 2 vsphere-ui users 4.0K Mar 15 05:07 com.example.plugin-1.2.1.27
drwxr-x--- 4 vsphere-ui users 4.0K Mar 15 05:18 com.vmware.vrops.install-8.0.200.10000
drwxr-xr-x 5 vsphere-ui users 4.0K Mar 19 14:22 com.vcplugin.plugin-1.2.1.27
root@vcsa8 [ ~ ]#
root@vcsa8 [ ~ ]# ll /etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity/com.vcplugin.plugin-1.2.1.27/
total 16M
-rw-r--r-- 1 vsphere-ui users  16M Mar 19 14:22 com.vcplugin.plugin-1.2.1.27.zip
-rw-r--r-- 1 vsphere-ui users  498 Mar 19 14:22 plugin-package.xml
drwxr-xr-x 2 vsphere-ui users 4.0K Mar 19 14:22 libraries
drwxr-xr-x 2 vsphere-ui users 4.0K Mar 19 14:22 plugins
drwxr-xr-x 2 vsphere-ui users 4.0K Mar 19 14:22 META-INF
root@vcsa8 [ ~ ]#

Though we can see in https://vcsa80.example.com/mob/?moid=ExtensionManager that plugin is getting registered, the deployment tasks is failing with time out for deploying a bundle.

Snippet from the vsphere_client_virgo.log

[2023-03-19T14:22:30.047Z] [INFO ] plugin-discovery-bus-thread   com.vmware.vise.plugin.registry.VcExtensionRegistryImpl           Associated plugin com.vcplugin.plugin:1.2.1.27 with VC vcsa8.example.com with guid 90e06f45-343c-4a44-9581-f0ae24277e6a.
[2023-03-19T14:22:30.047Z] [INFO ] plugin-discovery-bus-thread   com.vmware.vise.plugin.registry.VcExtensionStateRegistry          Adding registry entry for plugin: 'com.vcplugin.plugin:1.2.1.27'.
[2023-03-19T14:22:30.047Z] [INFO ] plugin-discovery-bus-thread   com.vmware.vise.plugin.registry.VcExtensionStateRegistry          Registering plugin: 'com.vcplugin.plugin:1.2.1.27' with vCenter: 'vcsa8.example.com (90e06f45-343c-4a44-9581-f0ae24277e6a)
[2023-03-19T14:22:30.047Z] [INFO ] plugin-discovery-bus-thread   com.vmware.vise.plugin.registry.VcExtensionStateRegistry          Checking entry existence: Plugin: 'com.vcplugin.plugin:1.2.1.27', State: '[FAILED_DOWNLOAD]'.
[2023-03-19T14:22:30.048Z] [INFO ] plugin-discovery-bus-thread   com.vmware.vise.plugin.registry.VcExtensionStateRegistry          Updating entry: Plugin: 'com.vcplugin.plugin:1.2.1.27', State: 'DOWNLOADING'
[2023-03-19T14:22:30.048Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.plugin.async.task.PluginDownloadTask              Starting download of plugin package: 'com.vcplugin.plugin:1.2.1.27'.
[2023-03-19T14:22:30.229Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.vim.extension.PluginStatusTaskManager             User: VSPHERE.LOCAL\vsphere-webclient-4ced4907-5f14-4884-89f8-7a206a1598df initiated plugin DOWNLOAD task for plugin com.vcplugin.plugin:1.2.1.27.
[2023-03-19T14:22:30.229Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.plugin.status.LocalPluginStatusServiceImpl        DOWNLOADING: Downloading plugin package com.vcplugin.plugin:1.2.1.27 from https://10.235.139.10:18080/storage/vcplugin-bundle.zip ...
[2023-03-19T14:22:30.233Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.plugin.download.PluginDownloadServiceImpl         Downloading plug-in package 'com.vcplugin.plugin:1.2.1.27' with temporary name '/etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity/com.vcplugin.plugin-1.2.1.27/plugin-package.tmp'.
[2023-03-19T14:22:30.234Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.plugin.download.PluginDownloadServiceImpl         Downloading plugin package from https://10.235.139.10:18080/storage/vcplugin-bundle.zip
[2023-03-19T14:22:31.572Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.plugin.download.PluginDownloadServiceImpl         Done downloading plugin package 'com.vcplugin.plugin:1.2.1.27' from 'https://10.235.139.10:18080/storage/vcplugin-bundle.zip'
[2023-03-19T14:22:31.572Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.plugin.download.PluginDownloadServiceImpl         Renaming downloaded plug-in '/etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity/com.vcplugin.plugin-1.2.1.27/plugin-package.tmp' to '/etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity/com.vcplugin.plugin-1.2.1.27/com.vcplugin.plugin-1.2.1.27.zip'
[2023-03-19T14:22:31.572Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.plugin.download.PluginDownloadServiceImpl         Successfully renamed /etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity/com.vcplugin.plugin-1.2.1.27/plugin-package.tmp to /etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity/com.vcplugin.plugin-1.2.1.27/com.vcplugin.plugin-1.2.1.27.zip.
[2023-03-19T14:22:31.572Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.plugin.download.PluginDownloadServiceImpl         Unzipping plug-in '/etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity/com.vcplugin.plugin-1.2.1.27/com.vcplugin.plugin-1.2.1.27.zip' into '/etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity/com.vcplugin.plugin-1.2.1.27' folder.
[2023-03-19T14:22:31.813Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.plugin.download.PluginDownloadServiceImpl         Successfully unzipped plug-in '/etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity/com.vcplugin.plugin-1.2.1.27/com.vcplugin.plugin-1.2.1.27.zip' into '/etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity/com.vcplugin.plugin-1.2.1.27' folder.
[2023-03-19T14:22:31.846Z] [ERROR] sdk-plugin-deployer-173       com.vmware.vise.plugin.signature.SignatureStateServiceImpl        Unable to verify the signature for plugin 'com.vcplugin.plugin:1.2.1.27'. com.vmware.vise.extensionfw.signing.PluginSignatureGeneralException: The signature verification process failed.
[2023-03-19T14:22:31.846Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.plugin.download.PluginDownloadServiceImpl         Successfully downloaded plug-in package 'com.vcplugin.plugin:1.2.1.27'.
[2023-03-19T14:22:31.881Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.plugin.status.LocalPluginStatusServiceImpl        DOWNLOADED: Downloaded plugin package com.vcplugin.plugin:1.2.1.27 from https://10.235.139.10:18080/storage/vcplugin-bundle.zip successfully.
[2023-03-19T14:22:31.882Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.vim.extension.VcExtensionManager                  Handling successfully downloaded plugin: 'com.vcplugin.plugin:1.2.1.27' registered in vCenter: 'vcsa8.example.com (90e06f45-343c-4a44-9581-f0ae24277e6a)'.
[2023-03-19T14:22:31.882Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.plugin.registry.VcExtensionStateRegistry          Updating entry: Plugin: 'com.vcplugin.plugin:1.2.1.27', State: 'DOWNLOADED'
[2023-03-19T14:22:31.882Z] [INFO ] sdk-plugin-deployer-173       com.vmware.vise.plugin.registry.VcExtensionStateRegistry          Updating entry: Plugin: 'com.vcplugin.plugin:1.2.1.27', State: 'DEPLOYING'
[2023-03-19T14:22:31.882Z] [INFO ] sdk-plugin-deployer-177       com.vmware.vise.plugin.async.task.PluginDeployTask                Starting deployment of plugin package: 'com.vcplugin.plugin:1.2.1.27' in vCenter server: 'vcsa8.example.com (90e06f45-343c-4a44-9581-f0ae24277e6a)'.
[2023-03-19T14:22:31.923Z] [INFO ] sdk-plugin-deployer-177       com.vmware.vise.vim.extension.PluginStatusTaskManager             User: VSPHERE.LOCAL\vsphere-webclient-4ced4907-5f14-4884-89f8-7a206a1598df initiated plugin DEPLOYMENT task for plugin com.vcplugin.plugin:1.2.1.27.
[2023-03-19T14:22:31.923Z] [INFO ] sdk-plugin-deployer-177       com.vmware.vise.plugin.status.LocalPluginStatusServiceImpl        DEPLOYING: Deploying plugin package com.vcplugin.plugin:1.2.1.27 ...
[2023-03-19T14:22:31.928Z] [INFO ] sdk-plugin-deployer-177       com.vmware.vise.extensionfw.impl.sandboxing.EsaBuilder            Creating OSGi sandbox for package com.vcplugin.plugin with scope fullyIsolated
[2023-03-19T14:22:32.997Z] [INFO ] plugin-deploy16               com.vmware.vise.extensionfw.impl.PackagesDeployer                 Deploying plugin package 'com.vcplugin.plugin:1.2.1.27'.
[2023-03-19T14:22:32.997Z] [INFO ] plugin-deploy16               com.vmware.vise.extensionfw.impl.DeploymentEventLoggerImpl        Event=PACKAGE_DEPLOY_BEGIN; Params=com.vcplugin.plugin, 1.2.1.27
[2023-03-19T14:32:32.997Z] [ERROR] sdk-plugin-deployer-177       com.vmware.vise.extensionfw.impl.PackagesDeployer                 Package deployer timed out waiting for com.vcplugin.plugin:1.2.1.27;
[2023-03-19T14:32:33.040Z] [ERROR] sdk-plugin-deployer-177       com.vmware.vise.plugin.status.LocalPluginStatusServiceImpl        DEPLOYMENT_FAILED: Error deploying plugin package com.vcplugin.plugin:1.2.1.27. Reason: Plugin deployment has timed out.
[2023-03-19T14:32:33.042Z] [ERROR] plugin-deploy16               com.vmware.vise.extensionfw.impl.PackagesDeployer                 Error deploying one of more bundles for the plugin package /etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity/com.vcplugin.plugin-1.2.1.27 java.lang.InterruptedException: null
[2023-03-19T14:32:33.042Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       Started validating com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.042Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       Begin VmwareNameMisuseCheck on com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.042Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       End VmwareNameMisuseCheck on com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.042Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       Begin DynamicImportUsageCheck on com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.042Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       End DynamicImportUsageCheck on com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.042Z] [WARN ] plugin-deploy16               com.vmware.vise.vim.extension.PluginStatusTaskManager             Skip posting tasks for plugin package com.vcplugin.plugin:1.2.1.27: No task to update. This has no impact on plugin package deployment but is expected only during plugin development. In Production the plugin package should be registered as a vCenter extension.
[2023-03-19T14:32:33.043Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       Begin RequireBundleUsageCheck on com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.043Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       End RequireBundleUsageCheck on com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.043Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       Begin UnversionedPackageImportCheck on com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.043Z] [ERROR] plugin-deploy16               com.vmware.vise.plugin.status.LocalPluginStatusServiceImpl        DEPLOYMENT_FAILED: Error deploying plugin package com.vcplugin.plugin:1.2.1.27. Reason: Deployment error. java.lang.InterruptedException: null
[2023-03-19T14:32:33.043Z] [ERROR] sdk-plugin-deployer-177       com.vmware.vise.vim.extension.VcExtensionManager                  Deployment of plugin package: 'com.vcplugin.plugin:1.2.1.27' registered in vCenter Server: 'vcsa8.example.com (90e06f45-343c-4a44-9581-f0ae24277e6a)' has failed. java.util.concurrent.CompletionException: com.vmware.vise.plugin.deploy.PluginDeploymentException: Deployment of plugin: 'com.vcplugin.plugin:1.2.1.27' in vCenter server: 'vcsa8.example.com (90e06f45-343c-4a44-9581-f0ae24277e6a)' failed!
Caused by: com.vmware.vise.plugin.deploy.PluginDeploymentException: Deployment of plugin: 'com.vcplugin.plugin:1.2.1.27' in vCenter server: 'vcsa8.example.com (90e06f45-343c-4a44-9581-f0ae24277e6a)' failed!
[2023-03-19T14:32:33.043Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       End UnversionedPackageImportCheck on com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.043Z] [INFO ] sdk-plugin-deployer-177       com.vmware.vise.plugin.registry.VcExtensionStateRegistry          Updating entry: Plugin: 'com.vcplugin.plugin:1.2.1.27', State: 'FAILED_DEPLOY'
[2023-03-19T14:32:33.043Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       Begin PluginPackageGlobalScopeCheck on com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.043Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       End PluginPackageGlobalScopeCheck on com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.043Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       Begin PluginDeploymentExceptionCheck on com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.043Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       End PluginDeploymentExceptionCheck on com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.043Z] [INFO ] plugin-validation1            com.vmware.vise.extensionfw.impl.OsgiUsageValidationService       Finished validating com.vcplugin.plugin:1.2.1.27
[2023-03-19T14:32:33.187Z] [ERROR] plugin-deploy16               com.vmware.vise.extensionfw.impl.DeploymentEventLoggerImpl        Event=PACKAGE_DEPLOY_FAIL; Params=com.vcplugin.plugin, 1.2.1.27

The eventlog.log for same duration

[2023-03-19T14:22:33.654Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=org.eclipse.virgo.web.dm:1.0.0 Type=RESOLVED
[2023-03-19T14:22:33.655Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=com.vmware.wsvim25:8.0.0 Type=RESOLVED
[2023-03-19T14:22:33.656Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=OSGIBundle:1.0.0 Type=RESOLVED
[2023-03-19T14:22:33.659Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=org.eclipse.virgo.web.dm:1.0.0 Type=STARTING
[2023-03-19T14:22:33.701Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=org.eclipse.virgo.web.dm:1.0.0 Type=STARTED
[2023-03-19T14:22:33.702Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=OSGIBundle:1.0.0 Type=STARTING
[2023-03-19T14:22:33.702Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=OSGIBundle:1.0.0 Type=STARTED
[2023-03-19T14:22:33.703Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=com.vmware.wsvim25:8.0.0 Type=STARTING
[2023-03-19T14:22:33.711Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=com.vmware.wsvim25:8.0.0 Type=STARTED
[2023-03-19T14:32:33.079Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=com.vmware.wsvim25:8.0.0 Type=STOPPING
[2023-03-19T14:32:33.081Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=com.vmware.wsvim25:8.0.0 Type=STOPPED
[2023-03-19T14:32:33.081Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=OSGIBundle:1.0.0 Type=STOPPING
[2023-03-19T14:32:33.082Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=OSGIBundle:1.0.0 Type=STOPPED
[2023-03-19T14:32:33.082Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=org.eclipse.virgo.web.dm:1.0.0 Type=STOPPING
[2023-03-19T14:32:33.083Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=org.eclipse.virgo.web.dm:1.0.0 Type=STOPPED
[2023-03-19T14:32:33.129Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=OSGIBundle:1.0.0 Type=UNRESOLVED
[2023-03-19T14:32:33.133Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=OSGIBundle:1.0.0 Type=UNINSTALLED
[2023-03-19T14:32:33.138Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=org.eclipse.virgo.web.dm:1.0.0 Type=UNRESOLVED
[2023-03-19T14:32:33.156Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=org.eclipse.virgo.web.dm:1.0.0 Type=UNINSTALLED
[2023-03-19T14:32:33.160Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=com.vmware.wsvim25:8.0.0 Type=UNRESOLVED
[2023-03-19T14:32:33.164Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=com.vmware.wsvim25:8.0.0 Type=UNINSTALLED
[2023-03-19T14:32:33.169Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=org.osgi.service.subsystem.region.context.7:1.0.0 Type=STOPPING
[2023-03-19T14:32:33.169Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=org.osgi.service.subsystem.region.context.7:1.0.0 Type=STOPPED
[2023-03-19T14:32:33.170Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=org.osgi.service.subsystem.region.context.7:1.0.0 Type=UNRESOLVED
[2023-03-19T14:32:33.172Z] [INFO ] plugin-deploy16               [BundleEvent] Bundle=org.osgi.service.subsystem.region.context.7:1.0.0 Type=UNINSTALLED

From these logs, it can be seen that the Bundle=com.vmware.wsvim25:8.0.0 is getting STOPPED and plugin deployment is getting timed out around same time, but no other log is showing the cause or reason for this time out. Can you please point us to what can cause this timeout?

Few points to be NOTED:

1. vcsa8.example.com is just for representation purpose. The actual VCSA hostname is resolvable FQDN.
2. Nothing suspicious seen from tcpdump on VCSA.
3. We've created single OSGIBundle:1.0.0 bundle having all the dependencies.

Reply
0 Kudos