VMware Cloud Community
Projjwal
Contributor
Contributor

vRealize Orchestrator migration failures

Hi,

We are trying to migrate out of vCenter Orchestrator and move to vRealize Orchestrator.

We need to migrate the running instances and any configurations and workflow definitions, scripts without issues.

We copied the key file for passwordencryptor and the clientcerts file from vco over to vro [which, by the way, had 2 additional files absent in the vco]

Then we tried to take a full dump of the vco's internal postgres db and externalized it. We then pointed the vRO to this external db and then it asked to update the schema , which we did and restarted the orchestration configuration server.

But herein lies the issue - All transactions that were in "Waiting for user interaction" state in the vco were in "Cancelled" state in the vRO after that activity.

Clicking on the workflow instances in the vRO showed it as "failed" status and the vRO server logs show "InvalidCipherTextException: pad block corrupted exception."

Need some help around this, hence marking this discussion as a question.

Please let me know if any additional info around this is needed.

Thanks.

Tags (2)
0 Kudos
3 Replies
dvatov
VMware Employee
VMware Employee

I have seen this error in relation to different passwordencryptor.key file. But this file is used to encrypt also database password (stored in vmo.properties I think). If you have connection to the database on the new instance and you are using passwordencryptor.key from the old system then you also migrated the encrypted database password otherwise vRO will not connect to DB. Can you post here the log with the errors?

0 Kudos
Projjwal
Contributor
Contributor

Sure. Please see the log below :

[localhost-startStop-1] ERROR {} [Config_Plugin] org.bouncycastle.crypto.InvalidCipherTextException: pad block corrupted

java.lang.RuntimeException: org.bouncycastle.crypto.InvalidCipherTextException: pad block corrupted

    at ch.dunes.util.PasswordEncryptor.decrypt(PasswordEncryptor.java:76)

    at ch.dunes.util.EncryptHelper.newDecrypt(EncryptHelper.java:61)

    at ch.dunes.util.EncryptHelper.decrypt(EncryptHelper.java:45)

    at ch.dunes.util.EncryptHelper.decrypt(EncryptHelper.java:28)

    at ch.dunes.vso.configuration.web.commons.plugin.Config_Plugin.loadFromStream(Config_Plugin.java:125)

    at ch.dunes.vso.configuration.web.commons.plugin.Config_Plugin.initialize(Config_Plugin.java:172)

    at ch.dunes.vso.configuration.web.commons.ConfigurationPanel.addConfigurationEntry(ConfigurationPanel.java:23)

    at ch.dunes.vso.configuration.web.commons.VSOConfiguration.addConfigurationEntry(VSOConfiguration.java:235)

    at ch.dunes.vso.configuration.web.commons.VSOConfiguration.startConfigurationXML(VSOConfiguration.java:166)

    at ch.dunes.vso.configuration.web.commons.VSOConfiguration.startup(VSOConfiguration.java:182)

    at ch.dunes.vso.configuration.web.commons.VSOConfiguration.getConfiguration(VSOConfiguration.java:117)

    at com.vmware.vmo.plugin.vi4.VimPluginAdaptor.getVersion(VimPluginAdaptor.java:190)

    at com.vmware.vmo.plugin.vi4.VimPluginAdaptor.setPluginName(VimPluginAdaptor.java:63)

    at ch.dunes.vso.sdk.SDKDatasource.initializeDatasource(SDKDatasource.java:176)

    at ch.dunes.vso.sdk.SDKModuleDescription.initializeDatasources(SDKModuleDescription.java:391)

    at ch.dunes.vso.sdk.SDKModuleDescription.initialize(SDKModuleDescription.java:347)

    at ch.dunes.vso.sdk.ModulesFactory.loadModule(ModulesFactory.java:403)

    at ch.dunes.vso.sdk.ModulesFactory.loadDunesArchive(ModulesFactory.java:181)

    at ch.dunes.vso.sdk.ModulesFactory.loadModulesInPath(ModulesFactory.java:164)

    at ch.dunes.vso.sdk.ModulesFactory.loadModules(ModulesFactory.java:155)

    at ch.dunes.vso.sdk.ModulesFactory.loadMapper(ModulesFactory.java:828)

    at ch.dunes.scripting.jsmodel.Definitions.loadMappers(Definitions.java:179)

    at com.vmware.o11n.sdk.EnhancedScriptingSDK.afterPropertiesSet(EnhancedScriptingSDK.java:258)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1612)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1549)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)

    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:304)

    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)

    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:300)

    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:195)

    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:700)

    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:760)

    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)

    at org.springframework.context.support.AbstractRefreshableConfigApplicationContext.afterPropertiesSet(AbstractRefreshableConfigApplicationContext.java:151)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1612)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1549)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)

    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:304)

    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)

    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:300)

    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:195)

    at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:973)

    at com.vmware.o11n.integration.initialization.VCOApplicationContextInitializer.tryToInitializeVcoApplicationContext(VCOApplicationContextInitializer.java:143)

    at com.vmware.o11n.integration.initialization.VCOApplicationContextInitializer.nextExecutionTime(VCOApplicationContextInitializer.java:53)

    at org.springframework.scheduling.concurrent.ReschedulingRunnable.schedule(ReschedulingRunnable.java:68)

    at org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler.schedule(ThreadPoolTaskScheduler.java:247)

    at org.springframework.scheduling.config.ScheduledTaskRegistrar.scheduleTasks(ScheduledTaskRegistrar.java:299)

    at org.springframework.scheduling.config.ContextLifecycleScheduledTaskRegistrar.onApplicationEvent(ContextLifecycleScheduledTaskRegistrar.java:64)

    at org.springframework.scheduling.config.ContextLifecycleScheduledTaskRegistrar.onApplicationEvent(ContextLifecycleScheduledTaskRegistrar.java:32)

    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:98)

    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:333)

    at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:776)

    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:485)

    at org.springframework.context.support.AbstractRefreshableConfigApplicationContext.afterPropertiesSet(AbstractRefreshableConfigApplicationContext.java:151)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1612)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1549)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)

    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:304)

    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)

    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:300)

    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:195)

    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:700)

    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:760)

    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)

    at org.springframework.context.access.ContextSingletonBeanFactoryLocator.initializeDefinition(ContextSingletonBeanFactoryLocator.java:143)

    at org.springframework.beans.factory.access.SingletonBeanFactoryLocator.useBeanFactory(SingletonBeanFactoryLocator.java:382)

    at com.vmware.o11n.service.spring.bootstrap.Bootstrap.loadContext(Bootstrap.java:19)

    at com.vmware.o11n.service.spring.bootstrap.Bootstrap.getInitialBeanFactory(Bootstrap.java:74)

    at com.vmware.o11n.service.spring.bootstrap.ServiceBootstrap.customizeContext(ServiceBootstrap.java:233)

    at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:402)

    at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:306)

    at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:106)

    at com.vmware.o11n.service.spring.bootstrap.ServiceBootstrap.contextInitialized(ServiceBootstrap.java:168)

    at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5016)

    at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5528)

    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)

    at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)

    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)

    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)

    at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1095)

    at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1930)

    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

    at java.util.concurrent.FutureTask.run(FutureTask.java:262)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

    at java.lang.Thread.run(Thread.java:745)

Caused by: org.bouncycastle.crypto.InvalidCipherTextException: pad block corrupted

    at org.bouncycastle.crypto.paddings.PKCS7Padding.padCount(Unknown Source)

    at org.bouncycastle.crypto.paddings.PaddedBufferedBlockCipher.doFinal(Unknown Source)

    at ch.dunes.util.PasswordEncryptor.decrypt(PasswordEncryptor.java:74)

    ... 88 more

[localhost-startStop-1] INFO  {} [VimPluginAdaptor] Plugin VC  version 6.0.2 build 2995112 will initialize

0 Kudos
dvatov
VMware Employee
VMware Employee

The passwordencryptor.key is not the one used to encrypt plugin's install password. I guess that you have encrypted password in _VSOPluginState.xml. I think it is safe to remove it (or blank it) just to check whether you will get beyond this point. Or figure out which passwordencryptor.key was used.

0 Kudos