vCenter Server Service Wont Start after P2V

[BostonTechGuy]

Hello community.

Have an issue I have not been able to resolve and looking for assistance.

Problem:

vCenter Server Service wont start after the Windows Server was P2V'd.  Getting an ERROR 2 from the Service.  Getting Windows System Event Log Error 7024.

History:

The client I am currently working with is trying to break up its DEV Location's VCENTER.  Now DEV Location vcenter runs everything for that site (domains, dns, and all developement servers).  So its that location's PRODUCTION VCENTER.  Currently the SQL Server (dedicated to vCenter), VCENTER and SSO are all sitting on the same box.  We are going to break up the SSO, SQL and VCENTER into their own servers.  However the client wants to re-use the current physical server for the SQL Server.  So I need to P2V the existing physical server before I can begin the new environment.

I have been able to get all the services working on the Virtual vCenter (SQL, SSO, Update Manager etc etc) but have been unable to get the actual vCenter Server Service to work.  Fails everytime with ERROR 2 Code.  I have been unable to get a KB article from VMWARE that fits the issue that I am having.  ERROR 2 code has many articles and none have worked.

Details:

vCenter 5.1 Build 799731

SQL 2008 R2

Windows 2008 R2

When the vCenter Service tries to start it will stop at the 50% mark and give this error: (I have removed names for security reasons)

Windows could not start the VMware VirtualCenter Server on <name of server>. For more information, review the System Event Log.  If this is a non-Microsoft service, contact the service vendor, and refer to service-specific error code 2

When I review the Windows Event System Log I get this error:

EVENTID: 7024

The VMware VirtualCenter Server service terminated with service-specific error The system cannot find the file specified..

When I review the vpxd log file I see this at the end of the log: (I have removed names for security reasons)

2013-12-03T16:08:21.843-05:00 [09168 info 'dbdbPortgroup'] [VpxdInvtDVPortGroup::PreLoadDvpgConfig] loaded [0] dvpg config objects

2013-12-03T16:08:21.844-05:00 [09168 info 'authvpxdMoSessionManager'] [SSO][SessionManagerMo::Init] Admin URI set to: https://<name of vcenter server:Port #>/sso-adminserver/sdk

2013-12-03T16:08:21.844-05:00 [09168 info 'authvpxdMoSessionManager'] [SSO][SessionManagerMo::Init] Downloading STS Root certificates ...

2013-12-03T16:08:21.858-05:00 [01440 info 'Default'] Thread attached

2013-12-03T16:08:21.880-05:00 [09168 error 'vpxdvpxdMain'] [Vpxd::ServerApp::Init] Init failed: Unexpected exception

--> Backtrace:

--> backtrace[00] rip 000000018018977a

--> backtrace[01] rip 0000000180100c98

--> backtrace[02] rip 0000000180101fae

--> backtrace[03] rip 000000018008aeab

--> backtrace[04] rip 00000000005b4971

--> backtrace[05] rip 0000000000551298

--> backtrace[06] rip 00000000005516c9

--> backtrace[07] rip 00000000004c0fae

--> backtrace[08] rip 00000001405bbfb8

--> backtrace[09] rip 000000013f4b0078

--> backtrace[10] rip 000000013f4b016a

--> backtrace[11] rip 000000013f4b0279

--> backtrace[12] rip 000000013f4b0609

--> backtrace[13] rip 000000013f7f2903

--> backtrace[14] rip 000000013ff9e4b9

--> backtrace[15] rip 000000013ff9835c

--> backtrace[16] rip 00000001401b8a3b

--> backtrace[17] rip 000007fefedca82d

--> backtrace[18] rip 000000007783652d

--> backtrace[19] rip 0000000077ccc521

-->

2013-12-03T16:08:21.881-05:00 [09168 warning 'VpxProfiler'] ServerApp::Init [TotalTime] took 4603 ms

2013-12-03T16:08:21.881-05:00 [09168 error 'Default'] Failed to intialize VMware VirtualCenter. Shutting down...

2013-12-03T16:08:21.882-05:00 [09168 info 'vpxdvpxdSupportManager'] Wrote uptime information

2013-12-03T16:08:29.280-05:00 [00788 warning 'VpxProfiler' opID=SWI-39d3665d] VpxUtil_InvokeWithOpId [TotalTime] took 12000 ms

2013-12-03T16:08:41.280-05:00 [00788 warning 'VpxProfiler' opID=SWI-da6b3df9] VpxUtil_InvokeWithOpId [TotalTime] took 12001 ms

2013-12-03T16:08:53.281-05:00 [00788 warning 'VpxProfiler' opID=SWI-5086bf7f] VpxUtil_InvokeWithOpId [TotalTime] took 12001 ms

2013-12-03T16:09:05.282-05:00 [00788 warning 'VpxProfiler' opID=SWI-3abd7c64] VpxUtil_InvokeWithOpId [TotalTime] took 12000 ms

2013-12-03T16:09:17.283-05:00 [00788 warning 'VpxProfiler' opID=SWI-b9c8c88a] VpxUtil_InvokeWithOpId [TotalTime] took 12001 ms

2013-12-03T16:09:17.353-05:00 [09168 info 'Default'] Forcing shutdown of VMware VirtualCenter now

I have pulled the VM into a lab and removed all network based issues that would occur from isolating it.  Now the VCENTER is yelling about that it cant find any of its hosts in the vdxp log but that shouldn't matter.  All the Services have been changed to Local Accounts.  All Network and ODBC settings are able to see the server.. aka the server can see itself.  I have not removed any software or drivers from the VM (techincal this is my third P2V of the same server for testing reasons).. And still the only thing that wont start is VCENTER SERVER... SSO is started, SQL is started..

Also everything has been left the same on the VM (DNS, IP, NAME etc etc). Shutting down the physical and booting it back up does not show any issues.  Only when its been P2V'd does this come up.

I just cant get this issue.  Need help.

Thoughts?

Thanks,

BostonTechGuy

[BostonTechGuy]

I am seeing a lot of other posts for both vCenter 5.1 and 5.5 having same issue.  The Services wont start.  Does anyone recognize the errors above?

Thanks,

BostonTechGuy

[julienvarela]

Hi,

According to this KB : VMware KB:    After making a change or restarting vCenter Single Sign-On server system, vCenter Serv...

Can you try this :

To resolve this issue:

1. Click Start, right-click Command Prompt, and click Run as administrator to open a command prompt as an administrator.
2. Set the Java home.  The default location is C:\Program Files\VMware\Infrastructure\jre Type:
   set JAVA_HOME=C:\Program Files\VMware\Infrastructure\jre
3. In the system where SSO is installed, locate and navigate to the SSO server installation directory. By default, this directory is located at C:\Program Files\VMware\Infrastructure\SSOServer\Utils.
4. Run this command:
   
   Note: This command replaces the masterPassword with the SSO Password  (admin@system-domain) rsautil manage-secrets -a recover -m <masterPassword>
5. Restart the SSO service.
6. Restart the VMware VirtualCenter Server service.

Hope it helps.

Regards,

Julien.

[BostonTechGuy]

Thank you for the post.  We have tried this solution and still having the issue.  Its the strangest thing. The physical is not having any issues and all the P2V'd servers are.

[julienvarela]

Can you attached the vpxd.log file please.

Regards,

Julien.

[homerzzz]

This might be a silly question, but are all of the dependencies for the vcenter service started on the p2v'd server?

[plug75]

Hi Guys,  anyone have any idea's as I'm having the same issue.

I have tried doing an upgrade to latest 5.1 Update 1c to see if that fixes anything but it fails with some SSO-related errors in the logs.  Strange, since the SSO service (and all other services) start just fine.

Also tested the ODBC connection to the Database and that also works fine.  Only VirtualCenter service doesn't start?

As far as I can see all dependencies are fine.  I am starting to doubt myself whether I stopped all services (maybe specifically the SSO service?) before I P2V'd.  I am 99.9% sure I stopped all services, but that would perhaps be a cause?

Any comments guys?

[BostonTechGuy · Accepted solution]

In this case, we ended up rebuilding the vCenter and migrating everything over.  Wish had better update but in the end thats what was done.

Thanks

BostonTechGuy

[virtualtech_wor]

Hi - The article you shared has helped me. Great! Thank you so much.

[tliperth]

I'm having the same issue after a P-V.

I'm moving a Physical 2008 R2 SP1 server to a Virtual (ESX5.5) standalone host.

I stopped all the VMware services and SQL services before P-V but I get the same problem - vCentre service refuses to start but all the other services do start (including SSO).

Did anyone come up with a better solution than - re-install it all again?

Is the STS-root certificate something that can be "fixed" if that is the issue?