VMware Cloud Community
Deas
Contributor
Contributor
‎07-17-2007 01:16 AM

virtualcenter server not starting after upgrade!

hello,

we just upgraded to the latest version (VMware-vpx-2.0.1-40644) and now the virtualcenter server service is not starting. the error message in the vpxd log file is:

'App' 2032 error] VM has invalid name

i searched the forum and the internet but found nothing about this error message. a repair won´t fix the problem!

please help!!!

brgds Deas

0 Kudos
23 Replies
masaki
Virtuoso
Virtuoso
‎07-17-2007 03:18 AM

Did you follow this recommendation?

http://virtrix.blogspot.com/2007/03/vmware-beware-patch-2.html

0 Kudos
masaki
Virtuoso
Virtuoso
‎07-17-2007 03:20 AM

I hope you didn' erase the DB.

If you succeed with VC service start look at Ha and DR configuration if you didn't disable them before.

Message was edited by:

masaki

0 Kudos
Deas
Contributor
Contributor
‎07-17-2007 03:39 AM

hello,

i clicked NO on the question about the existing virtual center repository!

and it was working short after the upgrade because i logged in immediately after and saw that all vmware hosts upgraded to the latest agent version. i also installed the latest client on all management workstations.

but a day later or so the service was not running any longer and when i try to start it i get this strange error message in the log...

here is the complete log when i start the service:

-cut-

Log for VMware VirtualCenter, pid=2368, version=2.0.1, build=build-40644, option=Release, section=2

\[2007-07-17 12:41:39.593 'App' 2340 info] Current working directory: C:\WINDOWS\system32

\[2007-07-17 12:41:39.593 'App' 2340 info] Initializing SSL context

\[2007-07-17 12:41:40.577 'App' 2340 info] Vmacore::InitSSL: doVersionCheck = true, handshakeTimeoutUs = 120000000

\[2007-07-17 12:41:40.608 'BaseLibs' 2340 info] NFC connection accept timeout: 180000 milliseconds

\[2007-07-17 12:41:40.608 'BaseLibs' 2340 info] NFC request timeout: 180000 milliseconds

\[2007-07-17 12:41:40.608 'BaseLibs' 2340 info] NFC read timeout: 60000 milliseconds

\[2007-07-17 12:41:40.608 'BaseLibs' 2340 info] NFC write timeout: 600000 milliseconds

\[2007-07-17 12:41:40.624 'App' 2340 info] Starting VMware VirtualCenter 2.0.1 build-40644

\[2007-07-17 12:41:40.624 'App' 2340 info] Account name: SYSTEM

\[2007-07-17 12:41:40.624 'App' 2340 info] \[VpxOsLayer] Enabled low-frag process heap.

\[2007-07-17 12:41:40.624 'App' 2340 info] \[VpxOsLayer] Enabled low-frag crt heap.

\[2007-07-17 12:41:40.624 'App' 2340 info] \[VpxLRO] 32 max LROs

\[2007-07-17 12:41:40.624 'App' 2340 info] \[VpxLRO] 6 reserved internal LROs

\[2007-07-17 12:41:40.624 'App' 2340 info] \[VpxLRO] 6 reserved blocker LROs

\[2007-07-17 12:41:40.624 'App' 2340 info] \[VpxLRO] 6 reserved short LROs

\[2007-07-17 12:41:40.624 'App' 2340 info] \[VpxLRO] 2 reserved long LROs

\[2007-07-17 12:41:40.624 'App' 2340 info] \[VpxLRO] 600-second task lifetime

\[2007-07-17 12:41:47.328 'Locale' 2340 info] Locale subsystem initialized from C:\Program Files\VMware\VMware VirtualCenter 2.0\locale/ with default locale en.

\[2007-07-17 12:41:48.672 'App' 2340 error] VM has invalid name

\[2007-07-17 12:41:48.781 'App' 2340 error] Invalid VM. Removing /vpx/vm/#458/

\[2007-07-17 12:41:48.781 'App' 2340 error] \[VpxdInvt] Delete failed for /vpx/vm/#458/

\[2007-07-17 12:41:48.812 'App' 2340 error] \[VpxdMain] Failed to initialize: vim.fault.DatabaseError

\[2007-07-17 12:41:48.812 'App' 2340 error] Failed to intialize VMware VirtualCenter. Shutting down...

-cut-

brgds Deas

Message was edited by:

Deas

0 Kudos
VirtualNoitall
Virtuoso
Virtuoso
‎07-17-2007 04:29 AM

Hello,

Definitely looks like a database issue. Is your database full or have you run out of space on your DB server?

What database engine are you using? and at what service pack is it?

Do you have any errors in your database server log?

0 Kudos
Deas
Contributor
Contributor
‎07-17-2007 04:57 AM

hello,

we are running it on a local sql 2005 sp1 server (i think) it is build 9.0.1406.

db service is running, db is set to unrestricted growth and we have a few gig left on the hdd.

i also checked the odbc settings - i am able to connect to the database without any problems.

i also checked some tables of my db and it looks ok. the "VC_Database.VPX_HOST" table e.g. contains all our esx servers. so the db looks ok.

one stupid question: where can i check what odbc user/pw is used by virtual center?!?

brgds Deas

0 Kudos
masaki
Virtuoso
Virtuoso
‎07-17-2007 05:10 AM

This has been choosen at VC installation time.

I hope you wrote it somewhere.

0 Kudos
masaki
Virtuoso
Virtuoso
‎07-17-2007 05:15 AM

It seems a db corruption or uncorrect data.

2007-07-17 12:41:48.672 'App' 2340 error] VM has invalid name

\[2007-07-17 12:41:48.781 'App' 2340 error] Invalid VM. Removing /vpx/vm/#458/

\[2007-07-17 12:41:48.781 'App' 2340 error] \[VpxdInvt] Delete failed for /vpx/vm/#458/

\[2007-07-17 12:41:48.812 'App' 2340 error] \[VpxdMain] Failed to initialize: vim.fault.DatabaseError

\[2007-07-17 12:41:48.812 'App' 2340 error] Failed to intialize VMware VirtualCenter. Shutting down...

You must find vm 458 and eventually think to remove from DB.

0 Kudos
Deas
Contributor
Contributor
‎07-17-2007 05:51 AM

hello,

no, the odbc user/pw is no problem.

what i forgot to mention is that the initial setup was not done by me.

there are 2 users on the sql server - one is named VC_Admin and the other VC_Database. setup is only working when i do it with the VC_Admin user. the VC_Database user is not able to make it. i´m not sure why my boss (who´s on vacation at the moment...) did this.

but as i already told you - after the upgrade all services were running fine and everything was working. so i don´t understand why this happened...

i checked now the eventlog completely and found this. i did the upgrade on 13th evening and it was running until 15th 3am local time. then the service terminated unexpectedly, this came in the app eventlog and the problem started:

-cut-

The description for Event ID ( 1000 ) in Source ( VMware VirtualCenter Server ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Check database connectivity before restarting. Error: Error\[VdbODBCError] (-1) "ODBC error: (40001) - \[Microsoft]\[ODBC SQL Server Driver]\[SQL Server]Transaction (Process ID 53) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction." is returned when executing SQL statement "UPDATE VPX_ENTITY SET NAME = ? , TYPE_ID = ? , PARENT_ID = ? WHERE ID = ?".

-cut-

brgds Deas

0 Kudos
Deas
Contributor
Contributor
‎07-17-2007 06:20 AM

hello,

can you explain "You must find vm 458" with a little bit more details?

in what table(s) must i search for what exactly?

would a recovery of the db before the crash help? what would we loose if no configuration changes were done?

brgds Deas

0 Kudos
VirtualNoitall
Virtuoso
Virtuoso
‎07-17-2007 07:45 AM

Hello,

Is the db on the VC server? I would try rebooting or restarting SQL if you have that options.

0 Kudos
Deas
Contributor
Contributor
‎07-17-2007 08:02 AM

hello,

yes, the db is local on the vc and i already restartet the sql server.

brgds Deas

0 Kudos
masaki
Virtuoso
Virtuoso
‎07-17-2007 08:11 AM

Yes,

try a db restore. You could loose some info.

There was a DB fault so DB may be corrupted or not aligned.

0 Kudos
Deas
Contributor
Contributor
‎07-17-2007 08:38 AM

hello,

no way to check the db and eliminate the corrupt entries? a restore is the last thing i´d like to do...

btw.: anybody ever heard of a vc service crash that corrupted the db?

brgds Deas

0 Kudos
masaki
Virtuoso
Virtuoso
‎07-17-2007 08:40 AM

Try looking for an entry 458 on VPX_VM table or something similar

VirtualNoitall
Virtuoso
Virtuoso
‎07-17-2007 08:57 AM

before attempting this I would stop VCMS and backup your database in case the restore does not work or you can reference some important info.

I might also open a support ticket before resorting to a restore. There maybe be a known issue or an easy fix

0 Kudos
Deas
Contributor
Contributor
‎07-17-2007 10:25 AM

hello,

here is another strange thing i noticed:

i have 2 tables named VPX_VM

dbo.VPX_VM and VC_Database.VPX_VM

is this normal? i found entry 458 in the dbo.VPX_VM

brgds Deas

0 Kudos
VirtualNoitall
Virtuoso
Virtuoso
‎07-17-2007 11:03 AM

That is not normal. It sounds like there are two VirtualCenter installs to the same database under two different user accounts. This is pretty messy. How big is your environment? How about starting from scratch?

You can try re-setup/reinstall your VirtualCenter server using the original db account to see what happens; likely it was a windows trusted user such as the local admin?

0 Kudos
Deas
Contributor
Contributor
‎07-17-2007 11:19 AM

hello,

i was lucky and my boss who did the setup called me back. Smiley Happy

the duplicate tables are from a earlier migration which went wrong. the dbo.VPX_* are the actual and correct ones.

he also told me that we are only loosing performance data and i can easily delete the db if i need to. the side effect would be that our db would be clean then... Smiley Wink

we have 10 vmware servers with about 120 virtual ones.

in the meantime i checked the table and found the entry mentioned above. i asked our db support team to insert the name of the server missing into the table because it is "Null" at the moment. let´s see if this helps...

brgds Deas

0 Kudos
masaki
Virtuoso
Virtuoso
‎07-17-2007 01:13 PM

So Deas my hints were at least helpful or not?

0 Kudos