Registration causes Virtual Center to crash?

[davesherman]

Yesterday afternoon I scp'd a virtual machine from one ESX farm to another. Once the copy was completed I registed the VM (through datastore browser) and immediately experienced a Virtual Center crash - error message was "Connection to the server has been lost.The application will now exit". Our Virtual Center server is a Dell PE2850 running VC2.02 v.50618 with a SQL 2K5 database running locally. The error in the VC log was:

Win32SEHTranslator: Access Violation (0xc0000005) ebp: 0x358ed24, eip: 0x573a5b

exception: Assert failed!

Backtrace:(backtraces not supported)

CoreDump: Writing minidump

Restarting the VC service and rebooting the VC server resulted in the error "Connection refused". I disabled the VC servers network connection and was able to start the Virtual Center application. The application started with all ESX (version 3.02 52542) servers disconnected naturally, so I enabled the VC server's network connection and started connecting the ESX servers one at a time until I found the offending server. It was the server I had just attempted to register the virtual machine I had just copied over. I restarted the VC agent on that ESX server, but had the same result - once I tried to reconnect the ESX server to Virtual Center, the VC service would crash and I would have to restart the VC service. I then un-registered the virtual machine in question and Virtual Center would start normally with all ESX servers connected. Once I registered the virtual machine (by vmware-cmd this time) it crashes VC immediately. Has anyone seen this issue? It appears to me that the issue is on the ESX side when it tries to send data to VC at Virtual Center start-up. Fortunately this occured in our Test environment and I was able to spend some time on it. Unfortunately, I had this exact issue on our Production environment (ESX 3.01/VC 2.01on SQL) two months ago and opened a SR, only to be told by VMware that it was a corrupt VC database and to re-install/re-initialize VC. Having experienced this issue twice in the last couple of months, I was wondering if other folks have had issues with registering vm's or issues with Virtual Center crashing as a result of trying to register a vm. Also, any info where I could find any extended error information from the ESX server side.

[boydd]

Have you tried registering the vm with just a local connection to the host using the VIC? Then VC might pick things up normally. Verify the file permissions for the VM.

DB

[davesherman]

Hi Boydd,

Thank you for your reply. I am able to register the Virtual Machine with the ESX host (vmware-cmd -l shows the machine registered), however, registration of the vm crashes Virtual Center no matter what method I use. If I create a new vm through VC and point to the original disk file, I am fine. So the registration piece is not really my concern, it is that once Virtual Center crashes, it will no longer start or run as long as the vm in question is registered. I am a bit worried that something this minor crashes Virtual Center. Our production environment is fairly busy (70 ESX hosts ~500 vms and growing rapidly) and it could be a pain to try and figure out what ESX host is bringing down Virtual Center. I am also nervous that I have been hit by this bug on two different environments, running different VC versions, so I am curious if other folks are seeing this issue? Any suggestions as to what would cause this and how we would eliminate the issue? This really has me re-thinking my VC architecture as an enterprise application and I may have to consider deploying more VC boxes to mitigate potential downtime.

[Jae_Ellers]

My advice is to unregister (which you've undoubtedly done by now to keep VC running). Move the vmx file to a backup name or location. Create a new VM based on the disk files.

I think this crash happens when the config file is wonky. I had the same error and VMware support thrashed about for over 24 hours while my VC wouldn't run. I gave up on them and read the logs, found the assert, traced it in the db to the offending system and unregistered it in 10 minutes the next morning.