Powered on servers showing as powered off

[dazed1]

Hello all,

I am running VIC 2.0.1 Build 40644 on Virutal infrastructure Version 2.0.1 Build 40644. I have servers that will become either Orphaned or show as powered off and neither scenario is true. If I us VIC to log directly into the server as ROOT it will show the correct status of the server in question. If I look at the same server on my standalone Center server it doesn't.

I am running the database for this on MSSQL 2000, and this is clustered. Wondering if perhaps that is the problem?

Everything thing from the ESX server (3.0.1, build 42829) to the O/S (W2K3 SP1 w/ 8GB RAM) has bee patched. If I remove the server from inventory and then re-add it then it usually will be fine for a few days and then this happens again. If I power clear either the ESX server / host itself or the VCenter server it doesn't make a difference. I do have both HA and DRS enabled and have tried disabling them thinking they may be at fault and there is not any change.

Needless to say this is getting really old really fast, I have been patient long enough as this has been going on for about 3 weeks now.

Does anyone have any thoughts beyond what I have tried?

Thanks!

Mike Snyder

University of Minnesota

m-snyd@umn.edu

[jayolsen]

Anything logged on your Virtual center server itself? check c:\windows\temp\vpx

And on the ESX server itself

/var/log/vmware/vpx and check the vpxa log files.

[esiebert7625]

I would start with the database since it shows OK when the VIC is connected to ESX directly. Get a SQL browser, and connect to the database, the VM information is in the VPX_VM table. There is a column called POWER_STATE and another called GUEST_STATE. Check those for the VM's that are showing the wrong information. If you have SQL clustered maybe that is causing the problem like you mentioned.

You can also use this SQL...

select b.name, guest_state, power_state from vpx_vm a, vpx_entity b where a.id = b.id order by b.name

fyi...if you find this post helpful, please award points using the Helpful/Correct buttons.

[dazed1]

I do see messages like the following;

\[2007-05-01 07:45:31.772 'App' 5573552 warning] ============BEGIN FAILED METHOD CALL DUMP============

\[2007-05-01 07:45:31.772 'App' 5573552 warning] Invoking \[GetResourcePool] on \[vim.VirtualMachine:3984]

\[2007-05-01 07:45:31.772 'App' 5573552 warning] Fault Msg: "The request refers to an object that no longer exists or has never existed."

\[2007-05-01 07:45:31.772 'App' 5573552 warning] ============END FAILED METHOD CALL DUMP============

On the ESX server however this was AFTER doing a system reboot and during the bootup or just after. The latest log from 09:45 today showed;

\[2007-05-01 09:56:01.044 'App' 11246512 warning] \[VpxaHalStats] Unexpected return result. Expect 1 sample, receive 2

\[2007-05-01 10:05:26.585 'Memory checker' 15055792 warning] Current value 80600 exceeds soft limit 76800.

There a "tons" of the mem checker errors... Is there a way to up the value?

mike

[BenConrad]

I've seen this happening to us a lot lately. I will shut down a VM for modification but VC still shows that it's running. However, I can right click and 'edit' and do things that are expected for a powered off VM. After the edits the VM usually shows that it's off and and I can power it back on again.

Ben

[esiebert7625]

Maybe it's a bug in VC or a problem with the SQL database.