Diagnostics of a VM hung, Console open but not responding. Ping still works.

[BoyQuiet]

I have a single ESXi (4.1) host on Dell hardware with 16GB main memory running 5 vms.

Four of these on a single switch forming an internal network.

One has stopped responding to Remote Desktop connections, Outgoing email reports from the machine have stopped. But I can still Ping the machine.

Tried telnet to port 3389 and that just hangs on a prompt.

I would like to inspect the machine state, before I force a power cycle.

Last time this happened (different VM similary symptoms) there was nothing untoward in the machine event log after restart.

Have saved the vm*.log files

What else can I do / save to investigate this - before/after a power cycle.

Thanks

[Virtualinfra]

From Virtual Machine side get these details:

What is the OS running on the  Guest?

How frequently the machine get HUNG. Is that its getting HUNG frequently or exact at certian period of time between 9 to 12 some thing like that.

what is applicaiton that is running on that machine.

what is the CPU and memory assigned on that virutal machine.

From ESXi:

What are all the guest os runing on the machine.

how much CPU and Memory are assiged to each machine.

what is the statistics in performance tab, what is the usage of CPU and Memory on teh ESXI host when the VM gets HUNG.

Is the storage is internal or a externa SAN.

Let try to figure out this..

[BoyQuiet]

Hi Virtualinfra

Thanks for your interest :

VM side

OS: Windows XP

Frequency: 7 - 21 days (only twice so far, but same symptoms)

Application: Syslog program EventLogAnalyser and own application to send an email each day.

ESXi

All machines 20%

Virtual Machine TAB Actual usage: Host CPU - Mhz Host Mem - MB Guest Mem %

92 Mhz : 2099 MB : 5%

117 Mhz: 169MB : 25%

46 Mhz : 281 MB : 36%

81 Mhz : 989 MB ; 6% This is the unresponsive machine

Average CPU under 24% Peek < 60%

Unresponsive machine still using CPU

2x 500GB disks - 1 for host (I know its too big 🙂 ) and one as datastore

Hope thats of use

Can you suggest any interventions before I power cycle the machine ?

Thanks

[Virtualinfra]

How frequently you reboot your virtual machine?.....

As its a windows XP machine i would suggest to reboot once a day for better performance.

What is the Count of vCPU 1 or 2? if its 2 reduce it to 1 for better performance..

What is the memory assigned? i would suggest to use 2 GB to 4 GB depening on the applicaiton usage in it.

[a_p_]

Two thoughts on this:

• Double check the physical switch port settings (no port security enabled, i.e. mode desktop)
• Verify whether there's no system with the same IP address in your network

André

[BoyQuiet]

Theses VM's provide services to each other. No users except management.

They are a pre-deployment replication of real hardware.

Those (real) machines run 24x7 amd are / do not need to be rebooted. Only in fact for Windows security updates. There is no noticable performance degredation.

This VM has previously been running 4 weeks. And was restarted after a windows update on 14/10/2011. Worked perfectly till Friday 😞

1 CPU, 1GB memory

Thanks

[BoyQuiet]

Andre Thank you.

Have checked. No change to Network Configuration, VSwitch,  NIC etc from "access OK" to "no response"

Regards

[Virtualinfra]

Please paste the vmware.log file - during the VM Hung.

If you want to run a vm 24/7 it is recommended to use server OS rather than desktop OS.

[BoyQuiet]

Agreed re OS, mission is to replicate existing real machines and "prove" I can dispence with them. It looks good except for these two looss of management in 4 weeks.

Just to be clear I used "reflect" to "store" the real machine, then restored it in the VM , Cleared Windows of the real machine and reregisted windows with microsoft from the VM.

Log attached.

Regards and thanks

[Virtualinfra]

From the logs i am able to find the below.

Error 1;

Virtual machine was trying to restore its state from the suspended stated and failed to come up. That is the reason for HUNG state.

Error 2:-

"Insufficient video RAM. The maximum resolution of the virtual machine will be limited to 1176x885. To use the configured maximum resolution of 2560x1600, increase the amount of video RAM allocated to this virtual machine by setting svga.vramSize="16384000" in the virtual machine's configuration file."

refer KB:- http://kb.vmware.com/kb/1024990

error 3:-

"

USBGL: SETCONFIGURATION=1 failed -1:16:Device or resource busy, work around triggered"

related to USB devices - is this virtual machine is configured to use USB ( I hope if your using, this is possible only with vpshere 4.1 or vsphere 5.0)

refer KB: http://www.vmware.com/support/kb/enduser/std_adp.php?p_faqid=774

What i have understood is this machine is converted from physical to virtual.- If this is correct please follow the below, if you have not done already.

1. Uninstall all the physical machine dirvers from the virtual machine.

2. Perform disk defragment

[BoyQuiet]

Thank you for looking at the log for me.

I sent the current current log. It contains the only action I took in an attempt to regain control. i.e. I suspended then the machine - it had already stopped responding.

I believe all the points you make are valid but I want to be sure I have not inadvertantly mislead you - Sorry if I did.

Just in case I enclose the immidiately preceeding logs that covers the period from power-on (on the 14/10) to the no response state.

Can I learn anything from taking a snapshot, or will it distroy evidence.?

Other points:

The Video resolution was fine for a real machine but is pointless for remote management so I would rather reduce the machine resolution but the display option in windows XP for this vm would not let me reduce the size.

There is a USB camera on the machine but it was reported as unplugged.

Thanks for the tip on uninstalling physical drivers.

Regards