VMware Cloud Community
NorbK
Enthusiast
Enthusiast

ESX3.5U1 Service Console Crash - VMs Still Running (HP Management Agents v8.0)

I've run into my second unique machine that has had the service console completely crash. Drops out of VC, can't SSH, at the server's console screen you can type in root for a logon but once you hit enter, it never gets to the password prompt. All VMs are running however, so it appears the hypervisor is still working. These were fresh installs of U1. The only additional software is an Altiris client for Linux and the HP Proliant Pack (management agents) version 8.0. The Service Console memory allocation was bumped to 800mb when U1 was installed. Is there a problem with memory leaks or something with the version 8 of the HP agents?
Has anyone else run into this scenario?
Reply
0 Kudos
6 Replies
rgardnerS1
Contributor
Contributor

We had something very similar happen with one of our servers. Dropped from VC, couldn't ssh to it, and the console would freeze once you hit enter. Only difference was there was a single scsi error printed to the console, nothing in the logs though. When you'd finally reboot it would all come back up fine. I changed syslog to log to another remote system and it ended up being that there were scsi errors and it was locking the filesystem, hence it wasn't getting logged originally. Ended up being a bad backplane/cable we replaced both at the same time so I'm not certain which it was. Though with this being 2 seperate machines both having the same problem I don't know if this is any help.

Good luck,

rgardnerS1
Contributor
Contributor

Sorry for the double post

Message was edited by: rgardnerS1

Reply
0 Kudos
ablej
Hot Shot
Hot Shot

You might want to check this post out from Duncan http://www.yellow-bricks.com/2008/08/27/why-i-dislike-agents-in-my-service-console/






David Strebel

www.holy-vm.com

If you find this information useful, please award points for "correct" or "helpful"

David Strebel www.david-strebel.com If you find this information useful, please award points for "correct" or "helpful"
NorbK
Enthusiast
Enthusiast

I've seen the yellow bricks article. While I concur that its looking more and more like ESX should be "pristine" on the servers, it appears that this happens without the management agents as well on HP servers. I'm trying to narrow down exactly what is the culprit since I'm not getting what the yellow bricks article posted for errors and I'm on U1 not U2.

Reply
0 Kudos
NorbK
Enthusiast
Enthusiast

We had something very similar happen with one of our servers. Dropped from VC, couldn't ssh to it, and the console would freeze once you hit enter. Only difference was there was a single scsi error printed to the console, nothing in the logs though. When you'd finally reboot it would all come back up fine. I changed syslog to log to another remote system and it ended up being that there were scsi errors and it was locking the filesystem, hence it wasn't getting logged originally. Ended up being a bad backplane/cable we replaced both at the same time so I'm not certain which it was. Though with this being 2 seperate machines both having the same problem I don't know if this is any help.

Yes, it almost appears like hardware since there has been no trace in the logs, but as you mentioned, these are two separate machines which while it could happen, would seem unlikely. Your symptoms though, are identical. I haven't seen anything before that takes out the SC like this...

Reply
0 Kudos
Texiwill
Leadership
Leadership

Hello,

I happen to see this all the time..... Look for processes within the SC in 'D' state or defunct. THis is generally caused by something pretty bad happening. I would fully vet the hardware as that is the most likely culprit.


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
Reply
0 Kudos