VMware

This Question is Possibly Answered

1 "correct" answer available (10 pts)
10 Replies Last post: Jul 10, 2008 2:06 PM by weinstein5  

ESX host disconnected, web access fail. posted: Jun 15, 2008 7:49 PM

Click to view ckboon's profile Enthusiast
ckboon
124 posts since
Oct 31, 2006
Hi,

One of my ESX host shows up as disconnected when viewed by the VCMS. But I can ping the host and all the VMs running under that host as well as ssh to that host. However, I cannot control the VMs or the host via VI client or browser. "Netstat -an" does not list port 80/8080/443, so that means the web server is dead.

I have tired to restart the following without success.
1) S85vmware-webAccess
2) S98mgmt-vmware
3) S99vmware-vpxa

Short of shutting down all the VMs and reboot the host, what else can I do to restart at least the web server?

Thanks for the help.

Regards,

Re: ESX host disconnected, web access fail.

1. Jun 15, 2008 7:55 PM in response to: ckboon
Click to view jdvcp's profile Enthusiast
jdvcp
57 posts since
Feb 21, 2007

I have run into similar scenarios, all related to too little service console memory (272 MB defaut). You can see in esxtop if physical mem is too low or if swap mem is too full. Also, look at swap in/out activity. check /var/log/messages for any processes which were killed. check /var/log/vmksummary - go to bottom, then all the way to the right. It will show you top 3 processes in use. If you see VMAP, this is buggy code and this is a runaway process eating up mem. Kill it, then restart the services as you did before.

If you have any agents running in the service console, consider turning them off (stop or kill service/processes). This could free up enough memory for you to turn other critical services back on.

Have vmware support walk you through correct service bounce/turn-up procedures to get this host back on VC.

Re: ESX host disconnected, web access fail.

2. Jun 15, 2008 8:35 PM in response to: jdvcp
Click to view ckboon's profile Enthusiast
ckboon
124 posts since
Oct 31, 2006
Not much via /var/log/messages & /var/log/vmksummary.
I tired to use vmware-cmd -l and it gave me this error:
"root@esx04 /# vmware-cmd -l
/usr/bin/vmware-cmd: Could not connect to server
(VMControl error -3: Invalid arguments: Login failed.)
root@esx04 /#"

It looks like some major error. Anyway, I have to reboot the host in order to increase the service console memory to 384MB.

Re: ESX host disconnected, web access fail.

3. Jun 15, 2008 8:45 PM in response to: ckboon
Click to view weinstein5's profile Guru
weinstein5
6,351 posts since
Nov 19, 2005
If you are resizing your service console memory - do not forget to resize he service console swap file to two times the service console to if you set the memory to 364 the service console will have to be at least 768

Re: ESX host disconnected, web access fail.

4. Jun 15, 2008 9:05 PM in response to: weinstein5
Click to view ckboon's profile Enthusiast
ckboon
124 posts since
Oct 31, 2006

Thanks for the heads-up.

I checked my swap size:
root@esx04 etc# free -m
Swap: 1027 4 1023

I have 1GB of swap space. That ought to be enough.


Re: ESX host disconnected, web access fail.

5. Jun 17, 2008 5:22 AM in response to: ckboon
Click to view ckboon's profile Enthusiast
ckboon
124 posts since
Oct 31, 2006
I have rebooted my ESX host and I still can't connect to it via VI Client.
When I tired to use the VI Client connect wizard using IP / hostname, I get an error "Unable to access the specific host..."
However, I can ssh to the host though.

I have tired to restart "mgmt-vmware" but it keeps dying after a while.
  1. /etc/rc3.d/S98mgmt-vmware status
vmware-hostd is stopped

I have also tired to add "AAMClient" to the firewall permit list, but it doesn't seem to take effect.

What can I do?

Re: ESX host disconnected, web access fail.

6. Jun 17, 2008 5:42 AM in response to: ckboon
Click to view ckboon's profile Enthusiast
ckboon
124 posts since
Oct 31, 2006
Found out while the stupid hostd keeps dying.

I made a backup copy of the "/etc/vmware/firewall/services.xml" in the same directory. So hostd / firewall keeps trying to load both copies.
Removed the backup copy, restart "S98mgmt-vmware".

HA is now running, I can connect via VI Client, etc.
Duh!

Re: ESX host disconnected, web access fail.

7. Jul 10, 2008 1:04 PM in response to: ckboon
Click to view Brian.Wing's profile Novice
Brian.Wing
21 posts since
Oct 16, 2007

How does one go about resizing the swap partition on this badboy? It looks like I didnt' allocate enough memory for the service console, and I've modified that (haven't rebooted yet tho)

Thanks

brian

Re: ESX host disconnected, web access fail.

8. Jul 10, 2008 1:26 PM in response to: Brian.Wing
Click to view weinstein5's profile Guru
weinstein5
6,351 posts since
Nov 19, 2005
in this thread there is discussion on how adjust the swap space - http://communities.vmware.com/message/484615

Re: ESX host disconnected, web access fail.

9. Jul 10, 2008 1:36 PM in response to: weinstein5
Click to view Brian.Wing's profile Novice
Brian.Wing
21 posts since
Oct 16, 2007

Thanks for this, I'm guessing though that by allowing the partitioning to be default at install I may have painted myself into a corner. I'm not super keen on the idea of trying a "file" swap space, I may just end up rebuilding an existing ESX server and manually partitioning.

Do you have any recommendations on partition layout? Unfortunatly we're using internal disk for VMs and won't be moving to SAN storage for a while. I do have ample disk space locally 1.5T or so.

Re: ESX host disconnected, web access fail.

10. Jul 10, 2008 2:06 PM in response to: Brian.Wing
Click to view weinstein5's profile Guru
weinstein5
6,351 posts since
Nov 19, 2005
yep painted inot a corner - I always setup my partions with a swap space of 2 GB so that if I need to increase service console memory I just reset the memory and reboot the host - the rest of of the default partition sizes are fine -

Actions

VMware Developer

SDKs, APIs, Videos, Learn and much more in the Developer community.

Learn More

Developer Sample Code

Increase your developer productivity with VMware API sample code.

Learn More

VMworld Sessions & Labs

Online access to the latest VMworld Sessions & Labs and online services.

Learn more

Purchase PSO Credits Online

Purchase credits to redeem training and consulting services online.

Buy Now

Community Hardware Software

View reported configurations or report your own.

Learn More

VMware vSphere

Come witness the next giant leap in virtualization.

Register Today

Communities