VSphere Esxi 4.1.0 can't login with vSphere client and can't start hostd daemon

[lcmm]

Hi Folks,

I have the following issue:

I Cant Login with vSphere Client.

Infos:

Hardware, Dell R310 (hack) (http://www1.la.dell.com/br/pt/empresa/Servidores/poweredge-r310/pd.aspx?refid=poweredge-r310&cs=brbs...)

Soft: vSphere Esxi 4.1.0 Build 260247

        vSphere Client 4.1.0 Build 258902

Sintoms:

I can: ping and ssh

I can't: connect throw vsphere cliente, https://XX.XX.XX.XX and http://XX.XX.XX.XX

Logs:

I have some "panic"  tags on my log:

Apr 20 00:07:25 Hostd: [2011-04-20 00:07:25.866 FFBFBE90 info 'Proxysvc'] Plugin started

Apr 20 00:07:25 Hostd: [2011-04-20 00:07:25.866 FFBFBE90 panic 'App'] error: Config Value not a boolean

Apr 20 00:07:25 Hostd: [2011-04-20 00:07:25.866 FFBFBE90 panic 'App'] backtrace:

Apr 20 00:07:25 Hostd: [00] rip 1e9808b3

Apr 20 00:07:25 Hostd: [01] rip 1e812cae

Apr 20 00:07:25 Hostd: [02] rip 1e7b2682

Apr 20 00:07:25 Hostd: [03] rip 1e79fbcd

Apr 20 00:07:25 Hostd: [04] rip 03dcc2d5

Apr 20 00:07:25 Hostd: [05] rip 03dcc663

Apr 20 00:07:25 Hostd: [06] rip 03dc73df

Apr 20 00:07:25 Hostd: [07] rip 1e7cd503

Apr 20 00:07:25 Hostd: [08] rip 1e7d4a89

Apr 20 00:07:25 Hostd: [09] rip 1e7cb643

Apr 20 00:07:25 Hostd: [10] rip 041abc02

Apr 20 00:07:25 Hostd: [11] rip 041a1af3

Apr 20 00:07:25 Hostd: [12] rip 041b0445

Apr 20 00:07:25 Hostd: [13] rip 1fda3f0c

Apr 20 00:07:25 Hostd: [14] rip 036bd491

Apr 20 00:07:25 Hostd: [2011-04-20 00:07:25.867 43981B90 warning 'Proxysvc Req00000'] Connection to localhost:8307 failed with error N7Vmacore15SystemExceptionE(Connection refused).

Apr 20 00:07:25 watchdog-hostd: 'hostd ++min=0,swap,group=hostd /etc/vmware/hostd/config.xml' exited after 8 seconds (quick failure 2) 255

Apr 20 00:07:25 watchdog-hostd: End 'hostd ++min=0,swap,group=hostd /etc/vmware/hostd/config.xml', failure limit reached

With some google help, i think i have something wrong on config.xml of hostd (/etc/vmware/hostd/config.xml).

I look for some "strange things" on config.xm, like "###" as i saw on google, but i can't figure out what is!!!!!

Also i can't start hostd daemon. Next, more infos:

~ # services.sh restart

Running sfcbd-watchdog stop

Running usbarbitrator stop

watchdog-usbarbitrator: Terminating watchdog with PID 731672

usbarbitrator stopped.

Running wsman stop

Stopping openwsmand

Openwsmand is not running.

Running slpd stop

Stopping slpd

Running hostd stop

**** VSI_GetInstanceListAlloc : No cartel by that name

VSI_GetInstanceListAlloc : No cartel by that name

watchdog-hostd: PID file /var/run/vmware/watchdog-hostd.PID not found

watchdog-hostd: Unable to terminate watchdog: Can't find process

sh: cannot kill pid 732106: No such process

Running lbtd stop

watchdog-net-lbt: Terminating watchdog with PID 727348

net-lbt stopped.

Running sensord stop

watchdog-sensord: Terminating watchdog with PID 678174

sensord stopped.

Running storageRM stop

watchdog-storageRM: Terminating watchdog with PID 678152

storageRM module stopped.

Running vobd stop

watchdog-vobd: Terminating watchdog with PID 731354

Vobd stopped.

Running vprobed stop

watchdog-vprobed: Terminating watchdog with PID 727236

vprobed stopped.

Running TSM-SSH stop

Stopping tech support mode ssh server

Running netlogond stop

Stopping Likewise Site Affinity Service...ok

Running TSM stop

Hiding TSM login

Running DCUI stop

Disabling DCUI logins

Running ntpd stop

Stopping ntpd

Running lwiod stop

Stopping Likewise IO Manager Service...ok

Running lsassd stop

Stopping Likewise Identity and Authentication Service...ok

Running lsassd restart

Starting Likewise Identity and Authentication Servicetouch: /var/lock/subsys/lsassd: No such file or directory

...ok

Running lwiod restart

Starting Likewise IO Manager Servicetouch: /var/lock/subsys/lwiod: No such file or directory

...ok

Running ntpd restart

Starting ntpd

Running DCUI restart

Enabling DCUI login: runlevel =

Running TSM restart

Displaying TSM login: runlevel =

Running netlogond restart

Starting Likewise Site Affinity Servicetouch: /var/lock/subsys/netlogond: No such file or directory

...ok

Running TSM-SSH restart

Starting tech support mode ssh server

Running vprobed restart

vprobed started.

Running vobd restart

Vobd started.

Running storageRM restart

storageRM module started.

Running sensord restart

sensord started.

Running lbtd restart

net-lbt started.

Running hostd restart

Running slpd restart

[735440] Begin 'hostd ++min=0,swap,group=hostd /etc/vmware/hostd/config.xml', min-uptime = 60, max-quick-failures = 1, max-total-failures = 1000000

Starting slpd

Running wsman restart

Starting openwsmand

Running usbarbitrator restart

Rescanning all adapters..

usbarbitrator started.

Running sfcbd-watchdog restart

I really apreciate if someone can help me, becaus it's a production server, up and running about 6 months.

I have more logs, but it's big! so, it's attached above!

Sorry about my bad english.

Greets from Brazil.

Luiz Claudio Maia

[MauroBonder]

Check if this kb helps http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=101427...

check too if you need open any port in your firewall to access your enviroment http://kb.vmware.com/kb/1012382

outra recomendação é que acesse via web browser o IP do host e atualize seu vSphere Client. (Aliás se fosse eu faria essa etapa antes de tudo).

Faça essas validações e volte a postar os resultados. Já já encontraremos a solução. ...rsrs

abcs

Message was edited by: MauroBonder

[lcmm]

Thanks mauro!

yes i already check both KB.

I can't view the Esxi web page, neither vSphere.

My only managment choice is ssh and localy.

the daemon Hostd can't start, it is for about 5 seconds, and die!

[MauroBonder]

você já tentou restartar os serviços ? e realizou o procedimento do KB Article: 1014270 ?

services.sh restart ?

verificou se está correto as informações de ip e velocidade da placa de rede...essas questoes relacionadas a rede, está ok ?

[opbz]

check the ammount of free space you have on your  ESX partitions.

seen this error when I was running out of space on / ended up deleting loads of files /var then services where able to start for me.

[lcmm]

Thanks guys,

In my first post should have a output from services.sh restart.

My partitions spaces are ok, not full.

i was trying to fix without restart the server, because the vm's was on, and it is a production server.

about 4 days trying to fix it without reboot, i rebooted, them the problem continue, and guess what, vm's does't startup, GREAt!

With some client pression and a lot of phone calls, I decided do jump to "PLAN Z", repair the instalation and reconfigure the LACP, vlans, ipaddress, license, password and import vlan.....

So, the server is now ok, but i don't know why hostd stoped and can't start.

Justo to register: I checked the .xml file inside /etc/vmware/hostd folder, aparently OK, also compared these files with the another (running) esxi host (usin diff) and everything is ok.

Thanks everyone!

[MauroBonder]

pode excutar o services.sh restart que não impacta na parada de maquina virtual.

so ira restartar os servicos de gerenciamento do host. NAO INTERFERE NAS VMS.

[opbz]

yes you can restart those services... it will not affect your running vms...

though you will get disconnect messages on virtual center about the host... those can be ignored

[kevinbrztowski]

Is this a fresh install with Update 1? Do you have a qlogic card installed??