VMware Cloud Community
jw1128
Contributor
Contributor

esxcli comand Connect to localhost failed: Connection failure via host ssh shell

My host dropped out of vcenter again, firewall crash suspected, but the comand is just not comming up.

~ # esxcli network firewall get
Connect to localhost failed: Connection failure

I have seen this issue a few times on a couple hosts, and running the command below always resolves and allos the host to reconnect, but for some reason this time the commands are not running, any suggestions other than host reboot, unfortuately host is isolated with critical vms running.

esxcli network firewall set --enabled false

Tags (3)
8 Replies
Virtualinfra
Commander
Commander

Did you try to restart the services, if not try that and see if the command goes fine.

#services.sh restart

check if hostd service status in /etc/init.d/

Thanks & Regards Dharshan S VCP 4.0,VTSP 5.0, VCP 5.0
Reply
0 Kudos
jw1128
Contributor
Contributor

yea multiple times just hangs, at this point its just dead stick.

Reply
0 Kudos
JCMorrissey
Expert
Expert

Hi,

Take a look at http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=100284... - worth going through the troubleshooting steps in that article.

Please consider marking as "helpful", if you find this post useful. Thanks!... http://johncmorrissey.wordpress.com/
Reply
0 Kudos
vmware_tam_neto
Contributor
Contributor

I just had this issue. In our case, problem was that the root filesystem was out of inodes and was related to a bug in sfcbd, but I suppose this can happen anytime "stat -f /" shows "Free: 0" for Inodes...

VMware KB: ESXi 5.x host is disconnected from vCenter Server due to sfcbd exhausting inodes

Reply
0 Kudos
dhanarajramesh

can you provide kernel logs? I hope there might be some reasons. are you able to execute any other esxcli command? if no the only way is need to reboot the esxi. but however we can analyze the root cause

Reply
0 Kudos
sarikrizvi
Enthusiast
Enthusiast

Try below cmds to check/start hostd service

/etc/init.d/hostd status >>>> /etc/init.d/hostd start  

Check hostd status , if it's not started then check /var/log/vmkernel.log , /var/log/hostd.log and /var/log/vpxa.log

When esxcli cmd not working you can run localcli cmd instead of esxcli .

# localcli network firewall get

# localcli network firewall set --enable false

# localcli system maintenanceMode set --enable true

# localcli vm process list

# localcli vm process kill -w <World ID> -t soft     (Shutdown VMs)

Regards,
SARIK (Infrastructure Architect)
vExpert 2018-2020 | vExpert - Pro | NSX | Security
vCAP-DCD 6.5 | vCP-DCV 5.0 | 5.5 | 6.0 | vCA-DCV 5 | vCA-Cloud 5 | RHCSA & RHCE 6 | A+ (HW & NW)
__________________
Please Mark "Helpful" or "Correct" if It'll help you
_____________________________________
@Follow:
Blog# https://vmwarevtech.com
vExpert# https://vexpert.vmware.com/directory/1997
Badge# https://www.youracclaim.com/users/sarik
jmkim15
Contributor
Contributor

Just a nitpick to sarikrizvi's answer,

# localcli network firewall get

# localcli network firewall set --enabled false     # "enabled" instead of "enable"

# localcli system maintenanceMode set --enable true

# localcli vm process list                          # Get the World ID of each VMs

# localcli vm process kill -w <World ID> -t soft    # Shutdown VM

Reply
0 Kudos
Loop_hole
Contributor
Contributor

I know this is an old post but I figured this might help someone in the future. 

My issue was not an ip configuration leading to a duplicate ip conflict. 

My hostd service was not running so I checked the hostd logs

When checking the hostd.log I came across 

error hostd[2098766] [Originator@6876 sub=Solo] Please check the validity of certificate files /etc/vmware/ssl/rui.key and /etc/vmware/ssl/rui.crt

I could of used openssl x509 -in rui.crt -text -noout to do a cert check but instead I winscp to see arbitrary characters to which I concluded became corrupted. I copied the last known host backup key and certificate from /etc/vmware/ssl/rui.bak to their respective locations (/rui.crt & rui.key). Rebooted and voila!

Reply
0 Kudos