nettech1
Expert
Expert

503 Service Unavailable Failed to connect to endpoint:

Jump to solution

Hi,

One of our ESXi 6.0 hosts was reporting back to vcenter and was accepting connections from the fat client, however connections to the host directly via https were being immediately redirected back to the log in page with "Failed to load host summary" error in all browsers.

SSH works

[root@esxi:/var/log] df -h

Filesystem   Size   Used Available Use% Mounted on

NFS          5.2T   2.7T      2.5T  52% /vmfs/volumes/nfs

VMFS-5       3.5T   1.3T      2.2T  38% /vmfs/volumes/ds

vfat       285.8M 205.6M     80.3M  72% /vmfs/volumes/5cc70836-eecb7702-d4b7-20677cd872b8

vfat       249.7M 205.6M     44.1M  82% /vmfs/volumes/94a5dae7-ce13fd61-32f5-239e1c8e6234

vfat       249.7M   4.0K    249.7M   0% /vmfs/volumes/1ddca3e4-be359492-f0b6-a8dbffd34bb7

after we attempting to restart hostd and vpxa we are seeing "503 Service Unavailable" instead of the log in page via HTTPS, esxi is no longer showing up as connected to vcenter, but we can still SSH to the host.

[root@esxi:/var/log] esxcli vm process list

Connect to localhost failed: Connection failure.

[root@esxi:~] /etc/init.d/hostd restart

watchdog-hostd: Terminating watchdog process with PID 34438 hostd stopped. Ramdisk 'hostd' with estimated size of 747MB already exists hostd started.

[root@esxi:~] /etc/init.d/vpxa restart

watchdog-vpxa: Terminating watchdog process with PID 35047 vpxa stopped.

vdf shows that I am out of space in the tmp directory

Ramdisk              1k-blocks      Used Available Use% Mounted on

root                     32768       240     32528   0% --

etc                      28672       232     28440   0% --

opt                      32768       396     32372   1% --

var                      49152       972     48180   1% --

tmp                     262144    262144         0 100% --

iofilters                32768         0     32768   0% --

hostdstats              764928      4720    760208   0% --

also hostd.log indicates a problem with the space as well

2020-05-16T11:43:52.603Z info hostd[FFF9EB70] [Originator@6876 sub=Vimsvc.ha-eventmgr]

Event 1 : The ramdisk 'tmp' is full.  As a result, the file /tmp/.SwapInfoHLS.lock.LOCK.9565543 could not be written.

2020-05-16T11:43:52.604Z panic hostd[FFC90B20] [Originator@6876 sub=Default] backtrace:

Any help is appreciated

0 Kudos
1 Solution
2 Replies
nettech1
Expert
Expert

for anyone having this issue on the HP server with a custom HP esxi image, the workaround is posted here

https://www.teimouri.net/ramdisk-tmp-full-vmware-esxi-hpe-proliant/

if anyone finds a solution please let us know.

Thanks

0 Kudos