cannot connect to ESXi host

[Tcpuniverse]

Hi;

First of all I'm completely newbie in Vmware. I was working on my project and suddenly the power went down and came back again. after that, I cannot connect to my ESXi 6.0 host. I can ping and even can connect remotely by SSH. But neither Vsphere Client nor HTTP/HTTPS work. I searched the net and found that "hostd" and "vpxa" are not running. I started the processes but after some seconds they stopped automatically. Here are commands:

[root@timaz-esxi:~] /etc/init.d/hostd status

hostd is not running.

[root@timaz-esxi:~] /etc/init.d/hostd start

Ramdisk 'hostd' with estimated size of 303MB already exists

hostd started.

[root@timaz-esxi:~] /etc/init.d/hostd status

hostd is running.

[root@timaz-esxi:~] /etc/init.d/hostd status

hostd is not running.

[root@timaz-esxi:~] /etc/init.d/vpxa status

vpxa is not running

[root@timaz-esxi:~] /etc/init.d/vpxa start

[root@timaz-esxi:~] /etc/init.d/vpxa status

vpxa is running

.

.

.

after a minute:

.

[root@timaz-esxi:~] /etc/init.d/vpxa status

vpxa is not running

[root@timaz-esxi:~] /etc/init.d/hostd status

hostd is not running.

I checked the log files and here they are:

------------------------------------------------------------------------------------------------------------------------------

[root@timaz-esxi:~] tail -f /var/log/vmkernel.log

2018-06-22T10:21:55.669Z cpu1:32788)NMP: nmp_ThrottleLogForDevice:3298: Cmd 0x1a (0x439dc081ee80, 0) to dev "mpx.vmhba1:C0:T0:L0" on path "vmhba1:C0:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE

2018-06-22T10:22:54.617Z cpu2:34300)NMP: nmp_ThrottleLogForDevice:3298: Cmd 0x85 (0x439dc08a3c80, 34300) to dev "naa.600508e000000000e86e9fee026aaa01" on path "vmhba0:C1:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE

2018-06-22T10:22:54.617Z cpu2:34300)ScsiDeviceIO: 2651: Cmd(0x439dc08a3c80) 0x4d, CmdSN 0x5 from world 34300 to dev "naa.600508e000000000e86e9fee026aaa01" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.

2018-06-22T10:22:54.618Z cpu2:34300)NMP: nmp_ThrottleLogForDevice:3298: Cmd 0x1a (0x439dc08a3c80, 34300) to dev "naa.600508e000000000e86e9fee026aaa01" on path "vmhba0:C1:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0. Act:NONE

2018-06-22T10:22:54.618Z cpu2:34300)ScsiDeviceIO: 2651: Cmd(0x439dc08a3c80) 0x1a, CmdSN 0x6 from world 34300 to dev "naa.600508e000000000e86e9fee026aaa01" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

2018-06-22T10:22:54.619Z cpu2:34300)NMP: nmp_ThrottleLogForDevice:3298: Cmd 0x85 (0x439dc08a3c80, 34300) to dev "naa.600508e000000000e86e9fee026aaa01" on path "vmhba0:C1:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE

2018-06-22T10:26:55.659Z cpu1:32861)NMP: nmp_ThrottleLogForDevice:3298: Cmd 0x12 (0x439dc0873940, 0) to dev "naa.600508e000000000e86e9fee026aaa01" on path "vmhba0:C1:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0. Act:NONE

2018-06-22T10:26:55.669Z cpu2:32789)NMP: nmp_ThrottleLogForDevice:3298: Cmd 0x1a (0x439dc0873940, 0) to dev "mpx.vmhba1:C0:T0:L0" on path "vmhba1:C0:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE

2018-06-22T10:31:55.658Z cpu2:32952)NMP: nmp_ThrottleLogForDevice:3298: Cmd 0x12 (0x439dc0886940, 0) to dev "naa.600508e000000000e86e9fee026aaa01" on path "vmhba0:C1:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x0 0x0. Act:NONE

2018-06-22T10:31:55.670Z cpu0:32787)NMP: nmp_ThrottleLogForDevice:3298: Cmd 0x1a (0x439dc0886940, 0) to dev "mpx.vmhba1:C0:T0:L0" on path "vmhba1:C0:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE

------------------------------------------------------------------------------------------------------------------------------

[root@timaz-esxi:~] tail -f /var/log/vmkwarning.log

0:00:00:00.000 cpu0:1)WARNING: Serial: 648: Invalid serial port config: mem-mapped to addr 0x0.

2018-06-22T09:52:05.016Z cpu1:33097)WARNING: LinuxSignal: 541: ignored unexpected signal flags 0x2 (sig 17)

2018-06-22T09:52:30.431Z cpu2:33185)WARNING: ScsiScan: 1643: Failed to add path vmhba0:C0:T0:L0 : Not found

2018-06-22T09:52:31.466Z cpu2:33185)WARNING: ScsiScan: 1643: Failed to add path vmhba0:C0:T1:L0 : Not found

2018-06-22T09:52:33.319Z cpu2:33049)WARNING: NetDVS: 659: portAlias is NULL

2018-06-22T09:52:43.643Z cpu1:33353)WARNING: Supported VMs 128, Max VSAN VMs 400, SystemMemoryInGB 40

2018-06-22T09:52:43.643Z cpu1:33353)WARNING: MaxFileHandles: 3840, Prealloc 1, Prealloc limit: 32 GB, Host scaling factor: 1

2018-06-22T09:52:43.643Z cpu1:33353)WARNING: DOM memory will be preallocated.

2018-06-22T09:52:45.552Z cpu1:33409)WARNING: FTCpt: 476: Using IPv6 address to start server listener

2018-06-22T09:53:32.872Z cpu2:33042)WARNING: LinuxThread: 340: Error cloning thread: -28 (bad0081)

------------------------------------------------------------------------------------------------------------------------------

And this is what I got while trying to connect to the ESXi host with Vsphere Client:

I have many VMs on that ESXi host. How can I fix the issue or get my VMs from that ESXi host? tnx.

[IT_pilot]

Do you use vSAN? Could it be that there ended the place? A lack of space can affect the inaccessibility of ESXi.

[Tcpuniverse]

no I don't use vSAN. ESXi host has a local disk.

[Tcpuniverse]

interestingly I upgraded it from 6.0 to 6.5 but the same problem is still there; That I can connect to the host via SSH but not via HTTP/HTTPS/Vsphere Client.

[vijayrana968]

You won't able to connect ESXI 6.5 with traditional vSphere c# client. In browser you have to connect with https://ipaddressofEsxiHost/ui

[Tcpuniverse]

It's impossible to connect to my ESXi host via HTTP/HTTPS; As I said, I can only connect to it via SSH.

[Finikiez]

Can you attach hostd.log file here so I could check?

[golddiggie]

Can you get into the BIOS/UEFI of the host and check to make sure all the hardware is still working properly?? Having power pulled hard (and in the wrong way) can cause actual hardware damage. Depending on the make/model of the host server, it should include some kind of memory test item (usually runs on boot), if you had disabled that, turn it back on and let it do a full check. Then go through the drive controller, and the rest of the hardware (including checking to make sure the hard drives are kosher).

IF everything comes back clean, you might have to do a wipe and reinstall of ESXi on the host.

Assuming this is a standalone host without a vCenter Server connected to it... Right?

[Tcpuniverse · Accepted solution]

None of actions helped me. I finally decided to install a clean version of ESXi while preserving the existing VMFS and it worked! After setup, I re-register the existing VM to the new ESXi host.