ESXI 7.0 Unexpected Hypervisor Reset

[mustafapasha]

Hi Folks,

We are facing "Unexpected Hypervisor Reset " In ESXI 7.0 , We also checked PSOD which set to 0 by default.

So any one please help me out in this issue, in finding the resolution to this issue.

Thanks.

[ashilkrishnan]

Hi @mustafapasha ,

If you are experiencing frequent resets, it could also be a hardware issue.

Does the host reboot or shuts down and a manual power on is required? 

Reviewing /var/log/vmkernel and /var/log/hostd.logs of that time should provide more information

[mustafapasha]

Hi Ashil,

Thanks for your quick reply,

The ESXI host is rebooting automatically, 

And also previously it use to reboot once in 2 months , But recently ESXI host unexpectedly reboot happen twice in a week.

As suggested i will check the logs and get back to u ASAP.

Regards.

[scott28tt]

What did VMware support say?

[msripada]

I suggest engage hardware vendor to perform thorough hardware diagnostics. If the unexpected restart is not a PSOD, then I assume there might be underlying hardware issue. If its a PSOD, request you to share screenshot if possible or raise SR with GSS.

thanks,

MS

[nachogonzalez]

Did you check VMkernel log?
HAve you checked driver/firmware versions?
Have you checked Hardware compatibility before going to vSphere 7.0?

[mustafapasha]

Hi Folks,

We are facing "Unexpected Hypervisor Restart " In ESXI 7.0 , We also checked PSOD which set to 0 by default.

These are the logs in esxupdate.log after server rebooted

When we checked esxupdate.log we are getting following logs. These are few logs i am sharing with you

2001-07-13T06:06:04Z esxupdate: 2097811: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/esxcfg-advcfg', '-q', '-g', '/UserVar
2001-07-13T06:06:04Z esxupdate: 2097811: vmware.runcommand: INFO: runcommand called with: args = '['/sbin/esxcfg-advcfg', '-q', '-g', '/UserVar
2001-07-13T06:06:04Z esxupdate: 2097811: root: INFO: Command = vib.list
2001-07-13T06:06:04Z esxupdate: 2097811: root: INFO: Options = {'depot': None, 'softwarespec': None, 'baseimageversion': None, 'dryrun': False,
2001-07-13T06:06:04Z esxupdate: 2097811: HostImage: INFO: Installer <class 'vmware.esximage.Installer.BootBankInstaller.BootBankInstaller'> was
2001-07-13T06:06:04Z esxupdate: 2097811: HostImage: INFO: Installers initiated are {'live': <vmware.esximage.Installer.LiveImageInstaller.LiveI

Can you also anyone please tell with any of the virtual machines in the esxi server also we may face this issue of hypervisor restart unexpectedly.

So any one please help me out in this issue, in finding the resolution to this issue.

Thanks.

[ashilkrishnan]

@mustafapasha ,

Need to check vmkernel and hostd.logs. If you don't see them in /var/log, you can check /var/run/log directory

[mustafapasha]

@ashilkrishnan 

Hi Ashil,

Thanks for your quick reply
These are only first few lines of the logs
At the time server rebooted these are the logs i got in vmkernel.log. I am sharing it with you.

2001-07-13T06:05:25.000Z cpu0:2097152)SysInitTable: 69: Finished sysInit step: NTPClock_Init in 1375 us.
2001-07-13T06:05:25.000Z cpu0:2097152)Initializing InitVMKernel: FDS_Init ...
2001-07-13T06:05:25.001Z cpu0:2097152)FDS: 441: Registered FDS Driver genscsi
2001-07-13T06:05:25.001Z cpu0:2097152)FDS: 441: Registered FDS Driver disks
2001-07-13T06:05:25.003Z cpu0:2097152)FSS: 968: Initialized file system switch to support 65536 [not clamped] file handles
2001-07-13T06:05:25.007Z cpu0:2097152)FSS: 1153: Registered fs vcfs, module 0, fsTypeNum 0xbcf5
2001-07-13T06:05:25.008Z cpu0:2097152)VisorFSRam: 878: root with (32,32,4096,21504,755)
2001-07-13T06:05:25.009Z cpu0:2097152)VisorFSRam: 878: etc with (28,28,1024,4096,755)
2001-07-13T06:05:25.009Z cpu0:2097152)VisorFSRam: 878: opt with (0,32,1024,8192,755)
2001-07-13T06:05:25.009Z cpu0:2097152)VisorFSRam: 878: var with (5,48,128,8192,755)
2001-07-13T06:05:25.009Z cpu0:2097152)FSS: 1153: Registered fs visorfs, module 0, fsTypeNum 0x1
2001-07-13T06:05:25.009Z cpu0:2097152)FSS: 1153: Registered fs devfs, module 0, fsTypeNum 0xdef5
2001-07-13T06:05:25.013Z cpu0:2097152)FDS: 441: Registered FDS Driver char
2001-07-13T06:05:25.013Z cpu0:2097152)FDS: 441: Registered FDS Driver cdrom
2001-07-13T06:05:25.013Z cpu0:2097152)SysInitTable: 69: Finished sysInit step: FDS_Init in 12982 us.
2001-07-13T06:05:25.013Z cpu0:2097152)Initializing InitVMKernel: VSCSI_Init ...
2001-07-13T06:05:25.015Z cpu0:2097152)VSCSIFilter: 1157: Successfully created directory for VSCSI-Filter character devices
2001-07-13T06:05:25.017Z cpu0:2097152)CpuSched: 815: user latency of 2097546 VSCSIPoll 0 changed by 2097152 bootstrap -6
2001-07-13T06:05:25.017Z cpu0:2097152)SysInitTable: 69: Finished sysInit step: VSCSI_Init in 4775 us.
2001-07-13T06:05:25.017Z cpu0:2097152)Initializing InitVMKernel: VSCSI_ResetInit ...
2001-07-13T06:05:25.019Z cpu0:2097152)SysInitTable: 69: Finished sysInit step: VSCSI_ResetInit in 1032 us.
2001-07-13T06:05:25.019Z cpu0:2097152)Initializing InitVMKernel: Storage_LateInit ...
2001-07-13T06:05:25.020Z cpu0:2097152)StoragePsaDriver: 129: STORAGE PSA Driver - Heap Initialized.
2001-07-13T06:05:25.020Z cpu0:2097152)SysInitTable: 69: Finished sysInit step: Storage_LateInit in 1202 us.
2001-07-13T06:05:25.020Z cpu0:2097152)Initializing InitVMKernel: SCSI_LateInit ...
2001-07-13T06:05:25.021Z cpu0:2097152)CpuSched: 815: user latency of 2097562 Storage-APD 0 changed by 2097152 bootstrap -6
2001-07-13T06:05:25.022Z cpu0:2097152)ScsiPeriodicProbe: 374: SCSI periodic probe interval changed to 300 Sec.
2001-07-13T06:05:25.022Z cpu0:2097152)ScsiPeriodicProbe: 374: SCSI periodic probe interval changed to 300 Sec.
2001-07-13T06:05:25.024Z cpu0:2097152)SysInitTable: 69: Finished sysInit step: SCSI_LateInit in 3807 us.
2001-07-13T06:05:25.024Z cpu0:2097152)Initializing InitVMKernel: NVME_LateInit ...
2001-07-13T06:05:25.027Z cpu0:2097152)NvmeDiscover: 1862: NVMe periodic probe interval changed to 60 Sec.
2001-07-13T06:05:25.027Z cpu0:2097152)SysInitTable: 69: Finished sysInit step: NVME_LateInit in 3759 us.
2001-07-13T06:05:25.027Z cpu0:2097152)Initializing InitVMKernel: Init_VMKArchNMISetup ...
2001-07-13T06:05:25.028Z cpu0:2097152)SysInitTable: 69: Finished sysInit step: Init_VMKArchNMISetup in 926 us.
2001-07-13T06:05:25.028Z cpu0:2097152)Initializing InitVMKernel: Chipset_InitInterrupts ...
2001-07-13T06:05:25.029Z cpu0:2097152)Chipset: 520: Chipset PCI bus irq routing information for segment 0000

[ashilkrishnan]

2001-07-13. Are these events recent or the date on ESXi host is incorrect? 

[JinuV]

Which Hardware model ?

[mustafapasha]

@ashilkrishnan ,

These are the logs dated back to the day on which ESXI host is incorrect.

[mustafapasha]

@JinuV ,

The hardware model is ESXI 7.0

[JinuV]

 I mean hardware model like HP or Dell

[mustafapasha]

@JinuV 

It is HP

[mustafapasha]

Hi Folks,

We are facing "Unexpected Hypervisor Restart " In ESXI 7.0 , We also checked PSOD which set to 0 by default.

I have attached the vmkernel.log. These are the logs when ESXI host got rebooted.

So any one please help me out in this issue on finding the resolution to this issue.

Thanks.