VMware Cloud Community
johill
Contributor
Contributor
‎10-10-2020 09:08 PM

ESXi 6.7 host fails with PSOD，pls help to check

It casued 3 times recently,and here is the log I have：

2012-01-04T11:14:21.190Z cpu0:2097642) [45m [33;1mVMware ESXi 6.7.0 [Releasebuild-10764712 x86_64] [0m

#GP Exception 13 in world 2097642:tq:tcpip4 @ 0x41803015d764

2012-01-04T11:14:21.190Z cpu0:2097642)cr0=0x8001003d cr2=0x7ffbb1a9fda4 cr3=0xc8c22000 cr4=0x216c

2012-01-04T11:14:21.190Z cpu0:2097642)frame=0x451a0a09bdf0 ip=0x41803015d764 err=0 rflags=0x10202

2012-01-04T11:14:21.190Z cpu0:2097642)rax=0x1c430940b15040 rbx=0x417fcde20e40 rcx=0x0

2012-01-04T11:14:21.190Z cpu0:2097642)rdx=0x0 rbp=0x4180301af8d8 rsi=0x430940b15040

2012-01-04T11:14:21.190Z cpu0:2097642)rdi=0x417fcde20e40 r8=0x4519c0000de8 r9=0xffffffffffffffff

2012-01-04T11:14:21.190Z cpu0:2097642)r10=0x3fc00000 r11=0x1 r12=0x0

2012-01-04T11:14:21.190Z cpu0:2097642)r13=0x13f0fda r14=0x0 r15=0x4180301af8d8

2012-01-04T11:14:21.190Z cpu0:2097642)pcpu:0 world:2097642 name:"tq:tcpip4" (S)

2012-01-04T11:14:21.190Z cpu0:2097642)pcpu:1 world:2099323 name:"vmx-mks:ros" (U)

2012-01-04T11:14:21.190Z cpu0:2097642)@BlueScreen: #GP Exception 13 in world 2097642:tq:tcpip4 @ 0x41803015d764

2012-01-04T11:14:21.190Z cpu0:2097642)Code start: 0x41802f400000 VMK uptime: 2:10:04:59.424

2012-01-04T11:14:21.190Z cpu0:2097642)0x451a0a09beb0:[0x41803015d764]callout_reset@(tcpip4)#<None>+0x90 stack: 0x16

2012-01-04T11:14:21.191Z cpu0:2097642)0x451a0a09bee0:[0x41803015da09]callout_timer@(tcpip4)#<None>+0x23e stack: 0xffffffffffffffff

2012-01-04T11:14:21.191Z cpu0:2097642)0x451a0a09bf30:[0x41802f43dcbd]VmkTimerQueueWorldFunc@vmkernel#nover+0x27e stack: 0x0

2012-01-04T11:14:21.191Z cpu0:2097642)0x451a0a09bfe0:[0x41802f709112]CpuSched_StartWorld@vmkernel#nover+0x77 stack: 0x0

2012-01-04T11:14:21.193Z cpu0:2097642)base fs=0x0 gs=0x418040000000 Kgs=0x0

2012-01-04T11:14:21.193Z cpu0:2097642)vmkernel             0x0 .data 0x0 .bss 0x0

2012-01-04T11:14:21.193Z cpu0:2097642)chardevs             0x41802fb31000 .data 0x417fc0000000 .bss 0x417fc0000440

2012-01-04T11:14:21.193Z cpu0:2097642)user                 0x41802fb39000 .data 0x417fc0400000 .bss 0x417fc0410a40

2012-01-04T11:14:21.193Z cpu0:2097642)procfs               0x41802fc34000 .data 0x417fc0a00000 .bss 0x417fc0a00240

2012-01-04T11:14:21.193Z cpu0:2097642)lfHelper             0x41802fc37000 .data 0x417fc0e00000 .bss 0x417fc0e03500

2012-01-04T11:14:21.193Z cpu0:2097642)vsanapi              0x41802fc3d000 .data 0x417fc1200000 .bss 0x417fc1203600

2012-01-04T11:14:21.193Z cpu0:2097642)vsanbase             0x41802fc55000 .data 0x417fc1600000 .bss 0x417fc16101a0

2012-01-04T11:14:21.193Z cpu0:2097642)vprobe               0x41802fc6a000 .data 0x417fc1a00000 .bss 0x417fc1a11d00

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_mgmt          0x41802fcb7000 .data 0x417fc1e00000 .bss 0x417fc1e00200

2012-01-04T11:14:21.193Z cpu0:2097642)iodm                 0x41802fcbd000 .data 0x417fc2200000 .bss 0x417fc2200128

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_4_0_0_mgmt_shim 0x41802fcc2000 .data 0x417fc2600000 .bss 0x417fc26002e8

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_3_0_0_mgmt_shim 0x41802fcc3000 .data 0x417fc2a00000 .bss 0x417fc2a00180

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_2_0_0_mgmt_shim 0x41802fcc4000 .data 0x417fc2e00000 .bss 0x417fc2e001a0

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_2_0_0_vmkernel_shim 0x41802fcc5000 .data 0x417fc3200000 .bss 0x417fc320c9c0

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_4_0_0_vmkernel_shim 0x41802fccd000 .data 0x417fc3600000 .bss 0x417fc3612bc0

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_3_0_0_vmkernel_shim 0x41802fcd3000 .data 0x417fc3a00000 .bss 0x417fc3a0ff20

2012-01-04T11:14:21.193Z cpu0:2097642)vmkbsd               0x41802fcdc000 .data 0x417fc3e00000 .bss 0x417fc3e07100

2012-01-04T11:14:21.193Z cpu0:2097642)vmkusb               0x41802fd25000 .data 0x417fc4200000 .bss 0x417fc4207a40

2012-01-04T11:14:21.193Z cpu0:2097642)ne1000               0x41802fd9d000 .data 0x417fc4600000 .bss 0x417fc4603100

2012-01-04T11:14:21.193Z cpu0:2097642)vmw_ahci             0x41802fdf2000 .data 0x417fc4a00000 .bss 0x417fc4a005c0

2012-01-04T11:14:21.193Z cpu0:2097642)iscsi_trans          0x41802fe13000 .data 0x417fc4e00000 .bss 0x417fc4e01740

2012-01-04T11:14:21.193Z cpu0:2097642)iscsi_trans_compat_shim 0x41802fe27000 .data 0x417fc5200000 .bss 0x417fc5201148

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_3_0_0_vmklinux_shim 0x41802fe28000 .data 0x417fc5600000 .bss 0x417fc56017c4

2012-01-04T11:14:21.193Z cpu0:2097642)vmkplexer            0x41802fe29000 .data 0x417fc5a00000 .bss 0x417fc5a00260

2012-01-04T11:14:21.193Z cpu0:2097642)vmklinux_9           0x41802fe2e000 .data 0x417fc5e00000 .bss 0x417fc5e08f80

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_2_0_0_iscsiInc_shim 0x41802fec8000 .data 0x417fc6200000 .bss 0x417fc6200850

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_3_0_0_iscsiInc_shim 0x41802fecc000 .data 0x417fc6600000 .bss 0x417fc6600850

2012-01-04T11:14:21.193Z cpu0:2097642)etherswitch          0x41802fed1000 .data 0x417fc6a00000 .bss 0x417fc6a17f80

2012-01-04T11:14:21.193Z cpu0:2097642)portcfg              0x41802ff1f000 .data 0x417fc6e00000 .bss 0x417fc6e16f40

2012-01-04T11:14:21.193Z cpu0:2097642)vswitch              0x41802ff3d000 .data 0x417fc7200000 .bss 0x417fc7200640

2012-01-04T11:14:21.193Z cpu0:2097642)netsched_fifo        0x41802ff8e000 .data 0x417fc7600000 .bss 0x417fc7600060

2012-01-04T11:14:21.193Z cpu0:2097642)netsched_hclk        0x41802ff90000 .data 0x417fc7a00000 .bss 0x417fc7a03ec0

2012-01-04T11:14:21.193Z cpu0:2097642)netioc               0x41802ffa0000 .data 0x417fc7e00000 .bss 0x417fc7e000a0

2012-01-04T11:14:21.193Z cpu0:2097642)lb_netqueue_bal      0x41802ffa7000 .data 0x417fc8200000 .bss 0x417fc8200388

2012-01-04T11:14:21.193Z cpu0:2097642)vmklinux_9_2_3_0     0x41802ffae000 .data 0x417fc8600000 .bss 0x417fc8608ad8

2012-01-04T11:14:21.193Z cpu0:2097642)cnic_register        0x41802ffb1000 .data 0x417fc8a00000 .bss 0x417fc8a001e0

2012-01-04T11:14:21.193Z cpu0:2097642)vmklinux_9_2_2_0     0x41802ffb3000 .data 0x417fc8e00000 .bss 0x417fc8e08798

2012-01-04T11:14:21.193Z cpu0:2097642)r8168                0x41802ffb6000 .data 0x417fc9200000 .bss 0x417fc9200380

2012-01-04T11:14:21.193Z cpu0:2097642)dm                   0x41803000f000 .data 0x417fc9600000 .bss 0x417fc9600000

2012-01-04T11:14:21.193Z cpu0:2097642)nmp                  0x418030012000 .data 0x417fc9a00000 .bss 0x417fc9a04630

2012-01-04T11:14:21.193Z cpu0:2097642)hpp                  0x418030043000 .data 0x417fc9e00000 .bss 0x417fc9e02bf8

2012-01-04T11:14:21.193Z cpu0:2097642)vmw_satp_local       0x41803004d000 .data 0x417fca200000 .bss 0x417fca200020

2012-01-04T11:14:21.193Z cpu0:2097642)vmw_satp_default_aa  0x418030051000 .data 0x417fca600000 .bss 0x417fca600000

2012-01-04T11:14:21.193Z cpu0:2097642)vmw_psp_lib          0x418030053000 .data 0x417fcaa00000 .bss 0x417fcaa00290

2012-01-04T11:14:21.193Z cpu0:2097642)vmw_psp_fixed        0x418030055000 .data 0x417fcae00000 .bss 0x417fcae00000

2012-01-04T11:14:21.193Z cpu0:2097642)vmw_psp_rr           0x418030058000 .data 0x417fcb200000 .bss 0x417fcb200060

2012-01-04T11:14:21.193Z cpu0:2097642)vmw_psp_mru          0x41803005e000 .data 0x417fcb600000 .bss 0x417fcb600000

2012-01-04T11:14:21.193Z cpu0:2097642)vmci                 0x418030060000 .data 0x417fcba00000 .bss 0x417fcba084c0

2012-01-04T11:14:21.193Z cpu0:2097642)healthchk            0x41803008b000 .data 0x417fcbe00000 .bss 0x417fcbe15ca0

2012-01-04T11:14:21.193Z cpu0:2097642)teamcheck            0x4180300a6000 .data 0x417fcc200000 .bss 0x417fcc2161c0

2012-01-04T11:14:21.193Z cpu0:2097642)vlanmtucheck         0x4180300bd000 .data 0x417fcc600000 .bss 0x417fcc616000

2012-01-04T11:14:21.193Z cpu0:2097642)heartbeat            0x4180300d8000 .data 0x417fcca00000 .bss 0x417fcca160c0

2012-01-04T11:14:21.193Z cpu0:2097642)shaper               0x4180300f0000 .data 0x417fcce00000 .bss 0x417fcce17e10

2012-01-04T11:14:21.193Z cpu0:2097642)lldp                 0x418030109000 .data 0x417fcd200000 .bss 0x417fcd200050

2012-01-04T11:14:21.193Z cpu0:2097642)cdp                  0x41803010f000 .data 0x417fcd600000 .bss 0x417fcd617300

2012-01-04T11:14:21.193Z cpu0:2097642)ipfix                0x41803012d000 .data 0x417fcda00000 .bss 0x417fcda16700

2012-01-04T11:14:21.193Z cpu0:2097642)tcpip4               0x41803014d000 .data 0x417fcde00000 .bss 0x417fcde18140

2012-01-04T11:14:21.193Z cpu0:2097642)dvsdev               0x4180302a6000 .data 0x417fce200000 .bss 0x417fce200030

2012-01-04T11:14:21.193Z cpu0:2097642)dvfilter             0x4180302a9000 .data 0x417fce600000 .bss 0x417fce600ac0

2012-01-04T11:14:21.193Z cpu0:2097642)lacp                 0x4180302ce000 .data 0x417fcea00000 .bss 0x417fcea00140

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_4_0_0_dvfilter_shim 0x4180302dd000 .data 0x417fcee00000 .bss 0x417fcee009e8

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_2_0_0_dvfilter_shim 0x4180302de000 .data 0x417fcf200000 .bss 0x417fcf2009f0

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_3_0_0_dvfilter_shim 0x4180302df000 .data 0x417fcf600000 .bss 0x417fcf6009e8

2012-01-04T11:14:21.193Z cpu0:2097642)crypto_fips          0x4180302e0000 .data 0x417fcfa00000 .bss 0x417fcfa019a0

2012-01-04T11:14:21.193Z cpu0:2097642)esxfw                0x41803030e000 .data 0x417fcfe00000 .bss 0x417fcfe16bc0

2012-01-04T11:14:21.193Z cpu0:2097642)dvfilter-generic-fastpath 0x418030329000 .data 0x417fd0200000 .bss 0x417fd02162b0

2012-01-04T11:14:21.193Z cpu0:2097642)vmkibft              0x41803034a000 .data 0x417fd0600000 .bss 0x417fd06039c0

2012-01-04T11:14:21.193Z cpu0:2097642)vmkfbft              0x41803034e000 .data 0x417fd0a00000 .bss 0x417fd0a02b60

2012-01-04T11:14:21.193Z cpu0:2097642)vfat                 0x418030351000 .data 0x417fd0e00000 .bss 0x417fd0e02820

2012-01-04T11:14:21.193Z cpu0:2097642)lvmdriver            0x41803035d000 .data 0x417fd1200000 .bss 0x417fd1203a40

2012-01-04T11:14:21.193Z cpu0:2097642)deltadisk            0x41803037e000 .data 0x417fd1600000 .bss 0x417fd1607ec0

2012-01-04T11:14:21.193Z cpu0:2097642)vdfm                 0x4180303b9000 .data 0x417fd1a00000 .bss 0x417fd1a001c0

2012-01-04T11:14:21.193Z cpu0:2097642)gss                  0x4180303be000 .data 0x417fd1e00000 .bss 0x417fd1e02b18

2012-01-04T11:14:21.193Z cpu0:2097642)vmfs3                0x4180303e6000 .data 0x417fd2200000 .bss 0x417fd2207c40

2012-01-04T11:14:21.193Z cpu0:2097642)sunrpc               0x41803050e000 .data 0x417fd2600000 .bss 0x417fd2603b40

2012-01-04T11:14:21.193Z cpu0:2097642)vmklink_mpi          0x41803052c000 .data 0x417fd2a00000 .bss 0x417fd2a02600

2012-01-04T11:14:21.193Z cpu0:2097642)swapobj              0x418030532000 .data 0x417fd2e00000 .bss 0x417fd2e032f8

2012-01-04T11:14:21.193Z cpu0:2097642)nfsclient            0x41803053c000 .data 0x417fd3200000 .bss 0x417fd3204340

2012-01-04T11:14:21.193Z cpu0:2097642)nfs41client          0x41803055e000 .data 0x417fd3600000 .bss 0x417fd3605700

2012-01-04T11:14:21.193Z cpu0:2097642)pciPassthru          0x4180305ce000 .data 0x417fd3a00000 .bss 0x417fd3a02f00

2012-01-04T11:14:21.193Z cpu0:2097642)vflash               0x4180305de000 .data 0x417fd3e00000 .bss 0x417fd3e03740

2012-01-04T11:14:21.193Z cpu0:2097642)procMisc             0x4180305ea000 .data 0x417fd4200000 .bss 0x417fd4200000

2012-01-04T11:14:21.193Z cpu0:2097642)nrdma                0x4180305eb000 .data 0x417fd4600000 .bss 0x417fd4617e40

2012-01-04T11:14:21.193Z cpu0:2097642)nrdma_vmkapi_shim    0x418030633000 .data 0x417fd4a00000 .bss 0x417fd4a012b8

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_4_0_0_rdma_shim 0x418030634000 .data 0x417fd4e00000 .bss 0x417fd4e01370

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_3_0_0_rdma_shim 0x418030635000 .data 0x417fd5200000 .bss 0x417fd5200fc0

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_3_0_0_nmp_shim 0x418030637000 .data 0x417fd5600000 .bss 0x417fd5600df0

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_4_0_0_nmp_shim 0x418030638000 .data 0x417fd5a00000 .bss 0x417fd5a00df0

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_2_0_0_nmp_shim 0x418030639000 .data 0x417fd5e00000 .bss 0x417fd5e00d68

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_4_0_0_iscsi_shim 0x41803063a000 .data 0x417fd6200000 .bss 0x417fd6201240

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_3_0_0_iscsi_shim 0x41803063b000 .data 0x417fd6600000 .bss 0x417fd6600970

2012-01-04T11:14:21.193Z cpu0:2097642)vmkapi_v2_2_0_0_iscsi_shim 0x41803063c000 .data 0x417fd6a00000 .bss 0x417fd6a00970

2012-01-04T11:14:21.193Z cpu0:2097642)vrdma                0x41803063d000 .data 0x417fd6e00000 .bss 0x417fd6e02580

2012-01-04T11:14:21.193Z cpu0:2097642)balloonVMCI          0x41803065e000 .data 0x417fd7200000 .bss 0x417fd7200000

2012-01-04T11:14:21.193Z cpu0:2097642)hbr_filter           0x41803065f000 .data 0x417fd7600000 .bss 0x417fd7600300

2012-01-04T11:14:21.193Z cpu0:2097642)ftcpt                0x418030690000 .data 0x417fd7a00000 .bss 0x417fd7a02fc0

2012-01-04T11:14:21.193Z cpu0:2097642)filtmod              0x4180306dd000 .data 0x417fd7e00000 .bss 0x417fd7e04180

2012-01-04T11:14:21.193Z cpu0:2097642)svmmirror            0x4180306ef000 .data 0x417fd8200000 .bss 0x417fd8200100

2012-01-04T11:14:21.193Z cpu0:2097642)cbt                  0x4180306fc000 .data 0x417fd8600000 .bss 0x417fd86000c0

2012-01-04T11:14:21.193Z cpu0:2097642)migrate              0x4180306ff000 .data 0x417fd8a00000 .bss 0x417fd8a05480

2012-01-04T11:14:21.193Z cpu0:2097642)vfc                  0x41803077f000 .data 0x417fd8e00000 .bss 0x417fd8e02cc0

Coredump to disk.

2012-01-04T11:14:21.243Z cpu0:2097642)Slot 1 of 1 on device t10.ATA_____Netac_SSD_120GB_________________________AA000800080009010575:9.

2012-01-04T11:14:21.243Z cpu0:2097642)Dump: 474: Using dump slot size 2684354560.

2012-01-04T11:14:21.253Z cpu0:2097642)Dump: 2845: Using dump buffer size 98304

search a lot online but still dont understand the reason,hope someone can help me to see if the hardware or software issue.Thanks.

0 Kudos
4 Replies
Lalegre
Virtuoso
Virtuoso
‎10-10-2020 10:21 PM

Could you please provide the details of the server you are using?

  • NICs
  • HBAs
  • CPU Model
  • Hardware Model
0 Kudos
NathanosBlightc
Commander
Commander
‎10-10-2020 11:46 PM

First of all, please check the compatibility of your server with the ESXi v6.7 in VMware Compatibility Guide

Then if you are ensure about its compatibility, check the generated logs of OOB platform (for example iLO for HP Proliant servers) to find any related issue ...

Please mark my comment as the Correct Answer if this solution resolved your problem
0 Kudos
scott28tt
VMware Employee
VMware Employee
‎10-11-2020 01:10 AM

Moderator: Some things for you to note:

1. Thread moved to the ESXi area, this is not an issue with the vSphere Host Client.

2. Screenshots of a PSOD are useful for others to see.

3. The Attach function in the bottom-right of the post creator/editor is the best way of attaching log dumps (rather than a copy/paste).


-------------------------------------------------------------------------------------------------------------------------------------------------------------

Although I am a VMware employee I contribute to VMware Communities voluntarily (ie. not in any official capacity)
VMware Training & Certification blog
0 Kudos
bluefirestorm
Champion
Champion
‎10-11-2020 01:59 AM

Few observations:

The PSOD appears to be networking related.

2012-01-04T11:14:21.190Z cpu0:2097642)@BlueScreen: #GP Exception 13 in world 2097642:tq:tcpip4 @ 0x41803015d764

2012-01-04T11:14:21.190Z cpu0:2097642)0x451a0a09beb0:[0x41803015d764]callout_reset@(tcpip4)#<None>+0x90 stack: 0x16

2012-01-04T11:14:21.191Z cpu0:2097642)0x451a0a09bee0:[0x41803015da09]callout_timer@(tcpip4)#<None>+0x23e stack: 0xffffffffffffffff

The system appears to be using a Realtek 8168 NIC which is not the VMware HCL.

2012-01-04T11:14:21.193Z cpu0:2097642)r8168                0x41802ffb6000 .data 0x417fc9200000 .bss 0x417fc9200380

The real time clock is off. It appears that the system powered up with a date of 01 Jan 2012.

2012-01-04T11:14:21.190Z cpu0:2097642)Code start: 0x41802f400000 VMK uptime: 2:10:04:59.424

Suggest fix the clock problem (possibly as simple as replacing a CR2032 battery) and set a proper system time. But it could also be a sign that there is already a problem with the motherboard as usually even if the battery is flat, the real time clock (RTC) on the motherboard can tick normally so long as it is plugged in with power.

It is best to use a NIC that is in the VMware HCL.

https://www.vmware.com/resources/compatibility/search.php?deviceCategory=io

0 Kudos