BSOD on Workstation Pro 15.5.6 (running RHEL 7.7 Guest).

[gbohn]

Hi;

I just got a BSOD running a Windows 10 Host (1909). I was running a RHEL 7.7 Linux Guest full-screen at the time. (I had just returned from a break).

I got a mini-dump and a memory.dmp.

Running windbg against the memory.dmp calls out what looks to the casual observer like Workstation driver and process references:

"FAULTING_PROCESSOR: 6

PROCESS_NAME:  vmware-vmx.exe

FAULTING_THREAD:  ffffdd86e5270080"

and

"

STACK_TEXT: 

ffff8603`bfddf468 fffff801`4f778228 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : vmx86+0x100a

ffff8603`bfddf470 fffff801`4f77abdb : 00000000`00000000 00000000`00000000 ffff8603`bfddf938 00000000`00000000 : vmx86+0x8228

ffff8603`bfddf5b0 fffff801`4f771f13 : 00000000`000001b0 00000000`00000001 ffff8603`bfddf938 ffffdd87`01c65430 : vmx86+0xabdb

ffff8603`bfddf680 fffff801`4f77293e : 00000000`00000000 fffff801`391543a9 00000000`00000000 fffff801`38f132d4 : vmx86+0x1f13

ffff8603`bfddf860 fffff801`394b2a2b : 00000000`000003f8 fffff801`38f13269 00000000`00000000 ffffdd86`fab5d7a0 : vmx86+0x293e

ffff8603`bfddf8c0 fffff801`394b22f6 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!IopXxxControlFile+0x71b

ffff8603`bfddf9e0 fffff801`38fd3c18 : 00000000`746c6644 ffff8603`bfddfa00 00000000`00000000 ffff8f85`bb997c40 : nt!NtDeviceIoControlFile+0x56

ffff8603`bfddfa50 00007ffe`f23dc154 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x28

00000073`dc9ff5c8 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x00007ffe`f23dc154

STACK_COMMAND:  .thread 0xffffdd86e5270080 ; kb

SYMBOL_NAME:  vmx86+100a

MODULE_NAME: vmx86

IMAGE_NAME:  vmx86.sys

BUCKET_ID_FUNC_OFFSET:  100a

FAILURE_BUCKET_ID:  CLOCK_WATCHDOG_TIMEOUT_INTERRUPTS_DISABLED_vmx86!unknown_function"

I'll attach the entire !analyze -v output. (I'm only familiar enough with WinDbg to produce the !analyze output at this time, so I'm not sure what else would be helpful).

Is this an issue with the workstation driver? The system has an E5-1650 V4 Xeon CPU, for what its worth.

Thanks;

[continuum]

CLOCK_WATCHDOG_TIMEOUT (101)

An expected clock interrupt was not received on a secondary processor in an

MP system within the allocated interval. This indicates that the specified

processor is hung and not processing interrupts.

That sounds like your CPU is having problems

[gbohn]

From the googling this error so far, I've seen some references that seemed to say that in general this BSOD could be either hardware or driver induced.

"...processor is hung and not processing interrupts."

So, when the analyze output shows

"CLOCK_WATCHDOG_TIMEOUT_INTERRUPTS_DISABLED_vmx86!unknown_function"

would that mean that interrupts had been disabled by something?

I get the impression that the output says it was in the VMware driver (and exe) at the time...

But not knowing anything about how that's supposed to work (in the VMware driver) it might be helpful if someone from VMWare could take a look as well.

I see a few others reporting a similar issue, but no definitive resolution there (one way or the other)... At Windows 10 crashed with bugcheck at vmx86.sys and Blue Screen caused by vmx86.sys VM Workstation 15.5.2

Just in case it's related my system has 12 virtual cores (6 real ones). I have 4 cores assigned to the Guest.

At least one of the other two reports I mentioned seems to show that they have at least up to processor 0xa (10).

Thanks.

[continuum]

try to boot the VM with just a single cpu

[gbohn]

> try to boot the VM with just a single cpu

The system is working at the moment. But I'll try setting a single core to see if that makes any difference in the future.

Two weeks ago it also had a BSOD (also while running Workstation with the same guest), but it didn't create a memory.dmp (or mini-dump) file so I couldn't analyze it that time.

On the hardware front, I've tried several types of 'stress' tests with no issues detected.

*) Multiple runs of the 'Intel Processor Diagnostic Tool' and it always passes.

*) I ran Passmark Memtest 86 for 4 full passes (over 11 hours using all 6 of the 'real' cores) without it detecting a problem.

*) I ran Prime 95 for 4 hours without issue.

Both times the system wasn't heavily loaded when it hit the BSOD. (The guest is used as a workstation).

[ChaosEnergy]

same issue here 
after i installed vmware workstation 16 now i have bsod instead of some days per week, i know have daily crashes and it seems to be always around the same time

[gbohn]

In my case (on V 15.5.6 since I don't have V 16), the BSODs always indicated a trap in the VMWare code.

You probably need to look at your memory dump file to see what specifically it is unhappy about.

As an update to my situation, replacing the CPU and Motherboard did not fix the issue.  I had a few more BSODs. (I have an Intel Xeon E5 1650 v4 cpu which isn't exactly main-stream in case that's related).

I changed a bunch of settings after that and haven't  had a BSOD in 9 weeks now (knock on wood).

I'm slowly going back and restoring the original settings one at a time to see what might have helped.

One of the things I did was to uncheck the Guests settings for Processors/ 'Virtualize Intel VT-X/EPT' and 'Virtualize CPU Performance counters'. I had these checked for some reason even though I didn't actually need them.

I mention this in case you have these checked on your guest.

[tanshen_daniel]

Hi ChaosEnergy, near time i happend same problem . so i want understand you new progress. My vmware version is 16 too.