WS14 Pro - Regulary BSODs when installed - Probably caused by : vstor2-mntapi20-shared.sys ( vstor2_mntapi20_shared+12bc )

[ColonelPolonel]

Hello,

I've recently bought an AMD 1700X PC so I could create a nested vsphere lab in WS14 pro and complete my Virtual-Jedi training, but as son as I install Workstation Pro, I get regular BSODs.

It seems to happen even when I'm not using Workstation Pro but can be forced with a little patience by rebooting VM's.

I'm using 14.1.1 build-7258167 on Windows 10, 64-bit 10.0.16299

To try and figure out the cause I started analysing the dumps, updated my drivers and BIOS and ran a few mem tests, but the problem has persisted.

After using the verifier tool, Windows wouldn't boot up and the latest dump file seems to point to vstor2-mntapi20-shared.sys.

Any tips on how I can narrow down the actual problem here?

*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck C4, {2000, fffff8010c5812bc, 0, 32545356}

*** WARNING: Unable to verify timestamp for vstor2-mntapi20-shared.sys
*** ERROR: Module load completed but symbols could not be loaded for vstor2-mntapi20-shared.sys
Probably caused by : vstor2-mntapi20-shared.sys ( vstor2_mntapi20_shared+12bc )

Followup:     MachineOwner
---------

0: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

DRIVER_VERIFIER_DETECTED_VIOLATION (c4)
A device driver attempting to corrupt the system has been caught.  This is
because the driver was specified in the registry as being suspect (by the
administrator) and the kernel has enabled substantial checking of this driver.
If the driver attempts to corrupt the system, bugchecks 0xC4, 0xC1 and 0xA will
be among the most commonly seen crashes.
Arguments:
Arg1: 0000000000002000, Code Integrity Issue: The caller specified an executable pool type. (Expected: NonPagedPoolNx)
Arg2: fffff8010c5812bc, The address in the driver's code where the error was detected.
Arg3: 0000000000000000, Pool Type.
Arg4: 0000000032545356, Pool Tag (if provided).

Debugging Details:
------------------

DUMP_CLASS: 1

DUMP_QUALIFIER: 400

BUILD_VERSION_STRING:  10.0.16299.192 (WinBuild.160101.0800)

SYSTEM_MANUFACTURER:  System manufacturer

SYSTEM_PRODUCT_NAME:  System Product Name

SYSTEM_SKU:  SKU

SYSTEM_VERSION:  System Version

BIOS_VENDOR:  American Megatrends Inc.

BIOS_VERSION:  3401

BIOS_DATE:  12/04/2017

BASEBOARD_MANUFACTURER:  ASUSTeK COMPUTER INC.

BASEBOARD_PRODUCT:  PRIME B350-PLUS

BASEBOARD_VERSION:  Rev X.0x

DUMP_TYPE:  2

DUMP_FILE_ATTRIBUTES: 0x8
  Kernel Generated Triage Dump

BUGCHECK_P1: 2000

BUGCHECK_P2: fffff8010c5812bc

BUGCHECK_P3: 0

BUGCHECK_P4: 32545356

BUGCHECK_STR:  0xc4_2000

CPU_COUNT: 10

CPU_MHZ: d42

CPU_VENDOR:  AuthenticAMD

CPU_FAMILY: 17

CPU_MODEL: 1

CPU_STEPPING: 1

CUSTOMER_CRASH_COUNT:  1

DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

PROCESS_NAME:  System

CURRENT_IRQL:  0

ANALYSIS_SESSION_HOST:  AMD1700X

ANALYSIS_SESSION_TIME:  01-24-2018 20:40:16.0374

ANALYSIS_VERSION: 10.0.16299.15 amd64fre

LAST_CONTROL_TRANSFER:  from fffff8010d02f2d3 to fffff8010c9fa6e0

STACK_TEXT: 
fffff687`d48767d8 fffff801`0d02f2d3 : 00000000`000000c4 00000000`00002000 fffff801`0c5812bc 00000000`00000000 : nt!KeBugCheckEx
fffff687`d48767e0 fffff801`0caf910f : fffff801`0cbdba7c 00000000`00002000 fffff801`0c5812bc 00000000`00000000 : nt!VerifierBugCheckIfAppropriate+0xdf
fffff687`d4876820 fffff801`0d02740c : 00000000`32545356 fffff801`0cbdba7c fffff801`0c5812bc 00000000`00000000 : nt!VfReportIssueWithOptions+0x103
fffff687`d4876870 fffff801`0d0251c1 : 00000000`32545356 00000000`00000000 00000000`00000000 00000000`00001001 : nt!VfCheckPoolType+0x90
fffff687`d48768b0 fffff801`0c5812bc : 00000000`0000002e ffffc30d`12a29070 ffffffff`80001930 00000000`0000002e : nt!VerifierExAllocatePoolEx+0x21
fffff687`d4876900 00000000`0000002e : ffffc30d`12a29070 ffffffff`80001930 00000000`0000002e ffffc30d`12a26160 : vstor2_mntapi20_shared+0x12bc
fffff687`d4876908 ffffc30d`12a29070 : ffffffff`80001930 00000000`0000002e ffffc30d`12a26160 fffff687`d4876a70 : 0x2e
fffff687`d4876910 ffffffff`80001930 : 00000000`0000002e ffffc30d`12a26160 fffff687`d4876a70 00000000`00000000 : 0xffffc30d`12a29070
fffff687`d4876918 00000000`0000002e : ffffc30d`12a26160 fffff687`d4876a70 00000000`00000000 fffff801`0ca0e5e0 : 0xffffffff`80001930
fffff687`d4876920 ffffc30d`12a26160 : fffff687`d4876a70 00000000`00000000 fffff801`0ca0e5e0 00000000`00000000 : 0x2e
fffff687`d4876928 fffff687`d4876a70 : 00000000`00000000 fffff801`0ca0e5e0 00000000`00000000 00000000`00000000 : 0xffffc30d`12a26160
fffff687`d4876930 00000000`00000000 : fffff801`0ca0e5e0 00000000`00000000 00000000`00000000 00000000`0000002e : 0xfffff687`d4876a70

THREAD_SHA1_HASH_MOD_FUNC:  33f778527ba02c5eaf57132bc47d05644b07d4bb

THREAD_SHA1_HASH_MOD_FUNC_OFFSET:  0a1cb83a57a0c9339944ee80d98c604f02890fd0

THREAD_SHA1_HASH_MOD:  011cf14e782a04b4077aa08c74782e0c58ddcb02

FOLLOWUP_IP:
vstor2_mntapi20_shared+12bc
fffff801`0c5812bc 440fb705644e0000 movzx   r8d,word ptr [vstor2_mntapi20_shared+0x6128 (fffff801`0c586128)]

FAULT_INSTR_CODE:  5b70f44

SYMBOL_STACK_INDEX:  5

SYMBOL_NAME:  vstor2_mntapi20_shared+12bc

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: vstor2_mntapi20_shared

IMAGE_NAME:  vstor2-mntapi20-shared.sys

DEBUG_FLR_IMAGE_TIMESTAMP:  590c2a38

STACK_COMMAND:  .thread ; .cxr ; kb

BUCKET_ID_FUNC_OFFSET:  12bc

FAILURE_BUCKET_ID:  0xc4_2000_vstor2_mntapi20_shared!unknown_function

BUCKET_ID:  0xc4_2000_vstor2_mntapi20_shared!unknown_function

PRIMARY_PROBLEM_CLASS:  0xc4_2000_vstor2_mntapi20_shared!unknown_function

TARGET_TIME:  2018-01-24T18:49:46.000Z

OSBUILD:  16299

OSSERVICEPACK:  192

SERVICEPACK_NUMBER: 0

OS_REVISION: 0

SUITE_MASK:  272

PRODUCT_TYPE:  1

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 10

OSEDITION:  Windows 10 WinNt TerminalServer SingleUserTS

OS_LOCALE: 

USER_LCID:  0

OSBUILD_TIMESTAMP:  2018-01-01 11:07:05

BUILDDATESTAMP_STR:  160101.0800

BUILDLAB_STR:  WinBuild

BUILDOSVER_STR:  10.0.16299.192

ANALYSIS_SESSION_ELAPSED_TIME:  770

ANALYSIS_SOURCE:  KM

FAILURE_ID_HASH_STRING:  km:0xc4_2000_vstor2_mntapi20_shared!unknown_function

FAILURE_ID_HASH:  {c7515f44-1811-26dc-3653-401fab8093dc}

Followup:     MachineOwner
---------

[admin]

see below KB article

VMware Knowledge Base

[ColonelPolonel]

I'm confused Ranchuab,

Was that the right KB?  I'm not using VDR...

[yanw]

Would you please upload the windows dump files here? thanks

[ColonelPolonel]

Hi, with pleasure!

I've included 2, one without Windows Verifier running (25th dec) and one with it running (30th Jan) which suggests the vstor driver is the issue.

If you need any more, just let me know

[POCEH]

Please provide exact version of vstor2 driver (details page of properties). Run "sc qc vstor2-mntapi20-shared" to see path for binary to check.

[ColonelPolonel]

Hi, thanks for the reply.

File version: 6.1.2.391

Product version: 6.1.2 build-5485813

Date: 05/05/2017

[POCEH]

Ok, this driver is not compatible with Windows 10... Did you upgrade or do clean install of WS 14?

[ColonelPolonel]

Brand new PC build, brand new installation of WS14 pro...

uninstalled, re-installed, complete desktop wipe, fresh install and updated every time!

It must be coming with the install package... any ideas where to get the correct driver from?  or links showing where it's not compatible?

[POCEH · Accepted solution]

Ok there is a tricky (non officially tested!) way.

Uninstall WS 14, install latest Converter 6.2 (it's free), check vstor2-mntapi2-shared binary and make a copy of it, install WS 14 - if driver is changed put the newer, is not changed - Windows 10 compatibility is done. The driver version should be: 6.1.2.439/build-6345372.

HTH

[ColonelPolonel]

Hi again

So... these are the steps I have taken:

uninstalled VMWS14.1
restarted

Checked C:\Windows\SysWOW64\drivers -  vstor2-mntapi20-shared.sys has been removed.

Downloaded VMware-converter-en-6.2.0-7348398.exe - from https://my.vmware.com/group/vmware/details?downloadGroup=CONV62&productId=701&download=true&fileId=9...

Installed VMware-converter-en-6.2.0-7348398.exe (as admin)

this added:

bmdrvr-x64.sys
vstor2-x64.sys - version 6.1.2 build-6345372

to C:\Windows\SysWOW64\drivers

created copy of above .sys files.

Downloaded new version of VMware-workstation-full-14.1.1-7528167.exe (same file, but just incase there was anything corrupt) - from https://my.vmware.com/group/vmware/details?downloadGroup=WKST-1411-WIN&productId=686&rPId=20814

Installed VMware-workstation-full-14.1.1-7528167.exe (as admin)

this has added vstor2-mntapi20-shared.sys (version 6.1.2 build 6345372) back into C:\Windows\SysWOW64\drivers
restarted

I now have both

Should I rename my copy of vstor2-x64.sys to vstor2-mntapi20-shared.sys and copy it back in?

[POCEH]

It's important what's written in registry about binary path - check with "sc qc vstor2-mntapi20-shared", could change in registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\vstor2-mntapi20-shared (the binary name is not so important, the service name and binary path are important)

Please share your experience with DriverVerifier.

[ColonelPolonel]

Oh you beauty!!

Thank you very much. 

A query of the service is now showing vstor2-x64.sys and so is the registry.

I've re-ran the verifier test and the PC has booted without fault.

I believe your excellent work here is done!

[MPKLLC]

First and foremost, thank you POCEH for offering and spending the time to help everyone to nail this down!

Either I did something wrong or I have a different issue causing my BSODs.

I followed the procedure (or at least I think I did). But I'm still getting the BSODs. I see the version of vstor2-64.sys is 6.1.2.439 build-6345372.

"sc qc vstor2-mntapi20-shared" yields the following:

C:\WINDOWS\system32>sc qc vstor2-mntapi20-shared

[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: vstor2-mntapi20-shared

        TYPE               : 1  KERNEL_DRIVER

        START_TYPE         : 2   AUTO_START

        ERROR_CONTROL      : 1   NORMAL

        BINARY_PATH_NAME   : SysWOW64\drivers\vstor2-x64.sys

        LOAD_ORDER_GROUP   :

        TAG                : 0

        DISPLAY_NAME       : Vstor2 MntApi 2.0 Driver (shared)

        DEPENDENCIES       :

        SERVICE_START_NAME :

I did capture a memory dump this afternoon, but I didn't realize I had my dump setting configured to do a complete memory dump. The resultant file was 64 Gbytes and couldn't be uploaded. I've changed the dump type to "Active Memory". Hopefully that will produce a dump file of a more reasonable size.

[POCEH]

You can analyze memory dump by yourself with windbg.exe part of Debugging Tools for Windows 10, such large dump file will be difficult to transfer.

Also share details about BSOD - is WS involved or not, which operation provoke BDOS, is there other suspect program(s), etc.

Regards