Kernel-mode debugging over network of guest Windows Server 2012 R2 or Windows 8.1 problem

[hydeiman]

Hi, All!

I tried setup network kernel debugger of guest Windows Server 2012 R2 on VMware Workstation 10.0.1 according article:
http://msdn.microsoft.com/en-us/library/windows/hardware/hh439346(v=vs.85).aspx

But I can't connect to guest with WinDBG. I configure bcdedit, reboot and see that guest OS boot very slow.
Next I run regedit, go to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\kdnet and see that
KdInitResultString value is "InitializeNetwork failed to get the Ethernet address of the host debugger.".
KdInitStatus value is "c00000b5".

I tried:
- change hardware version to 9
- change guest Ethernet adapter to e1000
- disable DHCP (guest debugger must use APIPA)
- change connection type (bridge NAT,host-only)

and every time I get "InitializeNetwork failed to get the Ethernet address of the host debugger." error. In guest Windows 8.1 x64 and Windows 8.1 x86 i get same result.
But I can debug guest Windows Server 2012 and Windows 8 over network without any problems. Also i can debug Windows Server 2012 R2 over network in Oracle VirtualBox.

Is it a problem of VMware Workstation? Can anybody debug guest WIndows Server 2012 R2 or Windows 8.1 over network in VMware Workstation?

Thanks!

[hydeiman]

During long experiments I found that I can debug guest Windows Server 2012 R2 over network if I install guest with UEFI support. It looks like bug of VMware...

[admin]

Check to see if your virtual NIC shares an IRQ with another device when booted with the standard BIOS. I don't think the Windows kernel debugger can handle that.

[hydeiman]

Thanks for trying to help!

I checked list of IRQ in Device Manager-Resource by connection-interrupt request and saw only one device "Intel(R) 82574L Gigabit Network Connection"
with IRQ = 10.

Also I ran local WinDBG in guest OS, saw output of !arbiter 4 command and didn't find any conflicts of resources.

[admin]

Do you see any network traffic at all between the two machines?

[hydeiman]

I used Wireshark, and when I started guest Windows Server 2012 R2, I saw only following packets :

1 0.000000000 192.168.2.1 192.168.2.255 NBNS 92 Name query NB WPAD<00>
2 0.000129000 192.168.2.1 224.0.0.252 LLMNR 64 Standard query 0xf8e3  A wpad
3 0.000212000 192.168.2.1 224.0.0.252 LLMNR 64 Standard query 0x19dd  AAAA wpad
4 0.416866000 192.168.2.1 224.0.0.252 LLMNR 64 Standard query 0xf8e3  A wpad
5 0.416902000 192.168.2.1 224.0.0.252 LLMNR 64 Standard query 0x19dd  AAAA wpad
6 0.744900000 192.168.2.1 192.168.2.255 NBNS 92 Name query NB WPAD<00>
7 1.494925000 192.168.2.1 192.168.2.255 NBNS 92 Name query NB WPAD<00>

When I started Windows Server 2012 R2 with UEFI I saw many network packets - DHCP,ICMP,ARP and finally UDP packets of debugging communication.

I'am a little debugged guest OS using VMware gdb stub and when i connected debugger to OS, I was inside kd_02_8086.dll, which actively used function

UndiGetRxPacket and UndiSendTxPacket. It seems to me that if guest NIC is not initialized properly these functions can't be used ...

[admin]

Please upload your .vmx file, so that I can match your virtual device configuration.

[hydeiman]

.vmx file in attach

[admin]

Thanks for the information.  I've reproduced your problem, and the right people are now looking at it.

[alexpi]

hydeiman wrote:

During long experiments I found that I can debug guest Windows Server 2012 R2 over network if I install guest with UEFI support. It looks like bug of VMware...

You're absolutely correct. Enabling EFI mode in the vmx and then reinstalling the OS now allows me to remote debug over the network. This should be the accepted answer.

[hydeiman]

Yes, this method works, but it's a workaround. I hope bug will be fixed in next versions of VMware Workstation.

[admin · Accepted solution]

Have you tried the Workstation Technology Preview July 2014? (https://communities.vmware.com/community/vmtn/workstation/vmware-workstation-technology-preview-2014) I believe this issue has been resolved.

[hydeiman]

Thanks!

Really, kernel debugging over network for guest Windows Server 2012 R2 U1 works on VMware Workstation TP July 2014.

[WoodruffB]

The answers to date have not been 100% effective.

EFI is not always available for VM configurations and in some cases (e.g. with Threshold/Windows 10 builds) has still resulted in the same boot failure (0x0000000A).

Additionally, the tech preview for VMWare Workstation 11 did not resolve the issue.

The workaround that I finally surmised/deciphered today and which has worked 100% in these cases for our systems is to add the following parameter to  the .VMX:

hypervisor.cpuid.v0 = "FALSE" 

Be sure that VMWare is closed out so that this does not get overwritten and is properly read during the load of the setting details for your VM.

Our conjecture behind this setting is that KDNET is kicking the Win 8.1+ VMs into a mode requiring hypervisor emulation, this setting above is the same requirement for running Hyper-V on Windows 8.0+ x64 VMs.

Note that in my x86 Windows 8.1+ VM, I require this setting to enable my KDNET connection.   Rebooting with this setting eliminates the boot error and the KDNET NIC enumerates and connects to WinDBG as expected.

YMMV.

- Bryan, BoxSpy LLC

[patrickc]

For those using VMware Workstation 10, Microsoft has released a hotfix that allows network kernel debugging to now work correctly with Windows 8.1 / Server 2012 R2 guests. You can pickup the hotfix at the link below.

https://support.microsoft.com/en-us/kb/3048469/