VMware Communities : Unanswered Threads - Replay-Based Debugging

Connecting to x64 target

kernelgrunt — Fri, 06 Nov 2009 03:41:52 GMT

I am trying to hook up to a Win7 x64 boot in VMWare WS 6.5 using host platform XP x64. GNU gdb 6.3 from MinGW. Settings in vmx file:

bios.bootDelay = "3000"
debugStub.hideBreakpoints = "TRUE"
#debugStub.listen.guest32 = "TRUE"
#debugStub.listen.guest32.remote = "TRUE"
#monitor.debugOnStartGuest32 = "TRUE"
debugStub.listen.guest64 = "TRUE"
debugStub.listen.guest64.remote = "TRUE"

From gdb command-line:

(gdb) set architecture i386:x86-64
The target architecture is assumed to be i386:x86-64
(gdb) target remote localhost:8864
Remote debugging using localhost:8864
Ignoring packet error, continuing...
Ignoring packet error, continuing...
Ignoring packet error, continuing...
Couldn't establish connection to remote target
Malformed response to offset query, timeout

If I use the 32-bit settings I can connect just fine at boot and step all the way until long mode is enabled.

Using the 64-bit settings I have tried connecting at boot, and I have tried connecting after I know 64-bit instructions are executing. Same behavior...no luck. Am I doing something obviously wrong?

Host could not participate in VCE after failed CPU replaced

Jawdat — Thu, 05 Nov 2009 11:48:47 GMT

This is an odd situation that we encounter which I would like to share in order to 1-shed some more light over, hopefully and 2- so it may be of help to someone in the same situation.

Couple of weeks ago we had a HW failure on one of the HP BL680c G5 blades where it just died, lights are on but no one home kind a thing. HP came in and with a new mobo, as they always do, replaced but still no joy, the savvey engineer then discovered that it is not one but two out of the four cpus are faulty. Ordered and replaced them and the host came back to life. Fine but for the life of me I could not get the 6 hosts cluster to recognise it as a VT enabled host although made sure the VT and the 'No-Execute Memory Protection' both are enabled in the BIOS. As a result this lone host would not vmotion to or from without the warning message about the CPU bit.

The only way to get the cluster EVC enabled again and be able to vmotion normally was to create a new cluster with EVC enabled and migrate all the hosts and VMs to it. When I did that the host that had a problem joined ok and was vmotioning as normal. I had manually recreate all the rules I had on the old cluster which was deleted and the new one renamed.

Now had anyone been in the same boat and what did they do if different.

Thank you

GDB debug stub does not respond correctly to "T" thread alive queries...

visbrero — Thu, 10 Sep 2009 09:16:56 GMT

I have been trying to attach recent builds of gdb (archer's python branch: http://sourceware.org/gdb/wiki/PythonGdb) to the VMware debug stub. I've found 2 apparent gdb bugs and 1 that appears to be an issue with the VMware stub.

The gdb bugs can be found here, for completeness:

http://sourceware.org/ml/archer/2009-q3/msg00188.html

http://sourceware.org/ml/archer/2009-q3/msg00198.html

With those bugs fixed, I can perform an "info threads" after providing the linuxoffsets and get a list of all the live threads. However, attempting to switch to the thread via "thread GDBTHREADNUMBER" fails because when gdb asks about the thread, VMware's response makes it think the thread is dead. This causes it to not switch to the thread. For example, if the GDB thread number is 179 and the underlying thread id is 3008, "thread 179" results in a payload of "Tbc0". VMware responds with "E00" which makes gdb think the thread is dead, as per the spec on the command.

If I modify gdb so that remote_thread_alive in remote.c always returns true (and does not issue the command), things work as they used to work in my older setup.

I am using VMware workstation 6.5.3 with up-to-date Ubuntu 9.04 i386 guests on an Ubuntu 9.04amd64 host. In my old setup I used the gdb debugger that shipped with Ubuntu 9.04 with Workstation 6.5.2 (or maybe 6.5.1?). The system gdb no longer works, so I presume there was an update to gdb on 9.04 that broke things, but I have not investigated heavily since I'd rather use a modern gdb works (and can be scripted with python!) than figure out what ancient version works.

My idea of what works is: Attach with archer-gdb without having specified an executable. Tell it about the linux offsets, do "info threads", find the first thread in the process I care about, switch to that thread. Detach and quit. Attach with archer-gdb having specified an executable. Things are now in a state where I have a valid backtrace and gdb's understanding of the executable's layout in memory appears correct.

Debugging after serious error in client VM

DAJames — Tue, 11 Aug 2009 14:48:01 GMT

I have a problem with some code that runs in real mode from a linux kernel. At some point after my code is entered I get a pop-up n VMware saying:

*** Virtual machine kernel stack fault (hardware reset) ***
(snip a few lines of explanation)
Press OK to reboot thevirtual machine of Cancel to shut it down.

What I'd really like to do, though, is to debug it.

I have remote debugging of the kernel running, connected to gdb in another VM, but at the time that I get the stack fault in the debuggee VMware won't even let me switch to the debugging VM to try to break in with gdb. It's not clear whether this isbecause of a limitation in the way VMware is implemented, or just a result of the fact that the dialog box VMware displays is modal?
Is there some way to make the stack fault generate a breakpoint instead of a pop-up and a reset when the VM in which it occurs is being debugged?

If not, can I make that a feature request, please?

Cannot access memory at address when trying to set a breakpoint

aldep — Wed, 08 Jul 2009 20:53:47 GMT

Hi,

I am trying to debug Linux guest kernel on a Windows host. I am using MingW gdb and VMWare workstation 6.5.2 (actually the same error happens with VMWare server).

I added debugStub.listen.guest32=1 to the vmx file and was able to attach gdb on port 8832 (as described here: http://stackframe.blogspot.com/2007/04/debugging-linux-kernels-with.html).

I can see a stack trace, I can interrupt guest execution with Ctrl+C.

But when I try to put a break point at any function, I get erro, e.g.,

(gdb) b sys_open
Cannot access memory at address 0x80284c59

The same error happens when I try to view contents of the variable:

(gdb) p init_task
Cannot access memory at address 0x80a0c700

Could you please tell me how to fix this strange behavior?

Thank you,

Alex

P.S. I am not using "replay debugging" feature at all, but this discussion seems to be the most relevant place for debugging question

debug rhel

hxl — Fri, 12 Jun 2009 13:46:14 GMT

hi, i have already installed the RHEL 5 and the correct kernel symbol on my vmware.

but i can not find the vmlinuz with kernel symbol.

can any one tell me how to find that.

or give me some information about the kernel symbol.

"source-less" debugging with Replay?

ohaya — Sun, 31 May 2009 17:04:33 GMT

Hi,

I'm new to VMware workstation, and I have a question regarding the Replay capability.

I understand that Replay can be used with VS for debugging C/C++ programs, but that assumes that you have have the source for the original program. Can Replay be used in situations where you do not have the original source code?

Thanks,

Jim

Problemm remote debug linux kernel in two VMWare.

mia1978 — Tue, 24 Feb 2009 14:03:14 GMT

The problem, is 2 VMWare with an identical kernel, at loading of the
module of a kernel, it is loaded to different addresses in different
kernels, how it to correct? Thanks.

Using KDB under Vmware

NigelCunningham — Tue, 27 Jan 2009 11:42:28 GMT

Hi all.

I'm trying to use KDB under VMware, but the keyboard doesn't respond after I press the break key. Is there a workaround available?

Regards,

Nigel

New bug when using remote kernel debugging

J.F.V — Sat, 24 Jan 2009 18:41:38 GMT

Hello all,

There is an annoying bug in remote debugging which is the following:

When disconnecting from the debugger on the client side, the vmware
wont restart but it will be impossible to connect again to the vmware-gdbserver.

This is quite annoying as we have to restart the VM after each quit of the client-side
debugger. Sometimes when the client-side debugger crashes, it makes kernel debugging
using vmware much more painful ;P

Would it be possible to patch it in the next WS ?

Thanks!!

-JFV

Why do we need the replay button?

Robert.Bu — Fri, 15 Feb 2008 02:26:54 GMT

If we do not allow user to modify the register/memory, I think the re-run action can re-produce the problem. Without user interruption/modification, the re-run shall be derterministic.

serial line debugging

devzero — Sun, 27 Jan 2008 22:39:33 GMT

hello !

it`s very nice to have such advanced debugging feature in vmware, but i rarely use gdb because i`m no programmer.

my question is, why doesn`t vmware support a more basic and well established debugging technique out of the box and why do i need to use external tools for being able to do that?

i mean, it`s quite useful and very common practise to let linux kernel printk to serial line, so why not being able to telnet to the serial port in vmware?

see my feature request at http://communities.vmware.com/message/830847 or in your support database

how to debug one guest OS when multiple guest OSes are running on VMware?

colyli — Mon, 14 Jan 2008 16:22:50 GMT

These days, I am trying VMware 6 debugging. I know use "target remote localhost:8832/64" can connect to the runing guest OS in VMware. But now I run 2 guest OSes at the same time by VMware..

How can I debug a specific OS kernel of these 2 running OSes ? Can any one give me any hint. Thanks in advance

Access physical memory of Guest OS

qzy — Mon, 31 Dec 2007 05:03:08 GMT

I need to debug the Guest application, and in order to do this, it seems I need to have access to the physical memory of the Guest OS in order to parse various kernel data structures (need to translate the Guest application's virtual address to the guest physical address), and set breakpoints in the Guest physical memory. All I need is something as GetGuestPhysicalMemory(void* addr). By the way, I'm running 32-bit Windows XP host and 32-bit Windows XP guest.

I know VMware has supported this for Linux Guest. However, I need to debug Windows Guest (instruction-level debugging is fine). So I think having access to the physical memory of the Guest is the first step to enable the Windows Guest app debugging.

It seems that in VMware workstation 5.0 (windows host) and before, the vmware-vmx.exe contains a filemapping object called VMwareMem<PID>Memory, <PID> is the process ID of vmware-vmx.exe (http://www.ddj.com/architect/184402033). It represents all the physical memory of the Guest OS. However, in VMware workstation 6.0.2 which is what I am using. It doesn't have this filemapping object anymore. What it has are several small filemapping objects named as VMwareMem<PID>PhysRegionX, X could be 0 - 20 or larger, and the number is not continues.

I think VMware must have this kind of information, since it has already implemented the Linux Guest application debugging, it has to parse the kernel data structure using physical memory of Guest as well. If I am wrong, please correct me:)

-Zhiyun

Problems importing from Backup Exec System Recovery 7

2ViDiskLib — Sun, 11 Nov 2007 17:41:25 GMT

I had no problems importing from BESR 6.5 but now that I have upgraded to BESR 7.0 I am no longer able to import into VMWare Server.

I am using BESR 7.0.2.23035 and VMWare Server 1.0.4 build-56528

After getting a copy of the latest V2iDiskLib.dll ver 1.0.2.0 from Symantec; I replaced V2iDiskLib.dll ver 1.0.1.0 (only found one reference of the file at C:\Windows\system32). After rebooting the system a number of times I am now getting this error in the log file:

I 2007/11/11 12:13:45 Validating embedded-mode output file C:\DOCUME~1\xx\LOCALS~1\Temp\1\vmware-xx\v2vTempFile2332.xml
I 2007/11/11 12:13:45 Embedded output file tested and cleared, path is valid
L 2007/11/11 12:13:45 V2VAPP <version> log: 11/11/2007 12:13:45 PM
L 2007/11/11 12:13:45 Version 1.5.2 build-45706
D 2007/11/11 12:13:47 Transition from step 0 to 1
D 2007/11/11 12:13:47 Transition from step 1 to 2
D 2007/11/11 12:14:07 Transition from step 2 to 3
I 2007/11/11 12:14:07 FromSourceVM asked to convert O:\xxxxx\xxxxxxx.sv2i
I 2007/11/11 12:14:07 MountDir is C:\mnt
I 2007/11/11 12:14:07 SupportDir is C:\DOCUME~1\xx\LOCALS~1\Temp\1\VpcCvt-2007.11.11-12.14
I 2007/11/11 12:14:07 InstallDir is C:\Program Files\Common Files\VMware\VMware Virtual Machine Importer\
D 2007/11/11 12:14:07 Checking script version against C:\WINDOWS\system32\vbscript.dll
I 2007/11/11 12:14:07 Helper system has VBscript version 5.7.0.5730
I 2007/11/11 12:14:07 V2V setup validated; installDir is C:\Program Files\Common Files\VMware\VMware Virtual Machine Importer\
D 2007/11/11 12:14:07 ValidateVmc validating O:\xxxxx\xxxxxxx.sv2i
W 2007/11/11 12:14:08 Caught an exception in ValidateVmc: errcode -2147221164
W 2007/11/11 12:14:08 TranslateError: original Source is Vmc2vmx.CoVPCConfiguration.1
W 2007/11/11 12:14:08 TranslateError: original Description is The converter has not been initialized.
W 2007/11/11 12:14:08 TranslateError: now err -2147213303: VMware Virtual Machine Importer has encountered an internal error: The converter has not been initialized.
E 2007/11/11 12:14:08 FromSourceVM error: VMware Virtual Machine Importer has encountered an internal error: The converter has not been initialized.

I am getting the same log errors on a Windows 2003 server and Windows XP

Thanks for in advance for any help.

some questions about porting linux driver to esx

Zaxedc — Tue, 06 Nov 2007 10:45:12 GMT

1. I can't us kmap_atomic. Is thers something else to substitute it?

2. how to dump my debug informations? dmesg is useless.

Thanks

cheers,

zax

Guest to host communication in VMWare

khushboo — Wed, 31 Oct 2007 22:46:08 GMT

Hi,

I am looking for some documentation on the guest to host communication in VMWare. I would like to use this channel to gain information about the CPU usage of the guest - either pushing information from the guest to the host, or the host pulling this information from the guest.

Thanks!

Cheer!

Khushboo

vmware-config error

Brodo — Sat, 20 Oct 2007 14:16:05 GMT

Hi All
I have just installed the VMWare player for Linux on my Intel E6700, Mandriva 2007.1 system.

When I run vmware-config.pl it gives me an error that it can't find the file "libXtst.so.6". Doing a file search shows that this file is alive and well in the
/usr/lib64 directory.

What directory should it be in so vmware-conf can find it or how do I tell this program the right place to look ?

And another newbie question, Will a Win XP guest be able to see and use a TV tuner/PVR card installed on the Linux host ?

TIA
Brodo

Installing LAT protocol/redirector on an NT4 virtual maching

rasimon3382 — Tue, 09 Oct 2007 13:28:11 GMT

I'm running XP with an NT4 virtual machine that needs to run the networking protocol LAT (a DEC terminal services protocol). I've installed the redirector and created the com ports, but can't connect to the remote machines. Does anyone know how to configure a machine to do this?