VMware Communities : All Content - Replay-Based Debugging

Connecting to x64 target

kernelgrunt — Mon, 09 Nov 2009 23:33:15 GMT

Is there a more appropriate community to post this question too?

Host could not participate in VCE after failed CPU replaced

Jawdat — Fri, 06 Nov 2009 15:42:42 GMT

The shame and embarrasement of it all...

Thank you Shrinand for point it out.

Debugging kernel shutdown in a linux guest OS

DAJames — Tue, 27 Oct 2009 09:32:04 GMT

That version of Windows gdb is the same one that I used unsuccessfully (though mine was from mingw not cygwin). I think you may need a later version.

It's a while since I tried this but I IIRC you are using the wrong IP address -- you need the IP of the VMware virtual network adapter, which on my machine is 192.168.109.1 (I don't know whether that can vary). I didn't find that the IP address I used made any difference, though, I couldn't get debugging from the host to work.

What I eventually did was to create a second linux VM (Debian Lenny, as it happens, I don't think it should matter what distro you use as long as it has a recent gdb) and run gdb in that VM. That enabled me to debug my first VM (using 192.168.109.1 as the remote IP).

AIUI the debug port number will always be 8832 for 32-bit targets and 8864 for 64-bit targets. Trying 8834, etc., won't work.

I hope this helps.

Daniel.

Howto : Debugging Windows Xp host and Linux VM guest or Linux VM Guest to Linux VM GUest

Shrinand — Wed, 21 Oct 2009 20:38:11 GMT

Generally, one uses the gdb server for debugging remotely. Say your target machine is A. And the machine with gdb is B.

Start the gdb server on A. Connect to it using its IP address and port number from machine B (target remote <A's ip>:<A's port #>). And then start debugging using gdb on machine B. However, I'm not if debugging the kernel is possible with this technique.

You should be able to do this if you have a linux host. Start the VM and then connect to it from the host using (target remote localhost:8832) and then start debugging.

See http://stackframe.blogspot.com/2007/04/debugging-linux-kernels-with.html

LVM 1355 - One or more devices not found

Jawdat — Tue, 13 Oct 2009 17:35:38 GMT

Hello swapd, and thank you for your feed back and guess what I have originally sorted the problem when opening this thread by storage Vmotion the data off the beleaguered stoarge to a new large enough LUN and vmfs.
Today I am faced with the same repeating error in the logs of one of the hosts and so your reply was nicely timed.
Except I only did the first option in the KB which is to rescan the storage on the esx host. Funnily this stoped the error in the log but the vmkfstools -q is still showing: (One or more partitions spanned by this volume may be offline). Also funnily all the VMs on the storage with error functioning ok, on the outset.
At this stage I have to leave it as we are migrating our LUNs in teh next couple of weeks to a new SAN, yeppy!

Again thanks for your input

Loading symbols for Linux user processes

rruvinsk — Fri, 09 Oct 2009 20:59:57 GMT

Thanks, I was able to get things working with the 7.0 RC by following the instructions in the document.

Host CPU rqmts for Replay?

ecl100 — Wed, 16 Sep 2009 20:20:23 GMT

Hi, cstrieder. Replay debugging works with any processor that is record/replay capable. What processors are record replay capable? I think you'll find some text like the following in the VMware Workstation Manual...

"Record/replay support is limited to certain processors on the host. If you use the record/replay feature on a host computer that does not have a supported processor, when you enable the record/replay feature and power on the virtual machine, a message appears, informing you that recording is not supported on your processor. Supported processors include Intel Pentium 4, Intel Core 2 and later versions, AMD Barcelona and later versions, Next‐Generation Intel Microarchitecture ‐ Nehalem, and Penryn/Harpertown. Other processors might operate more slowly during recording and replaying."

GDB debug stub does not respond correctly to "T" thread alive queries...

visbrero — Thu, 10 Sep 2009 09:16:56 GMT

I have been trying to attach recent builds of gdb (archer's python branch: http://sourceware.org/gdb/wiki/PythonGdb) to the VMware debug stub. I've found 2 apparent gdb bugs and 1 that appears to be an issue with the VMware stub.

The gdb bugs can be found here, for completeness:

http://sourceware.org/ml/archer/2009-q3/msg00188.html

http://sourceware.org/ml/archer/2009-q3/msg00198.html

With those bugs fixed, I can perform an "info threads" after providing the linuxoffsets and get a list of all the live threads. However, attempting to switch to the thread via "thread GDBTHREADNUMBER" fails because when gdb asks about the thread, VMware's response makes it think the thread is dead. This causes it to not switch to the thread. For example, if the GDB thread number is 179 and the underlying thread id is 3008, "thread 179" results in a payload of "Tbc0". VMware responds with "E00" which makes gdb think the thread is dead, as per the spec on the command.

If I modify gdb so that remote_thread_alive in remote.c always returns true (and does not issue the command), things work as they used to work in my older setup.

I am using VMware workstation 6.5.3 with up-to-date Ubuntu 9.04 i386 guests on an Ubuntu 9.04amd64 host. In my old setup I used the gdb debugger that shipped with Ubuntu 9.04 with Workstation 6.5.2 (or maybe 6.5.1?). The system gdb no longer works, so I presume there was an update to gdb on 9.04 that broke things, but I have not investigated heavily since I'd rather use a modern gdb works (and can be scripted with python!) than figure out what ancient version works.

My idea of what works is: Attach with archer-gdb without having specified an executable. Tell it about the linux offsets, do "info threads", find the first thread in the process I care about, switch to that thread. Detach and quit. Attach with archer-gdb having specified an executable. Things are now in a state where I have a valid backtrace and gdb's understanding of the executable's layout in memory appears correct.

How

ecl100 — Tue, 25 Aug 2009 15:44:42 GMT

Hi, mayurdeolasee. I'm not sure what you mean. First, I assume we're talking about replay debugging (the subject of this forum). Now, are you interested in how to use replay debugging? If so, the replay debugging blog (http://www.replaydebugging.com/2008/08/getting-started-with-replay-debugging.html) is a good place to start. Or are you asking how it's implemented? If so, there's very little publicly available at this point. Perhaps we should write some papers! E.

Debugging after serious error in client VM

DAJames — Tue, 11 Aug 2009 14:48:01 GMT

I have a problem with some code that runs in real mode from a linux kernel. At some point after my code is entered I get a pop-up n VMware saying:

*** Virtual machine kernel stack fault (hardware reset) ***
(snip a few lines of explanation)
Press OK to reboot thevirtual machine of Cancel to shut it down.

What I'd really like to do, though, is to debug it.

I have remote debugging of the kernel running, connected to gdb in another VM, but at the time that I get the stack fault in the debuggee VMware won't even let me switch to the debugging VM to try to break in with gdb. It's not clear whether this isbecause of a limitation in the way VMware is implemented, or just a result of the fact that the dialog box VMware displays is modal?
Is there some way to make the stack fault generate a breakpoint instead of a pop-up and a reset when the VM in which it occurs is being debugged?

If not, can I make that a feature request, please?

Cannot access memory at address when trying to set a breakpoint

aldep — Wed, 08 Jul 2009 20:53:47 GMT

Hi,

I am trying to debug Linux guest kernel on a Windows host. I am using MingW gdb and VMWare workstation 6.5.2 (actually the same error happens with VMWare server).

I added debugStub.listen.guest32=1 to the vmx file and was able to attach gdb on port 8832 (as described here: http://stackframe.blogspot.com/2007/04/debugging-linux-kernels-with.html).

I can see a stack trace, I can interrupt guest execution with Ctrl+C.

But when I try to put a break point at any function, I get erro, e.g.,

(gdb) b sys_open
Cannot access memory at address 0x80284c59

The same error happens when I try to view contents of the variable:

(gdb) p init_task
Cannot access memory at address 0x80a0c700

Could you please tell me how to fix this strange behavior?

Thank you,

Alex

P.S. I am not using "replay debugging" feature at all, but this discussion seems to be the most relevant place for debugging question

debug rhel

hxl — Fri, 12 Jun 2009 13:46:14 GMT

hi, i have already installed the RHEL 5 and the correct kernel symbol on my vmware.

but i can not find the vmlinuz with kernel symbol.

can any one tell me how to find that.

or give me some information about the kernel symbol.

"source-less" debugging with Replay?

ohaya — Mon, 01 Jun 2009 18:02:57 GMT

Hi,

You said:

"I would expect that replay-based debugging in Visual Studio w/o source would work as well as normal debugging in Visual Studio w/o source".

I know that you also said that you hadn't done that, but I myself have not successfully used VS to do debugging w/o source either.

I did find this:

http://msdn.microsoft.com/en-us/library/0bxe8ytt.aspx

and:

http://msdn.microsoft.com/en-us/library/3s68z0b3.aspx

which talks about how to "attach" VS to an executable, but I haven't tried that yet.

Note, BTW, the comment at the end of the first page above, where it indicates that this doesn't work with VS2008, which may explain why I haven't been able to make that work ...

Anyway, assuming that I can do something like attach VS per above, how would that all work in conjunction with replay?

Thanks,

Jim

Memory watchpoints emulation ?

Shrinand — Thu, 30 Apr 2009 00:21:44 GMT

I was running into the same issue with older versions of gdb (6.3, 6.1 and even the released version of 6.8). I just tried with gdb-6.8.50.20090130 and it seems to work. You might want to give it a try.

-Shri

Getting started: kernel symbols

grubby23 — Tue, 28 Apr 2009 15:30:34 GMT

Hi Slava,

I have basically the same problem as the original poster. I am running on my host windows and on the VMWare I have Ubuntu running.

Now is the question, where could I get the symbol table for my Kernel: vmlinux-2.6.28? On the redhat site I can see a lot of similar ones,

but can I use one of them e.g.

kernel-debuginfo-2.6.9-55.0.6.EL.i686.rpm\	352952 KB	31/08/2007	00:00:00
kernel-debuginfo-2.6.9-55.0.9.EL.i686.rpm\	352967 KB	27/09/2007	00:00:00
kernel-debuginfo-2.6.9-55.EL.i686.rpm\	348228 KB	21/04/2007	00:00:00
kernel-debuginfo-2.6.9-55.EL.i686.rpm\

many thanks

Debugging a recording made after the application has started?

xjchen77 — Sat, 25 Apr 2009 16:49:17 GMT

Hi Hajo,

We are working on Remote replay based debugging. With that developers could debug recordings on QA's machines. QA just needs to have Workstation installed, not Visual Studio. Thanks for the excellent feedback.

Eric

debugStub.listen.guest64.remote = 'TRUE' doesn't open a socket for remote connections

ecl100 — Thu, 09 Apr 2009 14:31:00 GMT

Hi, Kirktrue. Thanks for the conclusion. E.

Debugging D programs

prashantd — Thu, 26 Mar 2009 23:18:34 GMT

Hi Vladimir, I am glad to hear you were able to get something going. I am curious about symbols issues. Is it specific to D program? How was your experience with replay debugging Win 32 App?

Regarding reading debugged process's memory, if your code is running in the context of visual studio debugger and if you have handle to process being debugged (handle returned by CreateProcessW) then you should able to use WIN32 API ReadProcessMemory(). We do not have any public APIs to communicate with process being debugged inside VM. I am wondering if you tried "Reverse Execution" feature with data breakpoints to debug memory corruption related problems.

Thanks,
Prashant

trace expansion on release version

ecl100 — Fri, 06 Mar 2009 17:41:45 GMT

Hi, fireboo2009. Sorry, tracing is disabled for release versions of Workstation. I'll pass along your interest in having this feature to the right people. I'll also report the bug (hanging when trying to trace).

Problemm remote debug linux kernel in two VMWare.

mia1978 — Wed, 25 Feb 2009 07:23:37 GMT

Greetings!. I think it problem VmWare. In Windows all works ok. In Virtual Box the image VmWare disk works ok (windows and linux virtual box edition), but it don't have remote debug . In Linux (x64 bit, 32bit i'am don't have) does not work!!! (VmWare), if virtual image work more 2(two), the loading address of kernel modules is allocated all time in different places (tried 30 times) in one image, other image running VmWare is control point for me (and debug mashin for remout debug).

Debugging linux kernel (inside vmware) from Mac OSX

ecl100 — Fri, 30 Jan 2009 17:12:54 GMT

Hi, Javier. My guess is that you have a line like the following in your .vmx file...

debugStub.listen.guest32 = "TRUE"

(Or guest64)

If you'd like to debug from a remote machine, ***add*** the following line...

debugStub.listen.guest32.remote = "TRUE"

(Or guest64)

Best of success to you.

Using KDB under Vmware

NigelCunningham — Thu, 29 Jan 2009 21:02:12 GMT

Hi again.

KDB (Not gdb!) is built into the kernel. Since the last message, I've discovered a work around:

Set up a serial port in the VM and configure the serial port as a console as well. Then run socat and minicom in the host.

You can then use kdb from minicom after pressing break in the VM.

A page such as http://blackmagic02881.wordpress.com/2007/02/05/linux-serial-console-how-to-with-vmware-server/ gives more detail.

Regarding gdb, I did try to use it, and can connect and so on. The monitor linuxoffsets command works most of the time, but the two together seem simpler and faster than gdb alone.

Nigel

New bug when using remote kernel debugging

J.F.V — Tue, 27 Jan 2009 17:43:00 GMT

Thanks ECL,

I cant reproduce your hint about pausing the VM to restart the gdbserver.

Basicaly, this is an example gdb session:

(gdb) target remote localhost:8832
Remote debugging using localhost:8832
New Thread 1
0x000f8997 in ?? ()
(gdb) bt
#0 0x000f8997 in ?? ()
(gdb) ^Z
[1]+ Stopped gdb
jfv$ kill -9 %1
[1]+ Stopped gdb
jfv$
[1]+ Killed gdb
jfv$

And then the VM shuts down.

Maybe it does such because some VM initialization has not been finished?

However I think I remember having that problem whatever value stands in $PC,
after having switched to pmode etc.

Because of this, I cant pause the VM before the debugger hangs or faults
(yes, it happens sometimes when trying weird things not displayed here

Im using Vmware WS 6.0.4 build-93057 (not the most recent one but thats the
one I have a license for).

Thanks again

-JFV

Bugs when using the remote debugging

ecl100 — Mon, 19 Jan 2009 18:24:45 GMT

C0, Thanks so much for the info! E.

linuxoffsets?

ecl100 — Thu, 15 Jan 2009 17:07:24 GMT

Hi, Andrew. Fascinating. Thanks for the links! E.

Debugging xnu-1228.9.59 using Fusion GDB server --> Can't get breakpoints to break

slava_malyugin — Tue, 06 Jan 2009 14:53:18 GMT

Thank you for the update. With regards to breakpoint, could you please check if putting "debugStub.hideBreakpoints=1" in .vmx file helps?

Sincerely,

Vyacheslav (Slava) Malyugin

2 questions - choosing a CPU to examine and reversing execution

ecl100 — Fri, 02 Jan 2009 16:15:45 GMT

212okins wrote:
By the way, below I've summarized certain steps that are helpful for module debugging in the guest OS.
Linux guest OS, kernel version > 2.6 assumed below. Sorry guys if you knew them already.

Hi, 212okins. Thanks so much for posting this. I'm sure others will find it useful. E.

Setting registers is not supported ?

J.F.V — Mon, 15 Dec 2008 23:07:55 GMT

Thanks for the answer

We also would like to write into memory for injecting new code, etc.

The problem is that we dont manage to set up breakpoints correctly (for ex: on 0x7c00)
without the hideBreakpoints option being TRUE. Writing 0xCC is very fine for us but it
doesnt seem to work for breakpointing on the MBR and such, when using vmware.

Our debugger is capable of relocating a .o into the guest address space, redirect guest functions on the newly
injected code by rewriting some prologs, etc. Such feature that we cannot enable without disabling early
breakpoints, since it requires to write into the guest memory in the current vmware version.

Qemu has a different behavior (obviously because of the different gdb stub) and we'd like very much to be
able to write into registers and memory while being able at the same time to breakpoint on 0x7c00 or earlier.

Thanks

-JFV

reverting to snapshot stops syncing

ecl100 — Mon, 01 Dec 2008 16:05:39 GMT

Hi, tstakland. You have posted your question on the "replay debugging" forum. Replay debugging is a feature of VMware Workstation 6.5 that allows developers to debug a recorded form of program execution. See www.replaydebugging.com for more information. My guess is that your question will be ignored here, because it has nothing to do with "replay debugging." You might try posting on a more appropriate forum. Best of success to you. E.

configure vm network setting in sabayon linux

ecl100 — Wed, 19 Nov 2008 16:00:13 GMT

Hi, entc and Kryptonic. You both have posted questions on the "replay debugging" forum. Replay debugging is a feature of VMware Workstation 6.5 that allows developers to debug a recorded form of program execution. See www.replaydebugging.com for more information. My guess is that your questions will be ignored here, because they have nothing to do with "replay debugging." You might try posting on a more appropriate forum. Best of success to you. E.

User-mode debugging via GDB stub on Windows

slava_malyugin — Mon, 03 Nov 2008 17:38:26 GMT

Hello Thomas. You are right - the gdb interface does not have any idea about Windows processes. If you want to debug Windows apps, you may want to give a try to a feature of VMware Workstation 6.5.x that allows you to use Visual Studion with Replay Debugging. The Visual Studio interface has support for process debugging. If you run into issues with it, please post in the forum.

The other, inferior, solution is to use hidden breakpoints (debugStub.hideBreakpoints=true in the .vmx file), then use conditional gdb breakpoint that checks PID of Windows process and continues if it is not "your" process. This will work, but is not going to be too fast as conditional breakpoints are quite slow.

Sincerely,

Vyacheslav "Slava" Malyugin

Support for Windows kernel mode debugging?

tdullien — Mon, 03 Nov 2008 15:24:52 GMT

While we don't do replay-based debugging, we have simply used gdb as debugger for windows kernel modules -- we
didn't have/need symbols, so this worked nicely for us.

Logs eval

ecl100 — Thu, 30 Oct 2008 16:12:22 GMT

Huh? Does this have anything to do with the "replay debugging" feature of Workstation 6.5?

Replay debugging and Visual Studio 2003?

ecl100 — Thu, 16 Oct 2008 15:38:34 GMT

Hi, bodzio131. Concerning the question of whether Workstation provides public APIs to access the replay engine, the answer is "yes" and "no" depending on what you want to do. If you want to write applications or scripts that simply create or replay recordings, you can use the VMware Vix API (http://www.vmware.com/support/developer/vix-api/ , see the Vix 1.6.1 Release Notes for Workstation 6.5). If you want to build applications or scripts that interact with programs running in recordings as they replay (like the replay debugging features), we don't have any public APIs. I think it would be great to provide such an API eventually, but replay debugging is currently an experimental feature, so it would be premature to try to provide such interfaces at this point.

Depending on what your goals are, you might find the gdb interface for replay debugging interesting (http://stackframe.blogspot.com/2007/04/workstation-60-and-death-of.html).

Best of success to you. Thanks. E.

Problems debugging a Java app in a guest Xp froma a Host Xp wit VMWorksattion 6.5 beta2 1100678

ecl100 — Fri, 19 Sep 2008 14:44:37 GMT

Hi. I'm sorry, but the replay debugging feature in Workstation 6.5 can only be used to debug Windows C/C++ applications using the Visual Studio debugger. Eclipse cannot at this time be used for replay debugging. Note that Workstation does include an extension to Eclipse that allows for remote debugging of Java programs running in virtual machines, but this can only be used to debug live programs (not recorded programs). Perhaps the replay debugging feature will be broadened to support Eclipse and/or Java in the future (but who knows?).

LVM: 1355 One or more PEs inaccessible

jbruelasdgo — Wed, 03 Sep 2008 18:43:51 GMT

Hi, today I have seen this kind of new message for me.

I was having a trouble with an iSCSI SAN (HDS), and after they changed a switch to access the disk, VMware started to send messages about not having the correct path to specific LUNs

after a manual rescan to find the new paths, this message appeared (One or more PEs inaccessible)

so anyone have any ideas?

Jose Ruelas

Interpreting an execution trace file of the recording

minxu — Thu, 28 Aug 2008 23:22:33 GMT

Can you be more specific about the load error? The content of the trace file is very low level, but understandable. It contains each instruction executed and the associated register state after the instruction execution. If your load error is something high level, such as "file not found", it is not obvious to get it from the trace. But if you know the necessary program symbol information, you may be able to use the trace file to tell you what high level program logic has been executed.

Does this answer your question?

about muti-guest os debug

slava_malyugin — Fri, 27 Jun 2008 20:25:48 GMT

Hi Yuanbor, I agree with you. Having an option allowing to specify particular port number seems like a good idea. Let me record this request in our database.

Sincerely,

Vyacheslav (Slava) Malyugin

Question about vmware coredump / vmw-gdb stub

slava_malyugin — Thu, 05 Jun 2008 21:03:45 GMT

Hi J.F.V. I agree: the crash appears to be related to the STI. You are in real mode, so there are not too many things that could cause triple fault. Perhaps IDT is pointing at nowhere, or SS is not set up properly? I probably would not be able to help you narrow down the problem, but it seems that you are right in suspectting that IRQ arrived at unexpected time.

The reason problem does not happen when you single step is because we suspend interrupts while single-stepping. Otherwise you'd end up in the timer handler after every step. You may want to use Record/Replay to record the crash, and then step thru it in Replay mode - you should be able to see the crash.

Sincerely,

Vyacheslav (Slava) Mlayugin

Debugging a PXE image

J.F.V — Thu, 08 May 2008 16:44:06 GMT

I resolved my problem -- I needed to connect on the host machine and not the guest machine ..

Breaking and stepping pre-boot code works very well now.

-JFV

Transfer Content

danpalacios — Fri, 25 Apr 2008 15:15:58 GMT

VMWare Converter will allow for copying a .vmdk to a physical hard drive (or possibly thumb drive) Whether it will boot will be a BSD question. First, the hardware will have changes from the VM to the physical box. My experience is that Linux/BSD is less forgiving on hardware changes that Windows, so you may need to repair/recompile the kernal to get it to work. Second, booting from a USB stick usually involves some trickery within the OS. I am sure there are threads out there describing the process, but again this is a BSD question and may require some serious modification to your image.

Record and replay scope question

rini — Thu, 17 Apr 2008 04:15:08 GMT

Hi Slava,

Thanks for checking. I hope we get access soon as well.

Best,

Rini

Original message ----

Date: Wed, 16 Apr 2008 19:38:35 -0700
From: slava_malyugin <communities-emailer@vmware.com>
Subject: Replay-based debugging New message: "Record and replay scope question" 2jCZPX-1PX5-3QH3
To: <kaushik1@uiuc.edu>

A new message was posted in the thread "Record and replay scope question":

http://communities.vmware.com/message/917541

Author : slava_malyugin
Profile : http://communities.vmware.com/people/slava_malyugin
Message:

Catching linux at the very __start

Natalie — Wed, 12 Mar 2008 19:40:33 GMT

Yes, you are correct: record/replay is not Server 2.0 supported feature.

Cannot access memory at address *** during replay

slava_malyugin — Wed, 20 Feb 2008 17:22:03 GMT

Yes, modifying the variable may or may not result in code divergence. However, since it would be impossible to prove that a given modification would not cause it, we disallowed modification in record/replay mode completely.

Why do we need the replay button?

Robert.Bu — Fri, 15 Feb 2008 02:26:54 GMT

If we do not allow user to modify the register/memory, I think the re-run action can re-produce the problem. Without user interruption/modification, the re-run shall be derterministic.

Recording format

MarcusGDaniels — Tue, 12 Feb 2008 00:52:41 GMT

Ah, I misunderstood what was meant by the `format'. I meant the actual replay. Thanks!

serial line debugging

devzero — Wed, 30 Jan 2008 23:03:18 GMT

thanks!

how to debug one guest OS when multiple guest OSes are running on VMware?

colyli — Mon, 14 Jan 2008 17:43:03 GMT

This is very helpful information, thank a lot.

I will try it right now

Guest kernel debugging only supported in Linux host?

qzy — Wed, 02 Jan 2008 22:23:40 GMT

slava_malyugin,

I see. Int3-based approach does has incompatibility issue with the way the VMM is implemented (translation cache, memory modification detection).

For the hidden breakpoint, why does it have to take effect on every process? Is it because of the way it is implemented? I heard that VMware uses the Just-in-time write to memory to implement the hidden breakpoint, if so, it should be able to be set for individual process.

I will try more about the hidden breakpoint. But can you please explain a bit more about how the Linux Guest application debugging is done? Since GDB can issue a command to switch to the desired process or thread's address space. If this functionality is also implemented using the conditional hidden breakpoint, then it may still have performance issue, right?

And for the Academics program, does it include the source code for VMware workstation 6.0.1 or later? (The version that contains replay functionality)

-Zhiyun

Access physical memory of Guest OS

qzy — Mon, 31 Dec 2007 05:03:08 GMT

I need to debug the Guest application, and in order to do this, it seems I need to have access to the physical memory of the Guest OS in order to parse various kernel data structures (need to translate the Guest application's virtual address to the guest physical address), and set breakpoints in the Guest physical memory. All I need is something as GetGuestPhysicalMemory(void* addr). By the way, I'm running 32-bit Windows XP host and 32-bit Windows XP guest.

I know VMware has supported this for Linux Guest. However, I need to debug Windows Guest (instruction-level debugging is fine). So I think having access to the physical memory of the Guest is the first step to enable the Windows Guest app debugging.

It seems that in VMware workstation 5.0 (windows host) and before, the vmware-vmx.exe contains a filemapping object called VMwareMem<PID>Memory, <PID> is the process ID of vmware-vmx.exe (http://www.ddj.com/architect/184402033). It represents all the physical memory of the Guest OS. However, in VMware workstation 6.0.2 which is what I am using. It doesn't have this filemapping object anymore. What it has are several small filemapping objects named as VMwareMem<PID>PhysRegionX, X could be 0 - 20 or larger, and the number is not continues.

I think VMware must have this kind of information, since it has already implemented the Linux Guest application debugging, it has to parse the kernel data structure using physical memory of Guest as well. If I am wrong, please correct me:)

-Zhiyun

Installing LAT protocol/redirector on an NT4 virtual maching

rasimon3382 — Sun, 16 Dec 2007 22:30:39 GMT

Yes, I found a solution!

You need to find a copy of some kind of LAT driver that will run under NT. Install it under the virtual machine. Then follow the instructions. Instructions are very important. If I had them, I would have found the solution much earlier than I did. There was a program that associates a comm port with a LAT session. When I found the program, everything setup seamlessly.

Just find a driver. Anymore questions, I'll try to help.

Debugging using VM WS6.0

francois.tiers — Thu, 13 Dec 2007 06:40:04 GMT

i had no time to test, i will test this afternoon...

real mode guest debugging

slava_malyugin — Thu, 13 Dec 2007 02:01:07 GMT

First of all, thanks for trying out the feature. The reason it did not work is because by default our debug stub will use "int3" to put a breakpoint. The breakpoint was overwritten with the BIOS loader, so it never hit. What you need is a hidden breakpoint - the one that does not modify Guest memory or registers, and that cannot be disabled by the Guest.

There are two ways to enable it. First (and preferrable) is to run the Guest in Record/Replay mode: you start recording, boot the Guest, then start Replay, attach debugger and put breakpoint. There are more details at stackframe.blogspot.com. The other approach is to put this line in the .vmx file:

debugStub.hideBreakpoints=true

One unrelated problem that you should be aware while debugging 16-bit code is that gdb is treating all addresses as linear. So, if you want to put a breakpoint at 0x8A:0x100, for example, you should use something like "b *0x9A0". I hope this helps.

Awesome

etieseler — Mon, 26 Nov 2007 23:47:52 GMT

Take a look here (pubs.vmware.com) for some good information about esxtop.

-Ed

Problems importing from Backup Exec System Recovery 7

2ViDiskLib — Mon, 12 Nov 2007 18:37:53 GMT

I found a good work around for my needs.

Instead of using the import feature from within the VMware Server console, I downloaded the VMware Converter and that worked even better as it also allows me to configure many options of the VMware server I am creating.

There is still a problem with the import feature but it's no longer an issue - maybe VMware could fold the converter app into the console.

some questions about porting linux driver to esx

Zaxedc — Tue, 06 Nov 2007 10:45:12 GMT

1. I can't us kmap_atomic. Is thers something else to substitute it?

2. how to dump my debug informations? dmesg is useless.

Thanks

cheers,

zax

Guest to host communication in VMWare

khushboo — Wed, 31 Oct 2007 22:46:08 GMT

Hi,

I am looking for some documentation on the guest to host communication in VMWare. I would like to use this channel to gain information about the CPU usage of the guest - either pushing information from the guest to the host, or the host pulling this information from the guest.

Thanks!

Cheer!

Khushboo

Problem installing 64 bit Guest OS

mdjones — Tue, 30 Oct 2007 14:45:29 GMT

Tom, Check out these documents regarding cpu compatibility.

I've included links for the Intel and AMD docs as you do not state which processors you have.

Intel http://kb.vmware.com/kb/1991

AMD http://kb.vmware.com/kb/1992

I had a similar problem with an AMD 850 processor at stepping level 3. Although you could run a 64 bit OS directly on the hardware you cannot host 64 bit 'guests' due to the 'segmentation' restriction corrected in later revisions of the processor.

Hope this helps you out.

Mark

Where do I post debug info and logs?

oreeh — Fri, 26 Oct 2007 07:15:51 GMT

On the left is a link "Start discussion"

64 bit apps issues - ESX 3.02

tom howarth — Thu, 25 Oct 2007 11:32:08 GMT

Firstly lets go back to basics, if it ws an issue whti the VM hosts hardware you would not even be able to run or install a 64 bit guest.

Now what application are we talking about here, this infomation woud help fault finding.

My first port of call would be the services console and and verify the account that is being used to start the service, as a previously stated it is this or a permission on a file that is a prerequsite for the application that is causing the error, not the account that you are using to log into the machine, there should be no corolation between the two.

If you found this or any other post helpful please consider the use of the Helpfull/Correct buttons to award points

Kind Regards

Tom,

vmware-config error

Brodo — Sun, 21 Oct 2007 01:33:49 GMT

Ta
Saw the "Guest Debugging" heading and thought this was the appropriate forum.

Brodo

Multi thread program not working on FC6 (On VM Player)

ecl100 — Sat, 20 Oct 2007 15:46:09 GMT

Hi. Check out the post on this forum with the subject "Getting started with replay-based debugging."

Note that if you think you have found a bug in VMware Player, replay-based debugging won't help you. In fact, this particular forum also will not be useful (because this forum is for developers using VMware Workstation to debug applications). Best of success. E.

Memory Allocation for a Citrix Terminal Server guest OS

VMware Rookie — Fri, 19 Oct 2007 16:23:36 GMT

Thanks for the suggestions. I gained little to no performance switching to 1 proc.

I ended up demoing EXS and my citrix box is now running like a well-oiled virtual machine.

mismatched symbols for RH4 32bit

cutter_brown — Thu, 18 Oct 2007 12:44:50 GMT

yeah, its compiled with -g. I think its clear now that the issue is with the kernels i'm building and not vmware. Thanks for the help. When I figure it out, I'll post it here in case there is anyone else that encounters this. As a side note these are the relivant .config options that I had enabled:

CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
CONFIG_KALLSYMS_EXTRA_PASS=y
....

CONFIG_DEBUG_KERNEL=y
CONFIG_DEBUG_SLAB=y
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_SPINLOCK_SLEEP=y
CONFIG_DEBUG_HIGHMEM=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_STACKOVERFLOW=y
CONFIG_DEBUG_STACK_USAGE=y
CONFIG_DEBUG_PAGEALLOC=y

in-guest recording control

minxu — Fri, 12 Oct 2007 21:20:33 GMT

goto: http://kb.vmware.com/ and search "inguest" should return a KB article about it. Thanks!

usb floppy not visible to an ms--dos vm

jimmieDV — Wed, 03 Oct 2007 08:20:48 GMT

Thank you for your answer.

4 port was open

outtuo — Tue, 02 Oct 2007 14:05:01 GMT

Thanks you ...

Getting started with replay-based debugging

jhuang — Fri, 28 Sep 2007 17:56:45 GMT

Oh but you can definitely start

A number of the features mentioned were already part of WS 6.0. Even better yet, we've recently release WS 6.0.1 that includes significant performance improvements to the feature ---

You can begin the eval process @

http://www.vmware.com/download/ws/eval.html

Or you already have a WS 6.0 key, you can update for free and just download WS 6.0.1 @

http://www.vmware.com/download/ws/

Let us know how things go for you!

Jane

Breakpoints

slava_malyugin — Thu, 27 Sep 2007 03:30:49 GMT

Hi Bradley,

Thank you for trying out replay debugger and reporting an issue. The "break *0xcxxxxx" command should work, so you may be hitting a bug. To be sure, could you verify that the code path with breakpoint is executed? The mistake that I frequently make is running one version of the kernel in the VM and supplying different version of the kernel to the gdb; this confuses the debugger and it ends up setting breakpoints at wrong addresses.

Could you please also upload vmware.log file from filing run, and output of your gdb session? Please issue "set debug remote 1" in gdb before inserting the breakpoint. It will motivate gdb to tell more about its interactions with the Workstation debugger.

If you put the breakpoint manually with "int3" then there is no way for debugger to intercept it. We store information about breakpoints installed with debugger, and if we hit int3 not matching one of "our" breakpoints, we immediately pass it to the Guest.

We have two different breakpoint mechanisms internally. If you are running in non-replay mode, we'll smash the Guest memory with int3. In replay mode, we use so called "invisible" breakpoints. They are like hardware breakpoints (no memory is modified), but you can have more than four of them at a time. Please let me know if you observed the problem in replaying mode, or non-replaying mode. In the latter case, does adding "debugStub.hideBreakpoints=1" to the configuration file solve the problem? This line forces invisible breakpoints in non-replay mode.

The invisible breakpoints operate on virtual addresses (another similarity with hardware breakpoints). So if you put them on particular address, it will hit in all address spaces. The "int3"-based breakpoints are smashing memory, so if you have different page in another address space, it won't hit. Are you using linux with 4G/4G patch by any chance?

Thanks again for trying out this feature. I believe the vmware.log and log of gdb session will help us narrow down the source of the problem.

New forum!

daniel_uk — Thu, 20 Sep 2007 11:31:34 GMT

Larstr, I agree.

Its more CPU lock stepping like www.marathontechnologies.com

Dan

VMware Communities : All Content - Replay-Based Debugging

Connecting to x64 target

Host could not participate in VCE after failed CPU replaced

Debugging kernel shutdown in a linux guest OS

Howto : Debugging Windows Xp host and Linux VM guest or Linux VM Guest to Linux VM GUest

LVM 1355 - One or more devices not found

Loading symbols for Linux user processes

Host CPU rqmts for Replay?

GDB debug stub does not respond correctly to "T" thread alive queries...

How

Debugging after serious error in client VM

Cannot access memory at address when trying to set a breakpoint

debug rhel

"source-less" debugging with Replay?

Memory watchpoints emulation ?

Getting started: kernel symbols

Debugging a recording made *after* the application has started?

debugStub.listen.guest64.remote = 'TRUE' doesn't open a socket for remote connections

Debugging D programs

trace expansion on release version

Problemm remote debug linux kernel in two VMWare.

Debugging linux kernel (inside vmware) from Mac OSX

Using KDB under Vmware

New bug when using remote kernel debugging

Bugs when using the remote debugging

linuxoffsets?

Debugging xnu-1228.9.59 using Fusion GDB server --> Can't get breakpoints to break

2 questions - choosing a CPU to examine and reversing execution

Setting registers is not supported ?

reverting to snapshot stops syncing

configure vm network setting in sabayon linux

User-mode debugging via GDB stub on Windows

Support for Windows kernel mode debugging?

Logs eval

Replay debugging and Visual Studio 2003?

Problems debugging a Java app in a guest Xp froma a Host Xp wit VMWorksattion 6.5 beta2 1100678

LVM: 1355 One or more PEs inaccessible

Interpreting an execution trace file of the recording

about muti-guest os debug

Question about vmware coredump / vmw-gdb stub

Debugging a PXE image

Transfer Content

Record and replay scope question

Catching linux at the very __start

Cannot access memory at address *** during replay

Why do we need the replay button?

Recording format

serial line debugging

how to debug one guest OS when multiple guest OSes are running on VMware?

Guest kernel debugging only supported in Linux host?

Access physical memory of Guest OS

Installing LAT protocol/redirector on an NT4 virtual maching

Debugging using VM WS6.0

real mode guest debugging

Awesome

Problems importing from Backup Exec System Recovery 7

some questions about porting linux driver to esx

Guest to host communication in VMWare

Problem installing 64 bit Guest OS

Where do I post debug info and logs?

64 bit apps issues - ESX 3.02

vmware-config error

Multi thread program not working on FC6 (On VM Player)

Memory Allocation for a Citrix Terminal Server guest OS

mismatched symbols for RH4 32bit

in-guest recording control

usb floppy not visible to an ms--dos vm

4 port was open

Getting started with replay-based debugging

Breakpoints

New forum!

Debugging a recording made after the application has started?