Sorry, what exactly was the question? Which of the above steps are broken for you?
Keith, I guess looking over my post I did not really have a question... The above works ok for loading Emmett Vprobes on Windows so far.
I do have a question though... What are we probing? The hypervisor? The VM? Why not just debug the VM from the inside? What is the theory behind this magical madness?
PS. Post more videos!
1 person found this helpful
Doing this stuff from outside the VM has some big pluses, and some big minuses. On the pro side:
Most importantly, VProbes is safe. Using a kernel debugger or something from within a guest provides some very dangerous opportunities: e.g., you could poke a hole in a critical guest memory structure. Also, kernel debuggers usually stop the target program to allow inspection. VProbes keeps things moving along while data is harvested, so it can be used even on mission-critical, highly available systems.
VProbes doesn't rely on any in-guest tools. If the target is, for instance, an appliance that doesn't even have a login shell, let alone /usr/bin/top, VProbes works no better or worse.
It doesn't care what state the guest is in. If the guest is hard hung, or is swapping too hard for a shell to come up, or whatever, VProbes can be a "tool of last resort" to try to harvest information.
VProbes is also immune to guest attempts to fool it. For instance, malware often tries to change in-guest instrumentation tools to make itself undetectable. Since the VM/hypervisor trust boundary protects vprobes, it is immune to this weirdness.
VProbes is neutral as to the identity of the guest; since it's machine-level, you can use the same tools for Windows, Linux, Solaris, FreeBSD, MacOS, etc. To the extent the preload scripts work, even some common, but OS-specific, notions, like processes, can be captured.
The big negatives all boil down to the same fundamental problem: the machine abstraction level is sometimes very far away from the application abstraction level. If it's a java application running, for instance, VProbes will sit there telling you: "java.exe is running!" Well, no kidding; you probably wanted to know, e.g., which method invocation it was in. We're thinking about ways to bridge that gap, but alas, have very little to offer for serious discussion at this time.
What documentation do you recommend to become a Vprobe expert?
1 person found this helpful
The processor manuals, of course, are for reference, not for reading cover-to-cover, though perhaps the protected mode memory management chapters of Intel vol. 3 are worth a thorough read.
I think books about concrete operating systems are more useful than the academic texts, which tend to spend too much real estate on ideas that turned out to be dead ends. Solomon and Russinovich's Inside Windows 200X series are excellent, providing broad, deep, well-written insight into the most commercially important operating system out there. I haven't seen anything quite as great in the Linux world, but Love's Linux Kernel Development is just fine.
I'm getting a bunch of these in the log file running vprobes per my Windows implementation...
Mar 04 16:25:29.339: vcpu-0| VirtualProtect Host_MemProtect(0x1a40000,0x4000) returns 0x57 Mar 04 16:25:29.339: vcpu-0| VProbe: Loading VP script Mar 04 16:25:29.354: vcpu-0| VProbes VP version 0.2 Mar 04 16:25:29.354: vcpu-0| VirtualProtect Host_MemProtect(0x1a40000,0x4000) returns 0x57 Mar 04 16:25:29.370: vmx| SOCKET 24 (1312) recv error 10054: An existing connection was forcibly closed by the remote host
I just got this same error when running: vmrun vprobeLoad Fedora10.vmx '(vprobe VMM1Hz (printf "hworld!\n"))'
on Vista 32-bit host.
Mar 20 06:11:46.117: vcpu-0| VirtualProtect Host_MemProtect(0x2580000,0x4000) returns 0x57
Mar 20 06:11:46.117: vcpu-0| VProbe: Loading VP script
Mar 20 06:11:46.170: vcpu-0| VirtualProtect Host_MemProtect(0x2580000,0x4000) returns 0x57
Mar 20 06:11:46.260: vmx| SOCKET 3 (1240) recv error 10054: An existing connection was forcibly closed by the remote host
I see that VirtualProtect() is used to set protection flags for a region of memory:
This function is returning 0x57 which according to:
means: ERROR_INVALID_PARAMETER 87 (0x57) - The parameter is incorrect.
hmm...seems like vprobes is using some combination of protection flags which is not legal -- one such possibility is trying to set ReadWriteExec on hardware with NX (no-execute) feature enabled.
Try disable NX feature in BIOS (Intel calls this "XD"). I will post result if I try this
Most excellent find.
Did it work? I disabled NX on my system and still getting same error. Maybe its due to some other problem??
No change here. I'm not sure what the purpose of this call is as it pertains to vprobes... Maybe it's just an exception when the memory was paged out???
looks like this error is harmless (though I'm still not sure what it means).
Sample vp script:
vmrun vprobeLoad Fedora10.vmx '(vprobe VMM1Hz (printf "hworld!\n"))'
keeps printing this message in vprobe.out (in VM directory) until you issue:
vmrun vprobeReset Fedora10.vmx