SIGAR: Fails to read registry and to filter processes in Windows 2000

[franciscoperedo]

My efforts to try to understand why Hyperic 4.1.2 fails to work properly in windows 2000 ( http://communities.vmware.com/thread/354965?tstart=0 ) have lead me to believe that the problem is at the SIGAR level.

According to

http://support.hyperic.com/display/SIGAR/Home

SIGAR supports: NT 4.0, 2000 Pro/Server, 2003 Server, XP, Vista, 2008 Server, 7

I decided to test that assumption in my Windows 2000:

C:\>cd "C:\Program Files\Hyperic HQ 4.1.2\agent-4.1.2\bundles\agent-4.1.2-1053\p
dk\lib"

C:\Program Files\Hyperic HQ 4.1.2\agent-4.1.2\bundles\agent-4.1.2-1053\pdk\lib>j
ava -jar sigar.jar

sigar> test

.test.TestRegistryKey.testRegistryRead.........ERROR
.test.TestRegistryKey.testHardwareValues.......ERROR
.test.TestRegistryKey.testSoftwareValues.......ERROR
.test.TestFileVersion.testCreate...............FAILED

Guess that solves the mistery, SIGAR fails on Windows 2000 when reading the registry (a feature needed for Tomcat autodiscovery).

My theory is that it also fails once Tomcat is running when it tries to verify if the process is running:

2009-10-21 08:57:21,671 DEBUG [Thread-1] [org.hyperic.hq.product.jmx.MxServerDetector] State.Name.sw=java,Args.*.sw=-Dcatalina.base= matched 0 processes

The failing code would the method inherited by MxServerDetector from ServerDetector:

protected static long[] getPids(String query) {
try {
return ProcessFinder.find(getSigar(), query);
} catch (SigarException e) {
return NO_PIDS;
}
}

And the root cause? the ProcessFinder.find finds nothing under Windows 2000... the strange thing is that the "ps" command work on SIGAR command line, so this ProcessFinder must be doing something differently.

[franciscoperedo]

On further examination it seems that org.hyperic.hq.product.jmx.MxServerDetector.getPids fails simply because on Windows, when Tomcat is running as a service, the name of the process is tomcat6w.exe not java. And that is true for Window 2003 too, so, that problem seems to be irrelevant for the moment.

But the failure to read registry keys is very relevant, and important, I wonder... how does SIGAR read registry keys? and why is the way it reads them incompatible with Windows 2000...

[franciscoperedo]

I wrote this code to test RegOpenKeyEx (and compiled it using VS C++ 2008) :

#include "stdafx.h"
#include <windows.h>
#include <stdio.h>
#include <strsafe.h>
#include <iostream>



BOOL RegOpenNode (HKEY hKeyRoot, LPTSTR lpSubKey)
{
LPTSTR lpEnd;
LONG lResult;
DWORD dwSize;
TCHAR szName[MAX_PATH];
HKEY hKey;
FILETIME ftWrite;

lResult = RegOpenKeyEx (hKeyRoot, lpSubKey, 0, KEY_READ, &hKey);

if (lResult != ERROR_SUCCESS)
{
if (lResult == ERROR_FILE_NOT_FOUND) {
std::cout<<"Key not found.\n";
return FALSE;
}
else {
std::cout<<"Error opening key:"<<lResult<<"\n";
return FALSE;
}
}
std::cout<<"hkey: "<<hKey<<"\n";
return TRUE;

}


int _tmain(int argc, _TCHAR* argv[])
{
std::cout<<"Registry OpenKeyTest\n";
BOOL bSuccess;
bSuccess = RegOpenNode(HKEY_LOCAL_MACHINE, TEXT("SOFTWARE"));
std::cout<<"Success:"<<bSuccess;
std::cin.get();
return 0;
}

Then i ran it on the same Windows 2000 machine where SIGAR is unable to open the SOFTWARE registry key...

And it works! How can that be? Why RegOpenKeyEx fails if invoked by SIGAR but works if invoked by this C++ program?

Message was edited by: franciscoperedo

[franciscoperedo]

This is the code that crashes in SIGAR:

public void testRegistryRead() throws Exception {
RegistryKey software =
RegistryKey.LocalMachine.openSubKey("SOFTWARE");
String[] keys = software.getSubKeyNames();
assertTrue(keys.length > 0);
software.close();
}

Here is the stacktrace:

1) org.hyperic.sigar.win32.test.TestRegistryKey:
org.hyperic.sigar.win32.Win32Exception: Error opening subkey
at org.hyperic.sigar.win32.RegistryKey.openSubKey(RegistryKey.java:227)
at org.hyperic.sigar.win32.test.TestRegistryKey.testRegistryRead(TestReg
istryKey.java:35)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at org.hyperic.sigar.test.SigarTestPrinter.runTests(SigarTestPrinter.jav
a:183)
at org.hyperic.sigar.test.SigarTestRunner.output(SigarTestRunner.java:13
9)
at org.hyperic.sigar.cmd.SigarCommandBase.processCommand(SigarCommandBas
e.java:188)
at org.hyperic.sigar.shell.ShellBase.processCommand(ShellBase.java:397)
at org.hyperic.sigar.cmd.Shell.processCommand(Shell.java:122)
at org.hyperic.sigar.shell.ShellBase.handleCommand(ShellBase.java:364)
at org.hyperic.sigar.shell.ShellBase.handleCommand(ShellBase.java:310)
at org.hyperic.sigar.shell.ShellBase.run(ShellBase.java:289)
at org.hyperic.sigar.cmd.Shell.main(Shell.java:222)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at org.hyperic.sigar.cmd.Runner.main(Runner.java:214)

It fails at TestRegistryKey.java:35 that is this line:

RegistryKey software =
RegistryKey.LocalMachine.openSubKey("SOFTWARE");

But... why does it fail? if I inspect the code inside openSubKey it AFAIK ultimately calls RegOpenKeyEx and RegOpenKeyEx seems to be working fine in Windows 2000...

[franciscoperedo]

Finally, I know why it fails!!!!
This call works perfectly in Windows 2000 and XP:

lResult = RegOpenKeyEx (hKeyRoot, lpSubKey, 0, KEY_READ, &hKey);

But this one fails in Windows 2000

lResult = RegOpenKeyEx (hKeyRoot, lpSubKey, 0, KEY_READ|KEY_WOW64_32KEY, &hKey);

The difference? The KEY_WOW64_32KEY! That is what is breaking compatibility with windows 2000! And SIGAR is using KEY_WOW64_32KEY at registrykey.cpp (stored inside the RegOpenOpts variable)

I remember reading somewhere in SIGAR JIRA that this support for 64 bit/32 bit registry access was added recently...

Message was edited by: franciscoperedo

[franciscoperedo]

Confirmed, KEY_WOW64_32KEY is not supported in windows 2000:

http://msdn.microsoft.com/en-us/library/ms724878%28VS.85%29.aspx

[franciscoperedo]

And, AFAIK, this is the JIRA that introduced KEY_WOW64_32KEY and broke compatibility with Windows 2000 in SIGAR:

http://jira.hyperic.com/browse/SIGAR-120

Now I only hope someone with the power of doing it submits a JIRA requesting to fix this problem (apparently the JIRA of SIGAR is not open for anyone to submit bugs, as the JIRAs of other opensource projects...)

[franciscoperedo]

So, since it seems that the Hyperic team is to busy to answer :'(
I decided to create my own "custom" patched version that works in Windows 2000 following the instructions here:

http://support.hyperic.com/display/SIGAR/Home

According to that the requirements are:

* JDK 1.4 or higher
* Ant 1.6.5 or higher
* Perl 5.6.1 or higher
* C compiler

I already have a C compiler (inside VS 2008) and I already have Ant 1.7.1, I also have JDK 1.6 so all I was missing was Perl, so I downloaded Strawberry Perl (wich BTW also includes gcc, so now I have 2 C compilers)

So I call the build.xml and I get this error:

D:\Users\XXXX\Workspaces\Sigar\sigar-1.6\bindings\java\build.xml:228: Execute
failed: java.io.IOException: Cannot run program "rc" (in directory "D:\Users\XXXX\Workspaces\Sigar\sigar-1.6\bindings\java\build\src"): CreateProcess error=
2, El sistema no puede hallar el archivo especificado

Apparently I also need to have rc.exe. What is rc you ask?:

http://en.wikipedia.org/wiki/Rc

A shell for unix... and unless I can find a version for win32 (guess I'll have to use cygwin for that?) it will not be possible to compile SIGAR in windows :'(

Message was edited by: franciscoperedo

[franciscoperedo]

Mmm, OTOH rc could be the "Microsoft (R) Windows (R) Resource Compiler Version" which means I need to build this from the "Visual Studio 2008 Command Prompt"...

Message was edited by: franciscoperedo

[franciscoperedo]

Yes! It compiled!

[franciscoperedo]

Mmmm, I got happy too soon, it turns out that the sources downloadable from

http://sourceforge.net/projects/sigar/files/

Are incomplete (no support for the "test" command, and the source to interact with the with the registry? well it is just NOT there!)

And news get even worse, if I download from:

http://svn.hyperic.org/projects/sigar_mirror/branches/sigar-1.6

It fails to compile with:

[cc] D:\Users\XXXX\Workspaces\Sigar\sigar-1.6\bindings\java\src\jni\wi
n32\metabase.cpp(13) : fatal error C1083: Cannot open include file: 'atlBase.h':
No such file or directory
[cc] wmi.cpp
[cc] Generating Code...

so... I am stuck again...

Message was edited by: franciscoperedo

[franciscoperedo]

Seems like I need the Windows 2003 SDK:

https://developer.mozilla.org/en/atlbase.h

Downloading...

Adding path in build.xml:

<path id="jni.include.dirs">
<path location="C:/Program Files/Microsoft Platform SDK for Windows Server 2003 R2/Include/atl"/>

Message was edited by: franciscoperedo

[franciscoperedo]

The Windows 2003 SDK did not solve my problem, it includes the .h file, but it is missing:

[cc] LINK : fatal error LNK1104: cannot open file 'atlthunk.lib'

So I decided to see if I could skip this atl thing.. and yes, in file sigar-1.6\bindings\java\hyperic_jni\jni-build.xml i found:

<compiler name="msvc" debug="${jni.debug}" if="win32">
<defineset>
<define name="WIN32"/>
<define name="_CRT_SECURE_NO_DEPRECATE"/>
<!-- XXX 2008 MSC express/SDK does not include atl headers -->
<define name="SIGAR_NO_ATL" if="jni.noatl"/>
</defineset>
</compiler>

If I remove the if="jni.noatl", it compiles successfuly... but... does it work...?

[franciscoperedo]

Nop, it does not... I does not include the test command, I guess without the atl SIGAR is crippled...

[dougm_hyperic]

Hi Francisco,

Thanks for digging into this. I have reproduced the problem and will get the fix into sigar, along with an updated binary for you to verify.

[franciscoperedo]

Seems like all I have to do is

http://social.msdn.microsoft.com/forums/en-US/vcgeneral/thread/c8298f5b-bdee-4707-990c-046f3c3dfe71/

So I comment out the line (inside atlbase.h):

#pragma comment(lib, "atlthunk.lib")

and it compiles correctly!... now... does it work?

Message was edited by: franciscoperedo

[franciscoperedo]

java -jar sigar.jar
sigar> test
unknown command: test
sigar>

What do I have to do to have the "test" command enabled??!!!

[dougm_hyperic]

looks like a classpath issue, it should work if you run from the bindings\java directory:

java -jar sigar-bin\lib\sigar.jar test

[franciscoperedo]

Dougm, thanks for answering!

So the next sigar will be able to read the registry in Windows 2000? Great! 🙂

BTW... can you tell me why the SIGAR I am compiling does not include the "test" command? what do I have to do to enable it?

[franciscoperedo]

thanks a lot! yes, it was a classpath issue.