Unable to install tools?

[pdovy]

I'm running the VIX 1.0 API under Ubuntu 7 and with Server 1.0.3, using C.

I've been piecing together code from the reference documentation, just to get a feel for what the API can do. However I've run in to a couple problems. I am able to connect to the server on my local machine, register a VM and power it on (at this point, if I open the server console I can watch it boot - the OS is Red Hat Linux).

The next logical step would be to install tools, and then log in. However I don't seem to be able to do this - if I call VixVM_InstallTools and wait on the handle, it returns successfully after a very short time, then when I call VixVM_LoginInGuest, it gives back an error saying tools is not running, and then core dumps. When I open the server console and boot the VM manually, it also says tools is not installed. I should also mention that during this time when Vix is supposedly installing tools and attempting to log in, if I watch the VM in the server console, it has not yet finished booting - so I'm not sure how it could have installed tools.

I also noticed that there are errors for registering an already registered VM and powering on an already powered on VM, however I also noticed that if I run my program while the VM is powered on - it does not complain about me trying to register or power on the VM.

Thanks in advance for your help!

\- Paul

[admin · Accepted solution]

OK, you caught us. VixVM_InstallTools() is functionally the equivalent of clicking Install Tools in the Service Console; it mounts the CD in the guest operating system, triggering any auto-run that the operating system may have. Since there is no automated installer for the Linux Tools, the most that this will do is mount the CD on the file system. You will still have to go through one of the installers yourself (RPM or tar-ball).

Basically there is no unattended way to install Tools on Linux OSes, so this function is not currently itself unattended.

This will be better documented in future releases.

As for the code dump, that shouldn't happen. Would you be willing to post the code where this happens, so I can take a look at it?

I'm not sure what you mean about there being errors for those things; are you basing this on the names of errors in vix.h? I'd say in general we try to be forgiving in terms of returning errors situations where the result of the operation has already been achieved, just to make coding a little easier.

[pdovy]

Thanks for your response - that makes much more sense, I'll just have to manually install the tools. As for the errors, I was just curious - a graceful failure is actually a nice behavior.

I've posted both the code and the program output for when it core dumps below. I should note though that I'm running in what I imagine is for you and untested/unsupported configuration. I'm running 64-bit Ubuntu, and I've used the -m32 flag to force gcc to let me use the 32-bit libvmware-vix.so library. With the exception of this problem - the API seems to behave normally though.

Code

\----

---

#include "vix.h"

#include <stdio.h>

int main(int argc, char * argv[])

{

VixHandle hostHandle = VIX_INVALID_HANDLE;

VixHandle jobHandle = VIX_INVALID_HANDLE;

VixHandle vmHandle = VIX_INVALID_HANDLE;

VixError err;

char* testVMPath = "/var/lib/vmware/Virtual Machines/ChangeMe/ChangeMe.vmx";

// Connect as current user on local host.

jobHandle = VixHost_Connect(VIX_API_VERSION,

VIX_SERVICEPROVIDER_VMWARE_SERVER,

NULL, // hostName

0, // hostPort

NULL, // userName

NULL, // password,

0, // options

VIX_INVALID_HANDLE, // propertyListHandle

NULL, // callbackProc

NULL); // clientData

err = VixJob_Wait(jobHandle,

VIX_PROPERTY_JOB_RESULT_HANDLE,

&hostHandle,

VIX_PROPERTY_NONE);

if (VIX_OK != err) {

// Handle the error...

printf("Error connecting to host...\n");

goto abort;

}

Vix_ReleaseHandle(jobHandle);

printf("Connected successfully...\n");

jobHandle = VixVM_Open(hostHandle, testVMPath, NULL, NULL);

err = VixJob_Wait(jobHandle, VIX_PROPERTY_JOB_RESULT_HANDLE, &vmHandle, VIX_PROPERTY_NONE);

if(VIX_OK != err)

{

printf("Could not successfully open the VM\n");

goto abort;

}

printf("Successfully opened the VM\n");

Vix_ReleaseHandle(jobHandle);

jobHandle = VixVM_PowerOn(vmHandle, VIX_VMPOWEROP_NORMAL, VIX_INVALID_HANDLE, NULL, NULL);

err = VixJob_Wait(jobHandle, VIX_PROPERTY_NONE);

if(VIX_OK != err)

{

printf("Could not successfully power on the VM: %s\n", Vix_GetErrorText(err, NULL));

goto abort;

}

printf("Successfully powered on the VM\n");

Vix_ReleaseHandle(jobHandle);

jobHandle = VixVM_InstallTools(vmHandle, 0, NULL, NULL, NULL);

err = VixJob_Wait(jobHandle, VIX_PROPERTY_NONE);

if(VIX_OK != err)

{

printf("Could not successfully install VMWare Tools on the VM: %s\n", Vix_GetErrorText(err, NULL));

goto abort;

}

printf("Successfully installed VMWare Tools on the VM\n");

Vix_ReleaseHandle(jobHandle);

jobHandle = VixVM_LoginInGuest(vmHandle, "root", "password", 0, NULL, NULL);

err = VixJob_Wait(jobHandle, VIX_PROPERTY_NONE);

if(VIX_OK != err)

{

printf("Could not successfully log in on the VM: %s\n", Vix_GetErrorText(err, NULL));

goto abort;

}

printf("Successfully logged in on the VM\n");

abort:

Vix_ReleaseHandle(jobHandle);

Vix_ReleaseHandle(vmHandle);

VixHost_Disconnect(hostHandle);

}

Output

\----

---

pdovy@devintern02:~/test$ ./a.out

Connected successfully...

Successfully opened the VM

Successfully powered on the VM

Successfully installed VMWare Tools on the VM

Could not successfully log in on the VM: Guest tools is not running

\*** glibc detected *** ./a.out: double free or corruption (!prev): 0x08811068 ***

======= Backtrace: =========

/lib32/libc.so.6\[0xf7a7f3ed]

/lib32/libc.so.6(cfree+0x90)\[0xf7a82a50]

/usr/lib/libvmware-vix.so.0(FoundryAsyncOp_DeleteAsyncOp+0x155)\[0xf7bc43d5]

/usr/lib/libvmware-vix.so.0(FoundryAsyncOp_FinishAsyncOp+0xca)\[0xf7bc3eec]

/usr/lib/libvmware-vix.so.0\[0xf7beb038]

/usr/lib/libvmware-vix.so.0\[0xf7ba941d]

/usr/lib/libvmware-vix.so.0\[0xf7ba91cd]

/usr/lib/libvmware-vix.so.0\[0xf7bc3471]

/usr/lib/libvmware-vix.so.0\[0xf7b9b579]

/usr/lib/libvmware-vix.so.0\[0xf7b9ab01]

/usr/lib/libvmware-vix.so.0\[0xf7b9aee4]

/usr/lib/libvmware-vix.so.0(Poll_LoopTimeout+0x165)\[0xf7b99f1d]

/usr/lib/libvmware-vix.so.0\[0xf7bc3f87]

/usr/lib/libvmware-vix.so.0\[0xf7bd19ff]

/lib32/libpthread.so.0\[0xf7a0031b]

/lib32/libc.so.6(clone+0x5e)\[0xf7ae879e]

======= Memory map: ========

08048000-08049000 r-xp 00000000 08:02 5324889 /home/pdovy/test/a.out

08049000-0804a000 rw-p 00000000 08:02 5324889 /home/pdovy/test/a.out

0804a000-088b1000 rw-p 0804a000 00:00 0 \[heap]

f7500000-f7521000 rw-p f7500000 00:00 0

f7521000-f7600000 ---p f7521000 00:00 0

f76ed000-f76ee000 ---p f76ed000 00:00 0

f76ee000-f776e000 rwxp f76ee000 00:00 0

f776e000-f776f000 rw-p f779f000 00:00 0

f77c0000-f77c1000 ---p f77c0000 00:00 0

f77c1000-f7841000 rwxp f77c1000 00:00 0

f7841000-f7850000 r-xp 00000000 08:02 4423700 /lib32/libresolv-2.5.so

f7850000-f7852000 rw-p 0000f000 08:02 4423700 /lib32/libresolv-2.5.so

f7852000-f7854000 rw-p f7852000 00:00 0

f7854000-f7858000 r-xp 00000000 08:02 4423693 /lib32/libnss_dns-2.5.so

f7858000-f785a000 rw-p 00003000 08:02 4423693 /lib32/libnss_dns-2.5.so

f785a000-f785c000 r-xp 00000000 08:02 4423732 /lib32/libnss_mdns4_minimal.so.2

f785c000-f785d000 rw-p 00001000 08:02 4423732 /lib32/libnss_mdns4_minimal.so.2

f785d000-f7866000 r-xp 00000000 08:02 4423694 /lib32/libnss_files-2.5.so

f7866000-f7868000 rw-p 00008000 08:02 4423694 /lib32/libnss_files-2.5.so

f7868000-f7870000 r-xp 00000000 08:02 4423696 /lib32/libnss_nis-2.5.so

f7870000-f7872000 rw-p 00007000 08:02 4423696 /lib32/libnss_nis-2.5.so

f7872000-f7885000 r-xp 00000000 08:02 4423691 /lib32/libnsl-2.5.so

f7885000-f7887000 rw-p 00012000 08:02 4423691 /lib32/libnsl-2.5.so

f7887000-f7889000 rw-p f7887000 00:00 0

f7889000-f7890000 r-xp 00000000 08:02 4423692 /lib32/libnss_compat-2.5.so

f7890000-f7892000 rw-p 00006000 08:02 4423692 /lib32/libnss_compat-2.5.so

f7892000-f78be000 r-xp 00000000 08:02 1146890 /usr/lib/vmware/lib/libssl.so.0.9.7/libssl.so.0.9.7

f78be000-f78c1000 rw-p 0002b000 08:02 1146890 /usr/lib/vmware/lib/libssl.so.0.9.7/libssl.so.0.9.7

f78c1000-f79a2000 r-xp 00000000 08:02 741748 /usr/lib/vmware/lib/libcrypto.so.0.9.7/libcrypto.so.0.9.7

f79a2000-f79b4000 rw-p 000e0000 08:02 741748 /usr/lib/vmware/lib/libcrypto.so.0.9.7/libcrypto.so.0.9.7

f79b4000-f79b9000 rw-p f79b4000 00:00 0

f79b9000-f79cc000 r-xp 00000000 08:02 433481 /usr/lib32/libz.so.1.2.3

f79cc000-f79cd000 rw-p 00012000 08:02 433481 /usr/lib32/libz.so.1.2.3

f79cd000-f79d2000 r-xp 00000000 08:02 4423687 /lib32/libcrypt-2.5.so

f79d2000-f79d4000 rw-p 00004000 08:02 4423687 /lib32/libcrypt-2.5.so

f79d4000-f79fb000 rw-p f79d4000 00:00 0

f79fb000-f7a0e000 r-xp 00000000 08:02 4423699 /lib32/libpthread-2.5.so

f7a0e000-f7a10000 rw-p 00013000 08:02 4423699 /lib32/libpthread-2.5.so

f7a10000-f7a12000 rw-p f7a10000 00:00 0

f7a12000-f7a14000 r-xp 00000000 08:02 4423688 /lib32/libdl-2.5.so

f7a14000-f7a16000 rw-p 00001000 08:02 4423688 /lib32/libdl-2.5.so

f7a16000-f7a17000 rw-p f7a16000 00:00 0

f7a17000-f7b52000 r-xp 00000000 08:02 4423685 /lib32/libc-2.5.so

f7b52000-f7b53000 r--p 0013b000 08:02 4423685 /lib32/libc-2.5.so

f7b53000-f7b55000 rw-p 0013c000 08:02 4423685 /lib32/libc-2.5.so

f7b55000-f7b58000 rw-p f7b55000 00:00 0

f7b58000-f7ea9000 r-xp 00000000 08:02 432171 /usr/lib/libvmware-vix.so.0.0.0.old.0

f7ea9000-f7eba000 rw-p 00351000 08:02 432171 /usr/lib/libvmware-vix.so.0.0.0.old.0

f7eba000-f7f62000 rw-p f7eba000 00:00 0

f7f6b000-f7f76000 r-xp 00000000 08:02 433478 /usr/lib32/libgcc_s.so.1

f7f76000-f7f77000 rw-p 0000a000 08:02 433478 /usr/lib32/libgcc_s.so.1

f7f77000-f7f7a000 rw-p f7f77000 00:00 0

f7f7a000-f7f96000 r-xp 00000000 08:02 4423682 /lib32/ld-2.5.so

f7f96000-f7f98000 rw-p 0001b000 08:02 4423682 /lib32/ld-2.5.so

ffa71000-ffa76000 rwxp ffa71000 00:00 0 \[stack]

ffffe000-fffff000 r-xp ffffe000 00:00 0 \[vdso]

Aborted (core dumped)

[pdovy]

After manually installing VMWare Tools on my Red Hat VM, and then starting the tools background program - calling VixVM_LoginInGuest() hangs forever. Are there any caveats to using this under a linux guest OS?

[lemke]

Add in a VixVM_WaitForToolsInGuest() between the PowerOn and the LoginInGuest(). PowerOn comes back when the virtual hardware is powered on, which is long before the OS is actually up. Waiting for the tools ensures the OS is fully bvooted and the tools are running and ready for commands.

[pdovy]

I'm able to log in once the VM has booted and is sitting at the login screen - however if I have the API power on the VM and then call VixVM_WaitForToolsInGuest, it returns immediately and the subsequent login attempt fails with the error message that tools is not running.

Is this expected behavior? I have no real other way of programmatically determining when the OS has booted other than attempting to log in continuously until it eventually succeeds because the machine has finally booted.

[admin]

Are you getting back any error from VixVM_WaitForToolsInGuest()?

Also, would you mind posting the code where you call VixVM_WaitForToolsInGuest()?

[zerovoid]

I too have experienced this... my solution was...

log("waitForToolsVM")

returnVal = waitForToolsVM()

If (returnVal=True) Then

log("Success")

Else

log("Error: " & functionErrorMessage) : exit sub

End If

log("Note: loginVM is expected to fail once if vm was not previously" & vbcrlf _

& " powered on, will retry up to 2 times after first failure")

logincount = 0 : returnVal = False

do while (returnVal = False)

if logincount > 2 then log("Error: Unable to Login") : : exit sub

log("loginVM : attempt " & logincount)

returnVal = loginVM()

If (returnVal=True) Then

log("Success")

Else

log("Error: " & functionErrorMessage)

End If

logincount = logincount+1

loop

As my comment states...

Note: loginVM is expected to fail once if vm was not previously powered on, will retry up to 2 times after first failure.

The second time it attempts, it connects fine.

This excerpt is from VixCOM and WSF/VBS

[admin]

Ok, that shouldn't[/i] be necessary. If you call WaitForToolsInGuest() while watching the guest in the service console, at what point in the boot process does it seem that WaitForToolsInGuest() finishes?

Can you post you implementation of waitForToolsVM()?

[zerovoid]

WaitForToolsInGuest() finishes after windows explorer seems to be loaded, in the time prior to the profile being fully loaded, like desktop icons and custom backgrounds. Around the normal time for windows services to be loaded.

' Needs nothing

Function waitForToolsVM()

'Checks for prereq based off of stage

prereqReturn = checkPrereq(4)

If Not (prereqReturn = True) Then waitForToolsVM = prereqReturn : exit function

set job = vm.WaitForToolsInGuest(0, Nothing)

myerr = job.WaitWithoutResults()

If VixLibA.ErrorIndicatesFailure(myerr) Then

on error resume next

waitForToolsVM = false

functionErrorCode = "-100" : functionErrorMessage = "error" : errMsg = VixLibA.GetErrorText(myerr, Nothing) : functionErrorCode = myerr : functionErrorMessage = errMsg

isToolsLoaded = False

waitForToolsVM = false : exit function

End If

isToolsLoaded = True

waitForToolsVM = True

End Function

Like I said, it calls the function when it should, but the first time reports an error, and the second time works perfectly if the machine is booted from an powered down state. If the system is already running, everything works fine.

' Needs vmUser and VMPass passed to it

Function loginVM()

'Checks for prereq based off of stage

prereqReturn = checkPrereq(5)

If Not (prereqReturn = True) Then loginVM = prereqReturn : exit function

set job = vm.LoginInGuest(vmUser, VMPass, 0, Nothing)

myerr = job.WaitWithoutResults()

If VixLibA.ErrorIndicatesFailure(myerr) Then

on error resume next

loginVM = false

functionErrorCode = "-100" : functionErrorMessage = "error" : errMsg = VixLibA.GetErrorText(myerr, Nothing) : functionErrorCode = myerr : functionErrorMessage = errMsg

isLoggedIn = False

loginVM = false : exit function

End If

isLoggedIn = True

loginVM = True

End Function

As you can probably tell, it was written as a class in another language and ported eventually to VBS. Hence variables are assigned in the setters and getters rather than being passed to the function... atleast for the non-override ones, which were removed from the VBS version due to some language limitations.

It works great with the work-around though.

BTW the Guest is Win2K3, if that helps.

[admin]

Are you logging in as an actual user on the virtual machine, or are you using one of the constants from vix.h/the type library (such as VIX_CONSOLE_USER_NAME)?

[zerovoid]

Actual user of the Virtual Machine.

I am going to break this thread off into its own, so people can quickly reference it as a different question.

[zerovoid]

Please respond to LoginInGuest issues in

http://www.vmware.com/community/thread.jspa?threadID=87550