pomattic
Contributor
Contributor

VI SDK session disconnecting (AxisFault) session not authenticate

Hi,

We are using the Java VI API 2.5 to communicate with VMware ESX Server 3.5.0.

We automate certain tasks that we want performed (say for example, power on machine, power off, revert to snapshot, attach device, and similar tasks).

The issue that is occurring is that the connection that we establish with the server only last for a short period of time. We require it to allow complete automatic for an 'indefinite' period of time (The length of time the connection remains for varies but these tasks can be run for about a day on average (sometimes longer, sometimes shorter) and then the session disconnects.)

{below I have included more details on my implementation}

Has anyone had such problems? (the forum has a couple references but no continuous discussions on this topic) Does anyone know what is causing this error? Does anyone know of a solution or where to look for a solution?

Details:

This is how we are establishing our initial connection: (with some changes to make things local and vague : )

ManagedObjectReference svcRef = new ManagedObjectReference();

svcRef.setType("ServiceInstance");

svcRef.set_value("ServiceInstance");

VimServiceLocator locator = new VimServiceLocator();

locator.setMaintainSession(true);

VimPortType connexion = locator.getVimPort(new URL("https://ip_add/sdk"));

ServiceContent sic = connexion.retrieveServiceContent(svcRef);

sindex = sic.getSearchIndex();

if (sic.getSessionManager() != null)

UserSession session = connexion.login(sic.getSessionManager(), "username", "password", null);

When this connection is established tasks are called on the connexion variable, these calls are so written that we can catch the Exception that occurs when we have been disconnected.

For Example:

try {

return connexion.findByInventoryPath("something", "something");

}

catch (RuntimeFault e) {e.printStackTrace();}

catch (RemoteException e) {e.printStackTrace();}

And the lovely print we receive when we have been thrown off is usually like such:

AxisFault

faultCode: ServerFaultCode

faultSubcode:

faultString: The session is not authenticated.

faultActor:

faultNode:

faultDetail:

NotAuthenticatedFault:80System.Read

The session is not authenticated.

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)

at java.lang.reflect.Constructor.newInstance(Unknown Source)

at java.lang.Class.newInstance0(Unknown Source)

at java.lang.Class.newInstance(Unknown Source)

at org.apache.axis.encoding.ser.BeanDeserializer.(BeanDeserializer.java:104)

at org.apache.axis.encoding.ser.BeanDeserializer.(BeanDeserializer.java:90)

at com.vmware.vim25.NotAuthenticated.getDeserializer(NotAuthenticated.java:87)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

at java.lang.reflect.Method.invoke(Unknown Source)

at org.apache.axis.encoding.ser.BaseDeserializerFactory.getSpecialized(BaseDeserializerFactory.java:154)

at org.apache.axis.encoding.ser.BaseDeserializerFactory.getDeserializerAs(BaseDeserializerFactory.java:84)

at org.apache.axis.encoding.DeserializationContext.getDeserializer(DeserializationContext.java:464)

at org.apache.axis.encoding.DeserializationContext.getDeserializerForType(DeserializationContext.java:547)

at org.apache.axis.message.SOAPFaultDetailsBuilder.onStartChild(SOAPFaultDetailsBuilder.java:157)

at org.apache.axis.encoding.DeserializationContext.startElement(DeserializationContext.java:1035)

at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(Unknown Source)

at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)

at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(Unknown Source)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)

at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)

at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)

at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)

at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)

at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)

at javax.xml.parsers.SAXParser.parse(Unknown Source)

at org.apache.axis.encoding.DeserializationContext.parse(DeserializationContext.java:227)

at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:696)

at org.apache.axis.Message.getSOAPEnvelope(Message.java:435)

at org.apache.axis.transport.http.HTTPSender.readFromSocket(HTTPSender.java:796)

at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:144)

at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)

at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)

at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)

at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)

at org.apache.axis.client.Call.invokeEngine(Call.java:2784)

at org.apache.axis.client.Call.invoke(Call.java:2767)

at org.apache.axis.client.Call.invoke(Call.java:2443)

at org.apache.axis.client.Call.invoke(Call.java:2366)

at org.apache.axis.client.Call.invoke(Call.java:1812)

I am going to continue to work on solving this and will keep things up to date here if any solutions are found.

I would appreciate any help I can get on this. If you would like more detail or more output let me know, I have plenty : )

Thanks in advance for your help.

M@

0 Kudos
10 Replies
Steve_Jin
Expert
Expert

Did you call the following code right after you login? Or it's after a while? There is a chance that your session has timed out. Please try the following code right after you login and see whether you get NotAuthenticatedFault or not.

try {

return connexion.findByInventoryPath("something", "something");

}

catch (RuntimeFault e) {e.printStackTrace();}

catch (RemoteException e) {e.printStackTrace();}

Steve JIN Author of VMware VI and vSphere SDK; Creator of open source VI Java API (http://vijava.sf.net); Blogger at http://www.doublecloud.org
0 Kudos
pomattic
Contributor
Contributor

Thanks for the quick response.

The code gets called both "right after" the connection is made and "after a while".

The program that uses this is intended to automate these tasks. So the connection is made once at the start then the tasks are repeatedly executed. And this execution of the tasks occurs numerous times successfully. Proving that the session is established and authenticated. And then after an extended period of time (after numerous repetitions of similar tasks) that exception is caught and the error message I have provided is printed.

So the initial execution (what I understand you are asking me to do) IS successful. No exception is thrown.

Hope this further explains what is happening. Let me know if I misunderstood you.

M@

Message was edited by: pomattic (spelling corrections made)

0 Kudos
akutz
Hot Shot
Hot Shot

It is probably timing out. Take a look at this sample code that

detects timeouts and logs the app back in:

http://code.google.com/p/akutz/source/browse/trunk/vmware/utilities/monet/java/monet/src/monet/Main....

--

-a

"condensing fact from the vapor of nuance"

Steve_Jin
Expert
Expert

It seems to me your session has timed out when you get the exception. You can change the timeout time as follows:

VimPortType vimService = serviceLocator.getVimPort(url);

((org.apache.axis.client.Stub)vimService).setTimeout(1200000); //change it to your possible maximum here.

BTW, since you are using Java, I would suggest you to consider using VI Java API I created. It will make your code much shorter and much more readable. A quick example here:

import java.net.URL;

import com.vmware.vim25.*;

import com.vmware.vim25.mo.*;

import com.vmware.vim25.mo.util.*;

/**

  • This sample shows how to get hold of the first VirtualMachine in an inventory and retrieve its properties.

  • @author sjin

*/

public class HelloVM

{

public static void main(String[] args) throws Exception

{

CommandLineParser clp = new CommandLineParser(new OptionSpec[]{}, args);

String urlStr = clp.get_option("url");

String username = clp.get_option("username");

String password = clp.get_option("password");

ServiceInstance si = new ServiceInstance(new URL(urlStr), username, password, true);

Folder rootFolder = si.getRootFolder();

VirtualMachine vm = (VirtualMachine) new InventoryNavigator(rootFolder).searchManagedEntities("VirtualMachine")[0];

VirtualMachineConfigInfo vminfo = vm.getConfig();

VirtualMachineCapability vmc = vm.getCapability();

System.out.println("Hello " + vm.getName());

System.out.println("GuestOS: " + vminfo.getGuestFullName());

System.out.println("Multiple snapshot supported: " + vmc.isMultipleSnapshotsSupported());

si.getServerConnection().logout();

}

}

You can download it for free at http://vijava.sf.net

Steve JIN Author of VMware VI and vSphere SDK; Creator of open source VI Java API (http://vijava.sf.net); Blogger at http://www.doublecloud.org
0 Kudos
pomattic
Contributor
Contributor

Thank you very much for your replies.

When I have the time I will weigh them both and then test them to see if they solve my problem.

I will let you know how it went.

Thanks again!

M@

0 Kudos
pomattic
Contributor
Contributor

Hi,

So I have looked through both of your solutions to see if they would help me with my problem. And sadly they do not work.

VimPortType vimService = serviceLocator.getVimPort(url);

((org.apache.axis.client.Stub)vimService).setTimeout(1200000);

I changed the timeout time to 60000 what I was made to believe from documentation to be 1 minute (60 000 milliseconds). And nothing happened, this was done in an attempt to reproduce the error I was getting. But nothing happened. The program ran for well over a minute.

As for the solution that akutz gave me. It is good for your implementation... but not for ours. The below snippet (I believe) is far too CPU intensive, no breaks just looping. So our required solution would be to throw in some sleep time in there. Which creates the small problem of, what if during the time the thread is sleeping the command gets executed and throws an exception. Then we have already lost an execution...

while ( m_bool_stay_logged_in )

{

// We can skip this turn if we're already logged in.

if ( m_bool_logged_in ) continue;

...

We have a tentative solution that may work. An example code snippet is:

while(true){

try {

return connexion.findByInventoryPath(index, path+name);

}

catch (RuntimeFault e) {

e.printStackTrace();

System.exit(0); }

catch (RemoteException e) { // this is as close to the excetion as we can get

e.printStackTrace();

reconnect(); // calls a function which should reconnect.

}

}

Not the nicest of solutions, but it may work. It has yet to be tested (since this error takes days to reproduce). Let me know what you think of this solution, if you find any faults. Or if you know how to get closer to the exception that is being thrown, so that it will be the only one to cause a reconnection, since RemoteException I beleive may entail more then just this error. Any other suggestions, or letting me know if I missed something in your previous posts would also be appreciated.

Thanks for helping out!

M@

0 Kudos
Steve_Jin
Expert
Expert

Hi Pomattic,

Good to know you've got a solution. As long as it works, it should be fine. By default, the server side will terminate an inactive session in about 30 minutes. Alternatively you can use a separate thread to fake a quick call every 25 minutes, just to keep your session active.

-Steve

Steve JIN Author of VMware VI and vSphere SDK; Creator of open source VI Java API (http://vijava.sf.net); Blogger at http://www.doublecloud.org
pomattic
Contributor
Contributor

Well the solution is yet to be tested, so we shall see how good it is : )

"the server side will terminate an inactive session in about 30 minutes"

Strange things is that my connection is "never!" idle for 30 minutes. I doubt it is ever inactive for longer then 5 minutes.

Anyhow, will keep the forum posted on wether or not the proposed solution will work.

M@

0 Kudos
pomattic
Contributor
Contributor

So the solution works (sort of). We get disconnected and upon noticing this (catching exception) we reconnect.

Now a new problem deeply related to the previous one is noticed.

All the Managed Object References (mors) are out of date. Or, to illustrate, in the case where we want to power off a machine and get disconnected before the task was complete the output likes to say: (first two lines are standard output by us)

Fri Jul 04 16:39:32 EDT 2008: Attempting connexion to ESX server...

Connected.

AxisFault

faultCode: ServerFaultCode

faultSubcode:

faultString: The request refers to an object that no longer exists or has never existed.

faultActor:

faultNode:

faultDetail:

ManagedObjectNotFoundFault:haTask-848-vim.VirtualMachine.powerOff-209589

The request refers to an object that no longer exists or has never existed.

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)

at java.lang.reflect.Constructor.newInstance(Unknown Source)

at java.lang.Class.newInstance0(Unknown Source)

at java.lang.Class.newInstance(Unknown Source)

at org.apache.axis.encoding.ser.BeanDeserializer.(BeanDeserializer.java:104)

at org.apache.axis.encoding.ser.BeanDeserializer.(BeanDeserializer.java:90)

at com.vmware.vim25.ManagedObjectNotFound.getDeserializer(ManagedObjectNotFound.java:119)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

at java.lang.reflect.Method.invoke(Unknown Source)

at org.apache.axis.encoding.ser.BaseDeserializerFactory.getSpecialized(BaseDeserializerFactory.java:154)

at org.apache.axis.encoding.ser.BaseDeserializerFactory.getDeserializerAs(BaseDeserializerFactory.java:84)

at org.apache.axis.encoding.DeserializationContext.getDeserializer(DeserializationContext.java:464)

at org.apache.axis.encoding.DeserializationContext.getDeserializerForType(DeserializationContext.java:547)

at org.apache.axis.message.SOAPFaultDetailsBuilder.onStartChild(SOAPFaultDetailsBuilder.java:157)

at org.apache.axis.encoding.DeserializationContext.startElement(DeserializationContext.java:1035)

at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(Unknown Source)

at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)

at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(Unknown Source)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)

at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)

at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)

at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)

at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)

at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)

at javax.xml.parsers.SAXParser.parse(Unknown Source)

at org.apache.axis.encoding.DeserializationContext.parse(DeserializationContext.java:227)

at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:696)

at org.apache.axis.Message.getSOAPEnvelope(Message.java:435)

at org.apache.axis.handlers.soap.MustUnderstandChecker.invoke(MustUnderstandChecker.java:62)

at org.apache.axis.client.AxisClient.invoke(AxisClient.java:206)

at org.apache.axis.client.Call.invokeEngine(Call.java:2784)

at org.apache.axis.client.Call.invoke(Call.java:2767)

at org.apache.axis.client.Call.invoke(Call.java:2443)

at org.apache.axis.client.Call.invoke(Call.java:2366)

at org.apache.axis.client.Call.invoke(Call.java:1812)

at com.vmware.vim25.VimBindingStub.retrieveProperties(VimBindingStub.java:26671)

...

So the task I am waiting to complete, the mor to it is useless. Maybe the task has completed so the mor is meaningless or the mor changes (not sure exactly how that little number works).

Either way we need to be able to continue and we need to know that the task is done.

Anyone have similar issues? Any thoughts on how to tackle this?

M@

(should I have posted this in a new thread?...since it is a new issue...)

Message was edited by: pomattic (corrected some spelling...)

0 Kudos
pomattic
Contributor
Contributor

So a solution to my problem has been found. It is rather specific to my problem so not sure how helpful of a solution it will be, but some things noticed may be helpful to others in the future. I will post what I did and what other possibilities I found.

Basicly we reconnect like this, we run the task and if a RemoteException gets thrown we will reconnect based on this. The task is placed in a loop that will not exit until the task will be complete (example: infinite loop surrounding method (including catch) that will do task, then return; so if the return is never reach because of exception the method will keep looping (watch for real infinite loops))

Sample:

// task to ESX server

catch (RemoteException e) {

System.out.println("RemoteException in findByName. Reconnect.");

System.err.println("RemoteException in findByName. Reconnect.");

e.printStackTrace();

reconnect();

}

This works great for reconnecting and the response from akutz can help you with how to do this.

The problem of the MOR is then handled by me in this way: It is pretty much ignored... where the exception is known to happen we are catching any throwable, and we throw our own exception. That exception then when caught by us allows us to know that the current 'step' was not complete and so it is noted, abandoned and handled later. Very specific to what I am doing since it is OK to abandon a 'step' for now and try it again later.

A solution I thought of that I did not try is to similarly throw a custom exception, and throw it as far as the furthest MOR. Then to handle it there by testing to see if the task completed (example if the task was PowerOn test to see if machine is poweredOn...) or if it apears to not be, to 'regenerate' the MOR. (NOTE: My understanding of MORs may be faulty, so this may not be the best of ideas.) The important thing to do is to catch the correct exception. I was about to start working on this when we found the other solution so did not get far, this was never tested, barely proof read, but a start that may be useful is something along the lines of:

catch (RemoteException e) {

// includes InvalidProperty

// includes RuntimeFault

if(e instanceof org.apache.axis.AxisFault) {

System.err.println("org.apache.axis.AxisFault");

org.w3c.dom.Element [] errors = ((org.apache.axis.AxisFault)e).getFaultDetails();

if (errors!=null){

if (errors.length>1)

System.err.println("More then one element ("errors.length")of fault details. Handling one at a time.");

for(int i=0; i

System.err.println("node"i"Name: " +errors[i].getNodeName());

}

for(int i=0; i

if (errors[i].getNodeName().contains("InvalidPropertyFault"))

//handle

else if (errors[i].getNodeName().contains("NotAuthenticatedFault"))

//handle

else if (errors[i].getNodeName().contains("TaskInProgressFault"))

//handle

else if (errors[i].getNodeName().contains("ManagedObjectNotFoundFault"))

//handle

// throw that new excpetion

else

//handle

}

}

else {

System.err.println("AxisFault has null error Elements.");

e.printStackTrace();

System.exit(0);

}

}

Thanks for all the help! Hope I don't misguide anyone, but instead that this can help some out.

M@

0 Kudos