This error happening when trying to add Datastore:
build: VMware-BigDataExtensions-1.0.0.332-1315424_OVF10.ova
error log:
/opt/serengeti/logs/serengeti.log
2013 Dec 03 18:49:36,857+0000 ERROR http-8443-2| com.vmware.bdd.rest.RestResource: rest call error
com.vmware.bdd.exception.BddException: Internal error: REST API transport layer error.
at com.vmware.bdd.exception.BddException.INTERNAL(BddException.java:77)
at com.vmware.bdd.exception.BddException.wrapIfNeeded(BddException.java:72)
at com.vmware.bdd.rest.RestResource.handleException(RestResource.java:603)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
...
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:679)
Caused by: com.vmware.bdd.utils.AuAssert: FAILURE
at com.vmware.bdd.utils.AuAssert.FAILURE(AuAssert.java:28)
at com.vmware.bdd.utils.AuAssert.check(AuAssert.java:43)
at com.vmware.bdd.utils.AuAssert.check(AuAssert.java:50)
at com.vmware.bdd.service.resmgmt.impl.ResourceService.refreshDatastore(ResourceService.java:196)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
...
at sun.proxy.$Proxy40.addDatastores(Unknown Source)
at com.vmware.bdd.rest.RestResource.addDatastore(RestResource.java:441)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:176)
at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:440)
at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:428)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherSer
Good find. I fixed the issue of name resolution, uniformly for the host, vcenter, serengeti. Still didn't work. So I reinstalled serengeti because the name resolution didn't fix the next REST problem related to:
1) 2013 Dec 05 21:27:10,076+0000 ERROR main| com.vmware.bdd.web.ResourceInitializer: Resource initialization failed.
com.vmware.aurora.exception.VcException: Failed to execute vCenter Server command: sun.proxy.$Proxy101 cannot be cast to com.vmware.vim.binding.vim.ClusterComputeResource.
2) tomcat failed to generate uuid in serengeti.properties file in 5 minutes. It's probably a vc connection issue, please check /opt/serengeti/logs/serengeti.log
I've attached the new logs based on reinstall and name resolution corrections.
Thanks,
M
I need this fixed to do configuration, I fail again when creating a new Hadoop Cluster:
2013 Dec 05 22:39:50,272+0000 INFO http-8443-4| com.vmware.bdd.manager.ClusterManager: added automation resource pools:
2013 Dec 05 22:39:50,273+0000 INFO http-8443-4| com.vmware.bdd.aop.logging.ExceptionHandlerAspect: Aspect for exception handling
2013 Dec 05 22:39:50,273+0000 ERROR http-8443-4| com.vmware.bdd.aop.logging.ExceptionHandlerAspect: Service error
com.vmware.bdd.exception.BddException: Internal error: Service AOP.
at com.vmware.bdd.exception.BddException.INTERNAL(BddException.java:77)
at com.vmware.bdd.exception.BddException.wrapIfNeeded(BddException.java:72)
Is there a .jar(s) I can replace, or a new .ova for the fix?
M
I found this reference to my current problem:
https://issuetracker.springsource.com/browse/SERENGETI-1042?attachmentSortBy=dateTime
And a comment:
Hong Cheng (c)<https://issuetracker.springsource.com/secure/ViewProfile.jspa?name=hongcheng> added a comment - 30/Jan/13 11:03 PM
It is fixed in the latest build.
How can I get the latest build?
Thanks
MN
Did you check the FQDN of VCVA ? Plz make sure it is not null and correct.
That reference you found differ with current problem. And that fixing has been put into the BDE 1.0. You said you had fixed the issue of name resolution, do you ping nunezm-vcenter.mater.us successfully now ?
Both forward and reverse lookups are working with fully qualified name. All components pings with FQDN correctly to my knowledge including ESX, vCenter, and the Serengeti Management-Server
Does it bother you at all that there is absolutely no useful information given to the user either in the UI of the CLI about where problems are occurring. This is basic lack of fit and finish. I shouldn't have to scrape your log files to understand where root problems occur.
I'm about to bail on this and go to a useful implementation. I'l just build up my own VM's as a Hadoop cluster and not waste my time here.
I do appreciate your help, but you shouldn't have to be doing this.
MN
Hi @mknunez,
I'm a developer of BDE. We are glad you are trying BDE. Sorry for the REST API error you suffered. We do want to help you to use BDE smoothly.
Would you mind deploying a new BDE 1.0 vApp again (the same build you used) after the following items are done ?
1) add all ESX hosts into a vcenter cluster
2) make sure DNS/IP forward and reverse resolution works
3) BDE vApp must deployed in a resourcepool
4) any other requirements mentioned in BDE Admin Guide for deploying BDE vApp
If you still see the REST API error or any other error, please send me the files in /opt/serengeti/logs/ . I believe we can get you through.
Thanks
Jesse Hu
There is a known issue on VC 5.5, please check whether that's applicable to your environment.
vCenter Virtual Appliance sets a wrong hostname after deployment
If the vCenter Server Appliance is deployed with OVF environment that has static IP configuration and the hostname is blank, reverse lookup from the IP is not performed properly.
To resolve this issue, set the desired vCenter Server Appliance hostname in the OVF properties during deployment.
We want to help you out the current issue.
could you elaborate out how you fixed the host name issue on vcva? Because it's used widely in many configuration within the vm, so simply manually changing host name or dns resolving file to make it pingable doesn't remove all issues underneath.
THe recommended way to resolve this vcva host name from vc engineering team, is to fresh reinstall vc and make sure host name is input as expected. Please take a look at the notes I sent out before.
I'm sorry for what you met during your experience to use BDE. I know that at this time, you probably just want to manual do everything related to Hadoop cluster.
But I still want to explain a little bit in case some time later you might still want to try BDE in your environment.
Basically for why this issue happen, is that VC 5.5 is not installed correctly, but you might not found problem in some other workload. All the solutions in this thread already explained how to fix env DNS issue, I think you've tried and fixed that issue. But after that you still need to reinstall VC 5.5 to completely remove this problem.
But if you don't reinstall VC 5.5, there is still one workaround in BDE. That is to change vc.properties at /opt/serengeti/conf directly. Replacethe wrong VC FQDN or IP in that file with correct ip, so that Serengeti will get the correct VC's URL to go ahead operations afterwards. This is the root cause for what you saw in the log.
And then, you need to remove /opt/serengeti/etc/serengeti-firstboot if it exists, and execute /opt/serengeti/sbin/serengeti-firstboot.rb
It's a little bit tricky so we didn't describe this workaround too much, but anyway, it's solution to continue your work in this status.
In addition, as you said, the error message in CLI/UI does not say anything useful for this error, that's annoying. We'll improve it in next release.
Thanks for giving us your feedback, and looking forward your next chance to try our product.
BR
Emma
We have added a discussion topic about this issue at Solution to REST API transport layer error during operations after deploying Big Data Extensions where you may find the cause related to your environments.
Regards,
Jun