VMware Cloud Community
balawiz
Enthusiast
Enthusiast
Jump to solution

vRA Multiple Linux VMs Deployment Timeout Error

vRA 7.2 with vSphere 6 Environment.

When 15 and above linux/windows vms deployed with the single blueprint catalog, the deployment is getting failed at some point with the error "Request failed: "testvm01" not found, and possibly deleted before provisioning finished". Also, we are seeing the following error in vRA Infrastructure-> Monitoring-> logs section from manager service.

Please note that this type of error occurs mostly into Linux (CentOS7) vms and also after creating few of same type of linux vms successfully with software component installed. The software component is actually just single line of curl command to install puppet agent. Another note that the Extensibility LifeCycle Message timeout value is 45mins.

testvm01: InstallSoftwareWorkflow Exception: Machine testvm01: InstallSoftwareWorkflow. Install software work item timeout.

System.ApplicationException: Machine testvm01: InstallSoftwareWorkflow. Install software work item timeout.

at DynamicOps.External.SoftwareWorkflows.InstallSoftwareWorkflow.Error_SendWorkitem(Object sender, EventArgs e)

at System.Workflow.ComponentModel.Activity.RaiseEvent(DependencyProperty dependencyEvent, Object sender, EventArgs e)

at System.Workflow.Activities.CodeActivity.Execute(ActivityExecutionContext executionContext)

at System.Workflow.ComponentModel.ActivityExecutorOperation.Run(IWorkflowCoreRuntime workflowCoreRuntime)

at System.Workflow.Runtime.Scheduler.Run()

Have checked the logs in the following places of failed (about to) vm's, but didn't get any clue except the below http transport error which was happening even for the successful VMs deployment. So, any further help/troubleshooting steps is appreciated.

Linux Guest Agent Connection Log

/usr/share/gugent/axis2/logs/gugent-axis.log

Linux Guest Agent Log

/usr/share/gugent/GuestAgent.log

Linux Application Services Agent logs

/opt/vmware-appdirector/agent/logs/agent_bootstrap.log

Linux Software Agent Logs

/opt/vmware-appdirector/agent/logs/darwin-agent<date>.log

[Fri Nov 16 11:50:36 2018] [...TRACE...] http_transport_sender.c(194) Entry:axis2_http_transport_sender_invoke

[Fri Nov 16 11:50:36 2018] [debug] http_transport_sender.c(246) ctx_epr:https://managerendpoint.domain.com:443/VMPS2

[Fri Nov 16 11:50:36 2018] [debug] http_transport_sender.c(805) using axis2 native http sender.

[Fri Nov 16 11:50:36 2018] [...TRACE...] http_sender.c(252) Entry:axis2_http_sender_send

[Fri Nov 16 11:50:36 2018] [debug] http_sender.c(416) msg_ctx_id:urn:uuid:e2b9cee2-e9d8-1e81-3906-000000000000

[Fri Nov 16 11:50:36 2018] [error] http_client.c(238) Error creating a socket. Most probably error returned by OS

[Fri Nov 16 11:50:36 2018] [error] http_client.c(239) Socket Creation failed.

[Fri Nov 16 11:50:36 2018] [error] http_client.c(544) client data stream  null or socket error for host managerendpoint.domain.com and 443 port

[Fri Nov 16 11:50:36 2018] [error] http_client.c(548) A read attempt(HTTP) for the reply without sending the request

[Fri Nov 16 11:50:36 2018] [error] http_sender.c(1381) status_code < 0

[Fri Nov 16 11:50:36 2018] [...TRACE...] http_transport_sender.c(596) Exit:axis2_http_transport_sender_invoke

[Fri Nov 16 11:50:36 2018] [error] engine.c(179) Transport sender invoke failed

Tags (1)
Reply
0 Kudos
1 Solution

Accepted Solutions
balawiz
Enthusiast
Enthusiast
Jump to solution

By checking with VMware support, re-installed the vRA agent to the template and deployed. The issue is not recurring so far.

View solution in original post

Reply
0 Kudos
2 Replies
daphnissov
Immortal
Immortal
Jump to solution

I would open an SR on this one, because it may be down to an internal concurrency and locking issue that they have an internal PR for. Possibly fixed in later versions.

Reply
0 Kudos
balawiz
Enthusiast
Enthusiast
Jump to solution

By checking with VMware support, re-installed the vRA agent to the template and deployed. The issue is not recurring so far.

Reply
0 Kudos