vRealize Automation 7.3 Installation Fails

[MuayThai88]

I am using the wizard to install vRealize Automation 7.3. When I run the installation the installation fails at 12% at "verify all services are running". I tried multiple things but no luck. Any advice on how to get past the "verify all service are running" portion of the installation? So I am using a distributed installation. 2 vRealize appliances (behind a NSX load balancer), 1 web IaaS Server (running website and model manager data service), 1 DEM server (running DEM Orchestrator Service and Manager Server Service) and 1 Agent Server (RunningAgent Service). I don't think it has to do with settings also because all servers are DNS resolvable and have network connectivity to one another. It fails at "verify all services are running" at the VA role (virtual appliance). I look at details and it says "Health Check Failed". I check the health and saw this error:

errors>

<error code="10114">

<message>Service Unavailable.</message>

<systemMessage>

The service shell-ui-app was not able to register the service information with the Component Registry service! This might cause other dependent services to fail. Error Message: Retriable operation failed after the maximum number of attempts - [200]. OperationDetails 'Registering [shell-ui-app] service into Component Registry'

</systemMessage>

</error>

</errors>

[MuayThai88]

I am using the Distributed setup so I only need one of each server. But I believe you can install multiple roles on one server which is what I will do.

All servers in the deployment are using time.nist.gov as their NTP server.

I am not adding a second appliance because I am using only 1 of each server.

The above picture got cut off but the services are being installed as follows: vRAIaaSWeb.Home.lan(Web/Model Manger, Manager Service), vRADem.Home.lan(DEM Service), vRAAgent.Home.lan(Agent Service).

All Prerequisites are green.

All Parameters are valid.

Generated new vRealize Appliance Certificate and kept existing.

Generated new Web Certificate and kept existing.

No need to create Manager Service Certificate because both Manager Server Service and Web role are on the same server.

No Load Balancers are being used in my deployment at this time.

Everything validates 100% like it always does.

Please zoom in with Google Chrome to see the pictures more clearly.

[sbeaver]

Did you install / upgrade the new management agent on the windows servers before the install?  You might need to try uninstalling and reinstalling the management agent.

[daphnissov]

Ok, some comments:

1. Your installation parameters don't line up with the roles you've assigned. You have set the Manager component to be installed on vRADem.Home.lan yet later you specify it on the web server. The Model Manager is a component of the website. The Manager is a separate component entirely. Go back through the wizard and specify the correct server as the Manager. It's probably easier if you put web and manager on the same box and the DEM worker and Agent on another. This is two Windows IaaS servers instead of three.
2. What are your DNS records, forward and reverse? Have you validated? Please post all IPs and hostnames.
3. Where does SpencerVraappliance.Home.lan resolve?
4. Specify all hostnames in lower case. Don't use mixed cases. If you have mixed in your DNS, change to lower case.
5. Using an external NTP server is not a good idea and should be pointed to an internal NTP server. Also, ensure the ESXi host(s) on which these servers are deployed have their times synched. Check all clocks for any skew and correct by pointing to an internal NTP server.

For simplicity's sake, can you consider just doing a minimal deployment here to reduce complexity while troubleshooting? There's something fundamentally wrong that needs to be identified, and until then doing just a single appliance and single Windows box will eliminate some variables.

Please address each of these and post back.

[MuayThai88]

Okay so, I started the installation over. I matched all the installation services as per the second image I posted. 2. All my DNS records resolve forward lookup and reverse lookup. 3. My vraappliance resolves to my internal DNS server and resolves both forward lookup and reverse lookup as well. 4. All hostnames are now completely in lowercase including my vraappliance. 5. All Windows servers and my vraappliance point to my internal NTP server 192.168.2.20.

IP Addresses: vraiaasweb.home.lan=192.168.2.41 (Web Service and Model Manager Data)

vraappliance.home.lan= 192.168.2.40 (I re-deployed this OVF file).

vradem= 192.168.2.42 (Manager Server Service, DEM Service)

vraagent= 192.168.2.43 (Agent Service)

I need to get this type of deployment working because at my job we will be doing a HA deployment which is more complicated. Also, I tried installing all the services onto 1 Windows server and it still failed with the same error. This is the error I am seeing this time:

[daphnissov]

Attach vcac-config.log, please.

[MuayThai88]

Where is the location of the log? I don’t see it.

[daphnissov]

It says in the failure message in the wizard.

[MuayThai88]

I see it, so I hit the "collect logs" button so as soon as it is finished running, I will attach the logs.

[MuayThai88]

Here is the log file. I attached it at the bottom.

[daphnissov]

I'm constantly seeing "Node unavailable" error messages, and where it's failing it hasn't even begun to process the other components which means something is fundamentally wrong with this deployment.

1. Show a screenshot of the vRA appliance in your vCenter's inventory that captures the name.
2. Edit settings on the VM and show the configuration of the virtual hardware.
3. Still in edit settings, show vApp options that have the OVF properties for the highlighted sections. Please expand and capture all settings in both categories.

4. Go into the VAMI of the appliance and screenshot the networking portion. Just close the wizard for now and don't restart.

Also, about your architecture, what you're attempting is a distributed but a non-HA architecture. Further, it doesn't make sense to put the Manager component on the same box as the DEM worker. If you absolutely must have a distributed vRA, do it with 1 Web; 1 Manager; and 1 DEM Worker + Agent.

[MuayThai88]

[daphnissov]

1. What version of vCenter is this?
2. Is this a nested virtualization environment?
3. How did you deploy this OVA file and where and when did you get it? Those properties I highlighted should be there.

[MuayThai88]

1. This is vCenter version 6.5.0.

2. My environment is nested. I built my ESXi Server in VMware Workstation Pro 12 and built vCenter within my ESXi Server.

3. I deployed the OVF file in my ESXi web console. I got the OVF file about 2 weeks ago from my.vmware.com.

[daphnissov]

2. Ah, ok. This might be some of the cause for problems. I've seen these types of deployments fail.

3. You must deploy the vRA appliance through vCenter and not ESXi, otherwise the OVF properties are not maintained. Re-deploy the appliance through the vSphere Web Client (Flex, not HTML5).

and

4. (new) Get an MD5 hash of the OVA and compare it to what is on VMware's site to ensure there is no corruption. Latest build file is VMware-vR-Appliance-7.3.0.536-5610496_OVF10.ova with the following published sums:

MD5SUM: 07494582f19cce3ffa267848663584c9

SHA1SUM: 35d39c9ab7c8f496cf03fe8d3ed846caa21127fb

SHA256SUM: 8b2a973ea553f9493c994990775c7e1371b4c70af4af8e50ac149fc6ba9132db

I'm not entirely positive that this will work in a nested environment, but I've not tried myself. For what that's worth. In any case, start with redeploying through vCenter, revert your Windows server(s) to clean state, then deploy the management agent on them once the new appliance is up.

[MuayThai88]

So originally I deployed the OVA file through vCenter  and had multiple failures. I But I will re-deploy the OVA through vCenter again and try the installation once more. The physical server that I am using only has 48GB of RAM. I think the issue my be related to this because I am thinking the VA doesn't have enough power to run the installation. Just a thought.

[daphnissov]

That's a possibility. Even if you had one physical ESXi host with that much RAM, you'd probably have an easier go of it, but nesting complicates things much more. Any way, you must deploy through vCenter. If you still get failures, post some screenshots and messages and we'll try and figure them out, but no promises in your setup.

[MuayThai88]

I am re-deploying the OVA now. And will continue the installation and let you know what happens.

Random question. So from what I read and understand about each component of the vRA deployment:

The Manager Server Service helps all the devices in the vRA deployment communicate with one another.

The DEM Orchestrator gives out tasks to the DEM Workers.

The DEM workers carry out the tasks such as deploying a new VM from a blueprint.

The Agent talks with outside cloud applications such as AWS or Microsoft Azure.

And the model manager data updates the web console with information provided by manager server service?

Is this correct?

[daphnissov]

You have some of it right. Except:

The Manager Server Service helps all the devices in the vRA deployment communicate with one another.

Kind of. The Manager service serves as the repository for IaaS entities and also has registration of other IaaS roles.

The DEM workers carry out the tasks such as deploying a new VM from a blueprint.

DEM workers carry out *some* tasks depending on the endpoint type. Deploying to the vSphere type is handled by the agent by communicating to the API endpoint on vCenter.

The Agent talks with outside cloud applications such as AWS or Microsoft Azure.

See previous.

And the model manager data updates the web console with information provided by manager server service?

Model Manager has the model data for what each IaaS type "looks like" for different resources like vSphere, AWS, etc.

[MuayThai88]

IaaS entities meaning the other servers with IaaS roles installed?

By the way, I am currently re-installing the vRA automation with a different setup. vraiaasweb (web role and model manager data), vradem (agent service, dem service), vraagent (manager server service). I know the names don't match up with services but I did not feel like renaming the vraagent server to match the service manager server service. lol.

[daphnissov]

Did the re-install fix anything once you deployed the appliance through vCenter?

IaaS entities like "models" of how a vSphere machine looks and its properties versus AWS, Azure, others.