VCenter 7.0 deployment from CLI stuck at waiting for deployment status

[krisjinka]

[root@clientVM1 lin64]# ./vcsa-deploy install /root/vcenter/config/embedded_vCSA_on_ESXi.json --accept-eula --no-esx-ssl-verify --no-ssl-certificate-verification --log-dir=/root/vcenter/log

Run the installer with "-v" or "--verbose" to log detailed information

Updating log file location, copying '/tmp/vcsaCliInstaller-2020-04-21-03-14-69v3uwr0/vcsa-cli-installer.log' to desired location as a backup: '/root/vcenter/log/workflow_1587438887538/vcsa-cli-installer.log.bak'

Consuming the installer build:15393115

Workflow log-dir /root/vcenter/log/workflow_1587438887538

=================================================== [START] Start executing Task: To validate CLI options at 03:14:47 ===================================================

Command line arguments verfied.

========= [SUCCEEDED] Successfully executed Task 'CLIOptionsValidationTask: Executing CLI optionsValidation task' in TaskFlow 'template_validation' at 03:14:47 =========

=========================================== [START] Start executing Task: To validate the syntax of the template. at 03:14:47 ===========================================

Template syntax validation for template '/root/vcenter/config/embedded_vCSA_on_ESXi.json' succeeded.

Syntax validation for all templates succeeded.

======== [SUCCEEDED] Successfully executed Task 'SyntaxValidationTask: Executing Template Syntax Validation task' in TaskFlow 'template_validation' at 03:14:47 ========

[START] Start executing Task: To check the version of each template, and for each older template that supports CEIP, convert it to the latest template format, and save

it to the Template Blackboard at 03:14:47

CEIP is not enabled because the template key 'ceip_enabled' in section 'ceip', subsection 'settings' in template '/root/vcenter/config/embedded_vCSA_on_ESXi.json' was

set to 'false'.

CEIP is not enabled because the template key 'ceip_enabled' in section 'ceip', subsection 'settings' in template '/root/vcenter/config/embedded_vCSA_on_ESXi.json' was

set to 'false'.

CEIP is not enabled because the template key 'ceip_enabled' in section 'ceip', subsection 'settings' in template '/root/vcenter/config/embedded_vCSA_on_ESXi.json' was

set to 'false'.

Template version processing for template '/root/vcenter/config/embedded_vCSA_on_ESXi.json' succeeded.

Version processing for all templates succeeded.

======= [SUCCEEDED] Successfully executed Task 'VersionProcessingTask: Executing Template Version Processing task' in TaskFlow 'template_validation' at 03:14:47 =======

============= [START] Start executing Task: To validate the template structure against the rules specified by a corresponding template schema. at 03:14:47 =============

Template structure validation for template '/root/vcenter/config/embedded_vCSA_on_ESXi.json' succeeded.

Structure validation for all templates succeeded.

===== [SUCCEEDED] Successfully executed Task 'StructureValidationTask: Executing Template Structure Validation task' in TaskFlow 'template_validation' at 03:14:47 =====

[START] Start executing Task: To create a dependency graph for the provided templates, with an edge pairing two templates that are dependent on each other. Such graph

relationships will affect whether certain templates can be deployed in parallel, or must be deployed sequentially. at 03:14:48

Dependency processing for all templates succeeded.

==== [SUCCEEDED] Successfully executed Task 'DependencyProcessingTask: Executing Template Dependency Processing task' in TaskFlow 'template_validation' at 03:14:48 ====

=================================== [START] Start executing Task: Validate that requirements are met in the source VCSA. at 03:14:48 ===================================

InstallRequirementCollector: Reached gathering requirement

================ [SUCCEEDED] Successfully executed Task 'SrcRequirementTask: Running SrcRequirementTask' in TaskFlow 'embedded_vCSA_on_ESXi' at 03:14:48 ================

=================================================== [START] Start executing Task: Perform precheck tasks. at 03:14:49 ===================================================

============================ [START] Start executing Task: Verify that the provided credentials for the target ESXi/VC are valid at 03:14:49 ============================

The certificate of server '1.2.3.4' will not be verified because you have provided either the '--no-ssl-certificate-verification' or '--no-esx-ssl-verify' command

parameter, which disables verification for all certificates. Remove this parameter from the command line if you want server certificates to be verified.

============================ [SUCCEEDED] Successfully executed Task 'Running precheck: TargetCredentials' in TaskFlow 'install' at 03:14:49 ============================

============================== [START] Start executing Task: Verify the target host type matches the one given in the template at 03:14:49 ==============================

============================== [SUCCEEDED] Successfully executed Task 'Running precheck: TargetHostType' in TaskFlow 'install' at 03:14:49 ==============================

========================== [START] Start executing Task: Verify that the target ESXi or VC version meets the minimum requirements at 03:14:49 ==========================

============================== [SUCCEEDED] Successfully executed Task 'Running precheck: TargetVersion' in TaskFlow 'install' at 03:14:49 ==============================

================================ [START] Start executing Task: Validate that the OVA image has the required OVA properties. at 03:14:50 ================================

============================== [SUCCEEDED] Successfully executed Task 'Running precheck: OVAProperties.' in TaskFlow 'install' at 03:14:50 ==============================

====================================== [START] Start executing Task: Validate the provided ip/fqdn is available to use at 03:14:50 ======================================

Pinging IPv4 address 5.6.7.8

5.6.7.8 seems not in use.

Pinging IPv4 address vcenter1.esxi1.lab

Pinging IPv4 address vcenter1.esxi1.lab

Pinging IPv4 address vcenter1.esxi1.lab

5.6.7.8 seems not in use.

=============================== [SUCCEEDED] Successfully executed Task 'Running precheck: IpFqdnInUse' in TaskFlow 'install' at 03:15:34 ===============================

===================== [SUCCEEDED] Successfully executed Task 'Running precheck: LinkingTargetHostnameValidation' in TaskFlow 'install' at 03:15:35 =====================

========================================= [START] Start executing Task: Validate the target ESXi management status at 03:15:35 =========================================

=========================== [SUCCEEDED] Successfully executed Task 'Running precheck: ESXManagementStatus' in TaskFlow 'install' at 03:15:35 ===========================

============================== [START] Start executing Task: Precheck CPU, memory and datastore size requirements for a host. at 03:15:35 ==============================

=============================== [SUCCEEDED] Successfully executed Task 'Running precheck: HostConfigs' in TaskFlow 'install' at 03:15:37 ===============================

=============================== [START] Start executing Task: Validate the provided target appliance name is available to use at 03:15:37 ===============================

============================== [SUCCEEDED] Successfully executed Task 'Running precheck: ApplianceName' in TaskFlow 'install' at 03:15:37 ==============================

============================================================== [START] Start executing Task:  at 03:15:37 ==============================================================

============================================== [SUCCEEDED] Successfully executed Task '' in TaskFlow 'install' at 03:15:37 ==============================================

====================== [START] Start executing Task: Check whether the datastore's free space accommodate the VCSA's deployment option at 03:15:38 ======================

============================ [SUCCEEDED] Successfully executed Task 'Running precheck: TargetDsFreespace' in TaskFlow 'install' at 03:15:38 ============================

=========================== [START] Start executing Task: Verify the provided SSO info is valid by connecting to the STS service. at 03:15:38 ===========================

============================== [SUCCEEDED] Successfully executed Task 'Running precheck: SSOCredentials' in TaskFlow 'install' at 03:15:38 ==============================

============================= [START] Start executing Task: Verify that the provided external SSO (PSC)/Embedded info is valid at 03:15:38 =============================

=============================== [SUCCEEDED] Successfully executed Task 'Running precheck: SSOExternal' in TaskFlow 'install' at 03:15:38 ===============================

======================= [SUCCEEDED] Successfully executed Task 'PrecheckTask: Running prechecks.' in TaskFlow 'embedded_vCSA_on_ESXi' at 03:15:38 =======================

=========================== [START] Start executing Task: Invoke OVF Tool to deploy VCSA for installation, upgrade, and migration at 03:15:39 ===========================

OVF Tool: Opening OVA source: /mnt/cdrom/vcsa-cli-installer/lin64/../../vcsa/VMware-vCenter-Server-Appliance-7.0.0.10100-15934039_OVF10.ova

OVF Tool: The manifest validates

OVF Tool: The provided certificate is in valid period

OVF Tool: Source is signed but could not verify certificate (possibly self-signed)

OVF Tool: Certificate information:

OVF Tool:   CertIssuer:/C=US/ST=California/L=Palo Alto/O=VMware, Inc.

OVF Tool:   CertSubject:/C=US/ST=California/L=Palo Alto/O=VMware, Inc.

OVF Tool:   -----BEGIN CERTIFICATE-----

OVF Tool:   ... <DELETED> ...

OVF Tool:

OVF Tool:   -----END CERTIFICATE-----

OVF Tool: Opening VI target: vi://root@1.2.3.4:443/

OVF Tool: Deploying to VI: vi://root@1.2.3.4:443/

OVF Tool: Disk progress: 99%

OVF Tool: Transfer Completed

OVF Tool: Powering on VM: Embedded-vCenter-Server-Appliance

OVF Tool: Task progress: 0%

OVF Tool: Task Completed

OVF Tool: Waiting for IP address...

OVF Tool: Received IP address: 5.6.7.8

OVF Tool: Completed successfully

====================== [SUCCEEDED] Successfully executed Task 'Deploying vCenter Server Appliance' in TaskFlow 'embedded_vCSA_on_ESXi' at 03:21:31 ======================

================================= [START] Start executing Task: Monitor the entire VCSA deployment using appliance REST API at 03:21:31 =================================

Checking if the target VCSA is powered on and ready for operations...

Successfully verified that the target VCSA 'Embedded-vCenter-Server-Appliance' has been powered on and is now ready for operations

Checking if the target VCSA appliance API is available for query

The certificate of server '5.6.7.8' will not be verified because you have provided either the '--no-ssl-certificate-verification' or '--no-esx-ssl-verify' command

parameter, which disables verification for all certificates. Remove this parameter from the command line if you want server certificates to be verified.

Where ,

1.2.3.4 - ESXi host

5.6.7.8 - vCenter IP

clientVM1 is on the another server , a RHEL VM created, which has access to the DNS server. Reverse mapping of DNS is also done on the DNS server.

[krisjinka]

I triggered the vCenter installer from another VM on another server. Like this:

Command:

./vcsa-deploy install /root/vcenter/config/embedded_vCSA_on_ESXi.json --accept-eula --no-esx-ssl-verify --no-ssl-certificate-verification --log-dir=/root/vcenter/log

Result: Failed with below logs on screen.

OVF Tool: Opening VI target: vi://root@10.251.105.18:443/

OVF Tool: Deploying to VI: vi://root@10.251.105.18:443/

OVF Tool: Disk progress: 99%

OVF Tool: Transfer Completed

OVF Tool: Powering on VM: Embedded-vCenter-Server-Appliance

OVF Tool: Task progress: 0%

OVF Tool: Task Completed

OVF Tool: Waiting for IP address...

OVF Tool: Received IP address: 10.251.105.113

OVF Tool: Completed successfully

====================== [SUCCEEDED] Successfully executed Task 'Deploying vCenter Server Appliance' in TaskFlow 'embedded_vCSA_on_ESXi' at 06:59:33 ======================

================================= [START] Start executing Task: Monitor the entire VCSA deployment using appliance REST API at 06:59:33 =================================

Checking if the target VCSA is powered on and ready for operations...

Successfully verified that the target VCSA 'Embedded-vCenter-Server-Appliance' has been powered on and is now ready for operations

Checking if the target VCSA appliance API is available for query

The certificate of server '10.251.105.113' will not be verified because you have provided either the '--no-ssl-certificate-verification' or '--no-esx-ssl-verify' command

parameter, which disables verification for all certificates. Remove this parameter from the command line if you want server certificates to be verified.

Error:     Problem Id: None     Component key: setnet     Detail:         Failed to set the time via NTP. Details: Failed to sync to NTP servers.. Code:

com.vmware.applmgmt.err_ntp_sync_failed         Could not set up time synchronization.     Resolution: Verify that provided ntp servers are valid.

============================================================ [FAILED] Task: MonitorDeploymentTask: Monitoring Deployment execution failed at 09:36:53

============================================================

=========================================================================================================================================================================

Error message: The appliance REST API was not yet available from the target VCSA 'Embedded-vCenter-Server-Appliance'because 'HTTPSConnectionPool(host='10.251.105.113',

port=5480): Max retries exceeded with url: /rest/vcenter/deployment (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fbd0e841ac8>:

Failed to establish a new connection: [Errno 110] Connection timed out',)). If the error message contains references about messages being expired, a login problem, or

its context indicates a timing issue, a possible reason could be system clock skew. Refer to error messages in 'taskflow.log' for more details. '. The VCSA might still

be starting up.

=============================================================================== 09:36:53 ===============================================================================

Result and Log File Information...

WorkFlow log directory: /root/vcenter/log/workflow_1587452127185

Questions:

1. how to check, what is wrong with NTP. Because, on the ESXi host, on the GUI, The NTP status is shown as 'sync' and running

Current date and time

Wednesday, April 22, 2020, 00:34:27 UTC

NTP service status

Running

NTP servers

1. 1. 10.251.101.234

2.

The appliance REST API was not yet available from the target VCSA 'Embedded-vCenter-Server-Appliance'because 'HTTPSConnectionPool(host='10.251.105.113',

port=5480): Max retries exceeded with url: /rest/vcenter/deployment (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fbd0e841ac8>:

Failed to establish a new connection: [Errno 110] Connection timed out',)). If the error message contains references about messages being expired, a login problem, or

its context indicates a timing issue, a possible reason could be system clock skew. Refer to error messages in 'taskflow.log' for more details. '. The VCSA might still

be starting up.

   . i have checked time is same on both the client (from where this installer is invoked) and ESXI host.

     . ESXi host is sync with NTP server

     . The client VM(10.251.108.132) is unable to reach the

The log file from workflow logs directory shows that:

vcsa-cli-installer.log

2020-04-21 07:10:03,049 - vCSACliInstallLogger - DEBUG - Retrieved all ip addresses from appliance Embedded-vCenter-Server-Appliance: ['10.251.105.113']

2020-04-21 07:10:03,067 - vCSACliInstallLogger - DEBUG - Querying REST endpoint '/rest/vcenter/deployment' on appliance '10.251.105.113' for deployment status

2020-04-21 07:10:03,067 - vCSACliInstallLogger - DEBUG - Proceed with certificate thumbprint check...

2020-04-21 07:10:03,068 - vCSACliInstallLogger - DEBUG - Requesting deployment status from target vCSA REST API endpoint 'https://10.251.105.113:5480/rest/vcenter/deployment'

I am unable to figure out the reason why this rest api is unreachable from my client VM

[pradeepjigalur]

Can you please attach vcsa-cli-installer.log file

Also by logging into the system please check whether appmgmt service is up and running /var/log/firstboot/*applmgmt*

[Venkatasureshba]

Hi,

what was the resolution for this issue?