Hello
The pre-flight check runs fine, but when I actually try to deploy the SDDC, it fails quite early in the process whilst adding the Admin role to the users.
Failed to assign role Admin to ESXi user svc-vcf-sfo01-m01-esx02
When I look at the ESXi hosts, the first ESXi host has four (4) new users created and the other three hosts do not have any!
Thanks Sean
2021-01-08T19:31:54.569+0000 [bringup,2fdef55c2c687c40,5f13] INFO [c.v.e.s.v.c.AssignRoleToEsxiUsers,pool-3-thread-17] Assigning Role Admin to local user svc-vcf-sfo0 1-m01-esx01 in ESXi host sfo01-m01-esx01.cc-demo.local
2021-01-08T19:31:54.579+0000 [bringup,2fdef55c2c687c40,5f13] WARN [c.v.v.s.h.ssl.TrustAllTrustManager,pool-3-thread-17] Trusting server
2021-01-08T19:31:54.630+0000 [bringup,2fdef55c2c687c40,5f13] INFO [c.v.e.s.c.c.v.vsphere.VsphereClient,pool-3-thread-17] Successfully logged in to https://sfo01-m01-e sx01.cc-demo.local/sdk
2021-01-08T19:31:54.630+0000 [bringup,2fdef55c2c687c40,5f13] INFO [c.v.e.s.c.c.v.vsphere.VcManagerBase,pool-3-thread-17] Retrieving ManagedObjectReference for host sf o01-m01-esx01.cc-demo.local
2021-01-08T19:31:54.630+0000 [bringup,2fdef55c2c687c40,5f13] DEBUG [c.v.e.s.c.c.v.vsphere.VcManagerBase,pool-3-thread-17] Finding hosts with fqdn sfo01-m01-esx01.cc-de mo.local
2021-01-08T19:31:54.639+0000 [bringup,2fdef55c2c687c40,5f13] DEBUG [c.v.e.s.c.c.v.vsphere.VcManagerBase,pool-3-thread-17] Assigning role Admin to user svc-vcf-sfo01-m0 1-esx01 on entity {"_type":"HostSystem","_value":"ha-host"}.
2021-01-08T19:31:54.661+0000 [bringup,2fdef55c2c687c40,5f13] DEBUG [c.v.e.s.c.c.v.vsphere.VcManagerBase,pool-3-thread-17] Found role ID -1 for role with name Admin
2021-01-08T19:31:54.670+0000 [bringup,2fdef55c2c687c40,5f13] DEBUG [c.v.e.s.c.c.v.vsphere.VcManagerBase,pool-3-thread-17] Successfully assigned role Admin to user svc- vcf-sfo01-m01-esx01
2021-01-08T19:31:54.670+0000 [bringup,2fdef55c2c687c40,5f13] INFO [c.v.e.s.v.c.AssignRoleToEsxiUsers,pool-3-thread-17] Successfully assigned role Admin to user svc-vc f-sfo01-m01-esx01.
2021-01-08T19:31:54.671+0000 [bringup,2fdef55c2c687c40,5f13] INFO [c.v.e.s.v.c.AssignRoleToEsxiUsers,pool-3-thread-17] Assigning Role Admin to local user svc-vcf-sfo0 1-m01-esx02 in ESXi host sfo01-m01-esx02.cc-demo.local
2021-01-08T19:31:54.682+0000 [bringup,2fdef55c2c687c40,5f13] WARN [c.v.v.s.h.ssl.TrustAllTrustManager,pool-3-thread-17] Trusting server
2021-01-08T19:31:54.730+0000 [bringup,2fdef55c2c687c40,5f13] INFO [c.v.e.s.c.c.v.vsphere.VsphereClient,pool-3-thread-17] Successfully logged in to https://sfo01-m01-e sx02.cc-demo.local/sdk
2021-01-08T19:31:54.731+0000 [bringup,2fdef55c2c687c40,5f13] INFO [c.v.e.s.c.c.v.vsphere.VcManagerBase,pool-3-thread-17] Retrieving ManagedObjectReference for host sf o01-m01-esx02.cc-demo.local
2021-01-08T19:31:54.731+0000 [bringup,2fdef55c2c687c40,5f13] DEBUG [c.v.e.s.c.c.v.vsphere.VcManagerBase,pool-3-thread-17] Finding hosts with fqdn sfo01-m01-esx02.cc-de mo.local
2021-01-08T19:31:54.741+0000 [bringup,2fdef55c2c687c40,5f13] ERROR [c.v.e.s.o.model.error.ErrorFactory,pool-3-thread-17] [C74G4I] FAILED_TO_ASSIGN_ROLE_TO_ESXI_USER Fa iled to assign role Admin to ESXi user svc-vcf-sfo01-m01-esx02
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to assign role Admin to ESXi user svc-vcf-sfo01-m01-esx02
other relevant info from the debug log...
2021-01-08T19:46:20.129+0000 [bringup,af5f4af322ab0767,cba1] DEBUG [c.v.e.s.o.c.ProcessingTaskSubscriber,pool-3-thread-3] Collected the following errors for task with name AssignRoleToEsxiUsers and ID 7f000001-76dd-1fac-8176-e21d287a041a: [ExecutionError [errorCode=null, errorResponse=LocalizableErrorResponse(messageBundle=com.vmware.evo.sddc.vsphere.contract.messages)]]
2021-01-08T19:46:20.225+0000 [bringup,af5f4af322ab0767,4381] DEBUG [c.v.e.s.o.c.ProcessingTaskSubscriber,pool-3-thread-10] Invoking task AssignRoleToEsxiUsers.UNDO Description: Assign role to ESXi user, Plugin: VsphereContractPlugin, ParamBuilder null, Input map: {esxiUserDetails=InitialEnvironmentSetup____2__esxiUserDetails}, Id: 7f000001-76dd-1fac-8176-e21d287a0418 ...
2021-01-08T19:46:20.236+0000 [bringup,af5f4af322ab0767,4381] DEBUG [c.v.e.s.o.c.c.ContractParamBuilder,pool-3-thread-10] Contract task Assign role to ESXi user input: {"esxiUserDetails":[{"esx":{"address":"sfo01-m01-esx01.cc-demo.local","username":"root","password":"*****"},"user":{"username":"svc-vcf-sfo01-m01-esx01","password":"*****"},"role":"Admin"},{"esx":{"address":"sfo01-m01-esx02.cc-demo.local","username":"root","password":"*****"},"user":{"username":"svc-vcf-sfo01-m01-esx02","password":"*****"},"role":"Admin"},{"esx":{"address":"sfo01-m01-esx03.cc-demo.local","username":"root","password":"*****"},"user":{"username":"svc-vcf-sfo01-m01-esx03","password":"*****"},"role":"Admin"},{"esx":{"address":"sfo01-m01-esx04.cc-demo.local","username":"root","password":"*****"},"user":{"username":"svc-vcf-sfo01-m01-esx04","password":"*****"},"role":"Admin"}]}
2021-01-08T19:46:20.237+0000 [bringup,af5f4af322ab0767,4381] DEBUG [c.v.e.s.o.c.ProcessingTaskSubscriber,pool-3-thread-10] Collected the following errors for task with name AssignRoleToEsxiUsers and ID 7f000001-76dd-1fac-8176-e21d287a041a: [ExecutionError [errorCode=null, errorResponse=LocalizableErrorResponse(messageBundle=com.vmware.evo.sddc.vsphere.contract.messages)]]
2021-01-08T19:46:20.357+0000 [bringup,af5f4af322ab0767,ad3c] WARN [c.v.e.s.o.c.ProcessingOrchestratorImpl,pool-3-thread-10] Processing State completed with failure
2021-01-08T19:46:21.235+0000 [bringup,af5f4af322ab0767,0024] INFO [c.v.e.s.o.core.OrchestratorImpl,pool-3-thread-9] End of Orchestration with FAILURE for Execution ID 0f1cfe06-2b2d-4324-82fe-151aab34eaf4
2021-01-08T19:46:21.235+0000 [bringup,af5f4af322ab0767,0024] DEBUG [c.v.e.s.o.d.i.OrchestratorDataImpl,pool-3-thread-9] Saving executionContext 0f1cfe06-2b2d-4324-82fe-151aab34eaf4 to DB
2021-01-08T19:46:27.283+0000 [bringup,af5f4af322ab0767,c77b] INFO [c.v.e.s.o.c.s.OrchestratorSubscriber,pool-3-thread-8] Ignoring unknown OrchestratorMessage {"executionId":"0f1cfe06-2b2d-4324-82fe-151aab34eaf4"}