Failed to rotate SSH keys Failed to rollback rotation of SSH keys

[drkytoo]

Cloud builder always fails with this error, I have validate thumbprints on:

2021-11-13T15:39:20.994+0000 [bringup,6a8653ce52fde667,2370] DEBUG [c.v.e.s.c.u.SshKeyManagementService,pool-3-thread-12] Updated ssh host key successfully on host daedalus.domain.name with doRollback flag:true.
2021-11-13T15:39:20.994+0000 [bringup,6a8653ce52fde667,2370] DEBUG [c.v.e.s.c.util.RetriableCallable,pool-3-thread-12] Starting retriable operation 'Try to get the rotated SSH Key daedalus.domain.name' with 10 retries.
2021-11-13T15:39:20.997+0000 [bringup,6a8653ce52fde667,2370] DEBUG [c.v.e.s.c.util.LocalProcessService,pool-3-thread-12] Executing the Local command: ssh -p 22 -o HostKeyAlgorithms=ssh-rsa daedalus.domain.name -o StrictHostKeyChecking=no -o UserKnownHostsFile=/opt/vmware/bringup/tmp/test6758787495850348897.txt -o PasswordAuthentication=no 2>/dev/null
2021-11-13T15:39:21.065+0000 [bringup,6a8653ce52fde667,51d6] DEBUG [c.v.e.s.v.c.RotateMachineSshKeys,pool-4-thread-19] Unable to negotiate with 172.16.9.16 port 22: no matching host key type found. Their offer: rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256

[drkytoo]

still not fixed yet

[viquarhcimca]

in the cloud build try to find / -iname known_hosts , if its vcf 4.2 it will be below path 

/opt/vmware/bringup/.ssh/known_hosts

run a keyscan , There u will find  keys for each of the four management nodes and the VxRail Manager( if its VCFonVxRail )  but not the 172.16.9.16 so it looks like the key exchange did not happen for the 172.16.9.16

So add the 172.16.9.16  ssh keys to the known_hosts files:

ssh-keyscan 172.16.9.16(FQDN) >> known_hosts

then it should bringup successfully. 

[drkytoo]

I ended up having to modify the encryption, it turned out the cloud builder was trying to use encryption that ESXi didn't know about.