Use the vCSA (vCenter Server Appliance) which is a separate ISO containing a deployment wizard that'll guide you through the process. Don't use Windows-based vCenter any more as this is the last release, lacks features, has poorer performance, and other reasons.
Certain CA's are not trusted by some versions of Windows. Gold option would be to go with vCenter appliance. If you have to make it work on the Windows, follow the below steps to disable the Microsoft policy “Turn off Automatic Root Certificates Update”. Disabling this setting will enable the installer to download the required certificates and verify the MSI package.
1) Click Start, and then click Run.
2) Type gpedit.msc, and then click OK.
3) Double-click Administrative Templates, double-click System, double-click Internet Communication Management, and then click Internet Communication settings.
4) Double-click 'Turn off Automatic Root Certificates Update', click disabled, and then click OK.
5) Close the Local Group Policy Editor.
6) Just for the sake of it, reboot the VM once
7) Attempt the installation.
Please consider marking this answer as "correct" or "helpful" if you think your questions have been answered.