The Reset all Certificates option in the certificate-manager stops with an access denied error (0x16c9a0f6)

[systraysol]

Hi,

I recently rebooted my vcsa appliance and after waiting a while for all services to start up my VMware vSphere Web Client started showing the following error message:

503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x000055fcde997a60] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)

I took a look at rhttpproxy.log file and it showed the following output:

2019-03-11T14:07:23.814+01:00 warning rhttpproxy[11879] [Originator@6876 sub=Default] Failed to connect socket; <io_obj p:0x00007f98b0001fa0, h:19, <TCP '127.0.0.1 : 36346'>, <TCP '127.0.0.1 : 8089'>>, e: 111(Connection refused)

2019-03-11T14:07:23.814+01:00 warning rhttpproxy[11879] [Originator@6876 sub=Proxy Req 00601 Tunnel] Error connecting tunnel on TCP socket: (null): Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.

2019-03-11T14:07:29.139+01:00 warning rhttpproxy[11871] [Originator@6876 sub=Default] Failed to connect socket; <io_obj p:0x00007f98a80172d8, h:19, <UNIX ''>, <UNIX '/var/run/vmware/vpxd-webserver-pipe'>>, e: 111(Connection refused)

2019-03-11T14:07:29.139+01:00 warning rhttpproxy[11871] [Originator@6876 sub=Proxy Req 00602] Connection to named pipe /var/run/vmware/vpxd-webserver-pipe failed with error N7Vmacore15SystemExceptionE(Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.)

--> [context]zKq7AVECAQAAAPJSnAAOcmh0dHBwcm94eQAAlLAqbGlidm1hY29yZS5zbwAAIDcbAG67GAD35yIAPIwlAISUJQAasSUA4bolAGlRIwChHiMAaiEjAB0IKwHUcwBsaWJwdGhyZWFkLnNvLjAAAp2MDmxpYmMuc28uNgA=[/context].

Which according to VMware Knowledge Base means that there is a certificate error. When I try resetting all certificates using /usr/lib/vmware-vmca/bin/certificate-manager, the following error message appears:

Status : 0% Completed [Reset Root Cert...]                 

Using config file : /var/tmp/vmware/root.cfg

Error: 382312694, VMCAAddRootCertificatePrivate() failedError: 382312694, Failed to add root certificate

Status : Failed

Error Code : 382312694

Error Message : Access denied, reason = rpc_s_auth_method (0x16c9a0f6).

Status : 0% Completed [Reset operation failed]

please see /var/log/vmware/vmcad/certificate-manager.log for more information.

Does anyone have any advice on how to solve this issue?

[GayathriS]

This article has similar error snippet , check out if this can help you :

https://kb.vmware.com/s/article/2133028

regards

Gayathri

[systraysol]

Already tried that. No luck.

[maj54]

Hello,

I have the same issue here:

2019-05-21T06:02:31.945Z INFO certificate-manager Command executed successfully
2019-05-21T06:02:31.945Z INFO certificate-manager Certificate backup created successfully
2019-05-21T06:02:31.953Z INFO certificate-manager Running command : ['/usr/lib/vmware-vmca/bin/certool', '--getrootca', '--server', 'localhost']
2019-05-21T06:02:32.128Z INFO certificate-manager Output :
Status : Failed
Error Code : 382312694
Error Message : Access denied, reason = rpc_s_auth_method (0x16c9a0f6).
2019-05-21T06:02:32.128Z INFO certificate-manager Regenerating Root Cert using VMCA...
2019-05-21T06:02:32.129Z INFO certificate-manager Running command :- ['/usr/lib/vmware-vmca/bin/certool', '--selfca', '--config', '/var/tmp/vmware/root.cfg', '--server', 'localhost']
2019-05-21T06:02:32.724Z INFO certificate-manager Command output :-
 Using config file : /var/tmp/vmware/root.cfg
Error: 382312694, VMCAAddRootCertificatePrivate() failedError: 382312694, Failed to add root certificate
Status : Failed
Error Code : 382312694
Error Message : Access denied, reason = rpc_s_auth_method (0x16c9a0f6).
2019-05-21T06:02:32.724Z ERROR certificate-manager Using config file : /var/tmp/vmware/root.cfg
Error: 382312694, VMCAAddRootCertificatePrivate() failedError: 382312694, Failed to add root certificate
Status : Failed
Error Code : 382312694
Error Message : Access denied, reason = rpc_s_auth_method (0x16c9a0f6).

Did you find a solution?

thanks

[Sri_KSK]

Hi All, Any solution for this issue ??

[Sri_KSK]

Any solution for this issue ??

[mmccaffe]

The access denied is because your server's password has expired while the disk was full and did not get reset.

you need to follow: VMware Knowledge Base  article 2147280.

Run this command to open the vdcadmintool:

/usr/lib/vmware-vmdir/bin/vdcadmintool

select 3

Enter the vCenter server FQDN@SSO DOmain.

example vcenter6.mydomain.com@vsphere.local

It will generate a random password for that account, but you may need to run more than once if it puts invalid characters in the password.

Note: The tool does not filter out invalid characters from the generated password such as:

& (ampersand)

; (semicolon)

" (double quotation mark)

' (single quotation mark)

^ (circumflex)

\ (backslash)

% (percentage)

Copy that password because you have to past it in the registry.

    Run these commands to update the password:

    /opt/likewise/bin/lwregshell

    cd HKEY_THIS_MACHINE\services\vmdir\

    set_value dcAccountPassword "new password"

    quit

Then reboot the VCSA.

You may need the EAM service also fail because of the full disk. this KB walks through fixing that: VMware Knowledge Base

-Marty-