vcsa 6.5 trying to automate the backup - authentication failure

[marv0]

Hello

I tried to make the Backup-VCSAToFile work in my development environment.

vSphere 6.5 – Automate VCSA Backup » Brian Graf's Virtualization Blog

I'm stuck here:

A server error occurred: 'com.vmware.vapi.std.errors.unauthorized': Unable to authorize user (Server error id:

'vapi.security.authorization.invalid'). Check $Error[0].Exception.ServerError for more details.

In Zeile:1 Zeichen:1

What did I do?

Windows 7 VM, installed VMware Powershell 6.5

Started ISE

loaded modules

Import-Module VMware.VimAutomation.Core

Import-Module VMware.VimAutomation.Vds

Import-Module VMware.VimAutomation.Cloud

Import-Module VMware.VimAutomation.PCloud

Import-Module VMware.VimAutomation.Cis.Core

Import-Module VMware.VimAutomation.Storage

Import-Module VMware.VimAutomation.HorizonView

Import-Module VMware.VimAutomation.HA

Import-Module VMware.VimAutomation.vROps

Import-Module VMware.VumAutomation

Import-Module VMware.DeployAutomation

Import-Module VMware.ImageBuilder

Import-Module VMware.VimAutomation.License

loaded the script itself as module (necessary?)

connected to vcsa with connect-viserver

Used given example in the script with my parameters.

Then I get a pop up

(Connection to CisServer)

I don't know what that is.:smileyconfused:

If I use my credentials for the vCenter connection - AD authentication, I get that error

Backup-VCSAToFile : A server error occurred: 'com.vmware.vapi.std.errors.unauthorized': Unable to authorize user (Server error id:

'vapi.security.authorization.invalid'). Check $Error[0].Exception.ServerError for more details.

In Zeile:1 Zeichen:1

Any hints?

[ivanerben]

This looks like Windows world solution -- restart to solve problem.

[Sbenito]

Same problem here and also there is no progress on this SR.

Could you resolve this? Could you tell me if something worked to you MarcelSwartjes?

A lot of thanks and sorry for the disturbances

[MatthewPinkston]

Sbenito, did you read through all the posts in this thread? What have you tried? As far as I can tell, Marcel's issue was different than what most people in this thread had. Typically, you're executing the backup job from the vcsa server in a cronjob anyway, not through powershell. Although it's interesting, if what he says is correct, calling the API through that command directly instead of using curl, may be more stable in general. Although, since both methods are using the same API call, I suspect something else was at play.

Things you can try from ssh into the vcsa:

Enable shell:

shell.set --enabled true

shell

Make sure there is available disk space:

df -h

Restart services:

service-control --stop applmgmt

service-control --start applmgmt

service-control --stop vmware-vapi-endpoint

service-control --start vmware-vapi-endpoint

[Sbenito]

First, thank you for your time,
Yes, I have read all posts.My problem is with powershell (cis-services), curl and APIexplorer... The result is same: "vapi.security.authorization.invalid"
I restarted services and servers, but it doesnt work for me. Also, I have free space...
I finded this:

2018-03-13T13:00:57.630510+01:00 XXXXXXX cli: vmware.appliance.vapi.auth Could not parse HOK Token
Traceback (most recent call last):
  File "/usr/lib/applmgmt/vapi/py/vmware/appliance/vapi/auth.py", line 183, in authenticate
    token.validate()
  File "/usr/lib/applmgmt/lib/extensions/py/vmware/appliance/extensions/authentication/authentication_sso.py", line 530, in validate
    reference = self.validate_signature(signing_chain)
  File "/usr/lib/applmgmt/lib/extensions/py/vmware/appliance/extensions/authentication/authentication_sso.py", line 763, in validate_signature
    'Invalid SAML token: <Reference> element has '
AuthenticationError: Invalid SAML token: <Reference> element has invalid digest.
2018-03-13T13:00:57.633099+01:00 XXXXXXX cli: twisted "127.0.0.1" - - [13/Mar/2018:12:00:57 +0000] "POST /api HTTP/1.1" 200 339 "-" "vAPI http client"

This occurs since I upgraded window server vmware 5.5 to vmware appilance 6.5.
Thank you very much.

[MarcelSwartjes]

I'v received a working solution from VMware. In the file authentication_sso.py on the vcsa you find these 2 lines:

digest_value = self.xpath(

'//ds:DigestValue', reference, expect=1)[0].text

Replace them with these 3 lines:

digest_value = str(self.xpath(

'//ds:DigestValue', reference, expect=1)[0].text).replace(

'\r', '').replace('\n', '')

Then stop and start the applmgmt service:

service-control --stop applmgmt

service-control --start applmgmt

Now it should work.

Don't know yet when the final solution is implemented in a patch for vcsa.

[Sbenito]

Great!!! MarcelSwartjes . A lot of thanks for your help. This work perfectly.

Thank you very much.

[srodenburg]

Works for me too. My script runs fine on VCSA "6.5 U1e" but on a new "6.5 U1f" installation it failed. This fixed it. Thanks.

[multihawk]

I've looked at both VCSA and PSC but can't find these lines of text in the file listed.  The only file I could find with that name is at /usr/lib/applmgmt/lib/extensions/py/vmware/appliance/extensions/authentication/authentication_sso.py   Is that the correct location of the file?

[Sbenito]

Hi,

Yes, the file is "/usr/lib/applmgmt/lib/extensions/py/vmware/appliance/extensions/authentication/authentication_sso.py"

In my case, It is in the line: 755 and next... In the version 6.5u1g still there is this error

    755         # verify digest

    756         digest_method = self.xpath(

    757             '//ds:DigestMethod', reference, expect=1)[0].get('Algorithm')

    758         #digest_value = self.xpath(

    759         #   '//ds:DigestValue', reference, expect=1)[0].text

    760         digest_value = str(self.xpath(

    761             '//ds:DigestValue', reference, expect=1)[0].text).replace('\r', '').replace('\n', '')

Regards

[srodenburg]

This is a bug, starting with vCenter 6.5 U1f  and  U1g

This is the fix:

Search for the file "authentication_sso.py" using the command "find / -name "authentication_sso.py"

There is only one file with this name. It's buried deep in the filesystem.

Then, in that file called "authentication_sso.py", search for this section:

digest_value = self.xpath(

'//ds:DigestValue', reference, expect=1)[0].text

and replace them with these 3:

digest_value = str(self.xpath(

'//ds:DigestValue', reference, expect=1)[0].text).replace(

'\r', '').replace('\n', '')

Then restart the "applmgmt" service:

service-control --stop applmgmt

service-control --start applmgmt

No need to reboot the appliance. If you run the script now, it should work just fine.

[multihawk]

I'm on 6.5 U1c so that must be the problem.  I'll patch and see if it shows up.  Moving my lab stuff to 6.7 so I might not get to this for a few days.

[PGinhoux]

Hi,

I had the same issue as reported and the action to stop/start the applmgmt has fixed the problem for me:

service-control --stop applmgmt
service-control --start applmgmt

Thanks to Johny and Matt for the help.

Now by looking at the looking at the vmware-vapi-endpoint service, I see it 'failed' and a stop/start doesn't fix it.
And in the log below, I see some errors :

vmware-vapi-endpoint[41452]: Warning: your password will expire in 3 days
Unable to write to the default log file: wrapper.log (Permission denied)

root@in52invc01 [ /storage/log ]# systemctl status vmware-vapi-endpoint.service
● vmware-vapi-endpoint.service - LSB: VMware vAPI Endpoint
   Loaded: loaded (/etc/rc.d/init.d/vmware-vapi-endpoint; bad; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2018-07-05 17:13:34 CEST; 26min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 41452 ExecStart=/etc/rc.d/init.d/vmware-vapi-endpoint start (code=exited, status=1/FAILURE)

Jul 05 17:13:27 in52invc01.vdrmvt.local vapi-endpoint[41600]: Unable to write to the configured log file: ${vapi_log_dir}/wrapper....tory)
                                                                Falling back to the default file in the current working directory:...r.log
Jul 05 17:13:27 in52invc01.vdrmvt.local vapi-endpoint[41600]: Unable to write to the default log file: wrapper.log (Permission denied)
                                                                Disabling log file.
Jul 05 17:13:34 in52invc01.vdrmvt.local vmware-vapi-endpoint[41452]: Waiting for vAPI Endpoint.........
Jul 05 17:13:34 in52invc01.vdrmvt.local vmware-vapi-endpoint[41452]: WARNING: vAPI Endpoint may have failed to start.
Jul 05 17:13:34 in52invc01.vdrmvt.local su[41524]: pam_unix(su:session): session closed for user vapiEndpoint
Jul 05 17:13:34 in52invc01.vdrmvt.local vmware-vapi-endpoint[41452]: Warning: your password will expire in 3 days
Jul 05 17:13:34 in52invc01.vdrmvt.local systemd[1]: vmware-vapi-endpoint.service: Control process exited, code=exited status=1
Jul 05 17:13:34 in52invc01.vdrmvt.local systemd[1]: Failed to start LSB: VMware vAPI Endpoint.
Jul 05 17:13:34 in52invc01.vdrmvt.local systemd[1]: vmware-vapi-endpoint.service: Unit entered failed state.
Jul 05 17:13:34 in52invc01.vdrmvt.local systemd[1]: vmware-vapi-endpoint.service: Failed with result 'exit-code'.
Hint: Some lines were ellipsized, use -l to show in full.
root@in52invc01 [ /storage/log ]#

I don't really understand these messages,

[Update] I have found of which password it is and I changed it to "Never expires" thru the Vami interface.

Any ideas are welcome.

[ManuelM_IT]

I was having the same issue and restarting the service was the fix for me. Thanks for the help.

[hespino2]

Estimados,

Junto con saludarlos, indicarles que me pasaba exactmente lo mismo, y luego de varios dias, reinicie el Host donde tengo instalado el VCSA 6.5 LO CUAL SOLUCIONO la situacion logrando realizar los BKP necesarioss.

Las validaciones las realice desde las API Explorer (), POSTMAN y mediante Codigo BASH funcionando en todas estas modalidades, identificando en estas pruebas que mediante POSTMAN, la ejecucion fue mas rapida. Adjunto print .