vIDM 3.3.4 - Unable to find oAuth2 client for end-user catalog for root tenant

[aszabomhm]

I was trying to enable multitenancy in Lifecycle Manager 8.4 and got an error: LCMVIDM72218 Failed to enable multi-tenancy on the vIDM instance.

Looking into vIDM horizon.log sort of explains it:

2021-05-07T15:49:31,917 ERROR (Thread-5) [VIDM01;6e26f69f-f783-4a91-b424-a6f1600b09fe;172.16.11.253;] com.vmware.horizon.tenant.EndUserCatalogTenantService - Unable to find oAuth2 client for end-user catalog for root tenant : oauth2.client.not.found
com.vmware.horizon.common.api.exception.NotFoundException: oauth2.client.not.found

2021-05-07T15:49:31,919 ERROR (Thread-5) [VIDM01;6e26f69f-f783-4a91-b424-a6f1600b09fe;172.16.11.253;] com.vmware.horizon.tenant.EndUserCatalogTenantService - Failed to get tenant VIDM01 information for end-user catalog @ https://****:443/catalog-portal/services/config/tenants: end.user.catalog.oauth2.client.not.found
com.vmware.horizon.tenant.exception.EndUserCatalogException: end.user.catalog.oauth2.client.not.found

2021-05-07T15:49:31,920 ERROR (Thread-5) [VIDM01;6e26f69f-f783-4a91-b424-a6f1600b09fe;172.16.11.253;] com.vmware.horizon.tenant.TenantRenameServiceImpl - Tenant rename to vidm-master failed for green box.
2021-05-07T15:49:31,920 ERROR (Thread-5) [VIDM01;6e26f69f-f783-4a91-b424-a6f1600b09fe;172.16.11.253;] com.vmware.horizon.restapi.manager.exception.mapper.AbstractExceptionMapper - Exception while handling jersey request.
javax.ws.rs.WebApplicationException: HTTP 500 Internal Server Error

I guess this is also the reason why the new Catalog UI doesn't load, just drops the usual "contact your IT administrator" sign.

This is a lab setup so I could rebuild it anytime, but I really want to understand the root cause, in case it happens to any of my customers. FYI it's a 3-node cluster, nodes are vidm[01..03], LB is called simply "vidm" (provisioned via NSX), and I'm trying to name the master tenant "vidm-master". Certs and DNS are OK, double-triple-checked.

Any ideas are more than welcome. Thanks!

[aszabomhm · Accepted solution]

Solved it myself: the client starting with "usercatalog" was missing from the oauth2 clients.I guess someone from my team had accidentally deleted it.

Kinda tricky to recreate it, but I figured it out using Postman (REST API).

[Prachimhptr]

Could you please let me know the API call and other fields that you used, as I am going through the same issue?

[aszabomhm]

Luckily I could find it in Postman history, so here it is... I assume you have an administrative credential which you can use to create oauth2 clients through API. I used token-based authorization for the API calls.

You're going to need a lowercase UUID in the name of the usercatalog client. It would be best to have the original ID of the client but I guess it could work with any generated one. I used a v4 generator at https://www.uuidgenerator.net/

Basically all I did a was a POST to https://vidm.fqdn.corp/SAAS/jersey/manager/api/oauth2client using this body:

{

    "clientId": "usercatalog-VIDM01-2bdf356d-593b-4c93-a747-26f2c4f0e3e9",

    "secret": "generate-a-unique-secret-for-this-client!",

    "scope": "user admin",

    "authGrantTypes": "client_credentials authorization_code",

    "redirectUri": "https://vidm.fqdn.corp:443/catalog-portal/ui/oauth/verify",

    "tokenType": "Bearer",

    "tokenLength": 23,

    "accessTokenTTL": 360,

    "refreshTokenTTL": 43200,

    "refreshTokenIdleTTL": 0,

    "rememberAs": "EndUserCatalog",

    "resourceUuid": null,

    "displayUserGrant": false,

    "internalSystemClient": false,

    "activationToken": null,

    "strData": null,

    "inheritanceAllowed": false,

    "returnFailureResponse": false,

    "_links": {

        "self": {

            "href": "/SAAS/jersey/manager/api/oauth2clients/usercatalog-VIDM01-2bdf356d-593b-4c93-a747-26f2c4f0e3e9"

        }

    }

}

Please make sure to write the same FQDN to both the API call and the redirectUri field of the body. I also provided Postman with the "Content-Type: application/vnd.vmware.horizon.manager.oauth2client+json" header and a unique string for the secret field. Also, please change "VIDM01" in this example to the tenant name where you experience this issue. Finally I did a full reboot of vIDM to restart all services etc. which could be dependent on this client.

Let me know how it goes!