All Tanzu Mission Control & VMware Cloud Director Discussion Board posts

Re: TMC Managed K8s Cluster - CLI authentication and Access Roles

jeffmace — Tue, 26 Sep 2023 17:35:33 GMT

We've been looking into this and can confirm you will be able to use roles other than 'tmc:admin' or 'tmc:member' to give access to specific resources. I believe this scenario would've worked if you had added the 'Organization Administrator' group to the 'organization.credential.view' role binding or some other 'organization.*' role.

Please try this after we GA a release with support for CSE 4.1 and let us know if you run into issues.

Re: TMC Managed K8s Cluster - CLI authentication and Access Roles

jeffmace — Wed, 13 Sep 2023 12:39:04 GMT

Yes, that makes sense. Thank you.

Re: TMC Managed K8s Cluster - CLI authentication and Access Roles

returntrip — Wed, 13 Sep 2023 12:36:51 GMT

My TMC Local is not working as I am waiting for a newer version compatible with CSE 4.1. But I was trying to use Acces Roles to manage/limit K8s API access (e.g: limit certain users to certain namespaces)

What I noticed was that you need either tmc-admin` or `tmc-member` roles to log onto TMC CLI (the command line interface for TMC), which allows you to access the k8s API via kubectl. Having tmc-admin` or `tmc-member` roles automatically gives full (admin) access to TMC managed K8s clusters and I am therefore unable to limit certain users or groups (i.e.: useer `johndoe` should only be able to list namesapces fro k8s cluster xyz).

I hope this makes sense. If not, lets wait for a new version of TMC that supports CSE 4.1. Will reinstall and can get into a meeting.

Re: TMC Managed K8s Cluster - CLI authentication and Access Roles

jeffmace — Wed, 13 Sep 2023 12:25:03 GMT

We are still looking into this but I want to make sure I understand what you would like to achieve.

Are you trying to use the "Cloud Administrator" role to grant access to the TMC-SM API/GUI so they can define policies/packages/etc in TMC-SM?

Re: Unable remove instance - CSE not found

returntrip — Tue, 05 Sep 2023 13:18:14 GMT

Is there a newer version of TMC local that is compatible with CSE 4.1?

I tried to install it but it is complaining that no CSE server is available

```

root@PhotonOS-001 [ ~ ]# /mnt/cdrom/linux.run create instance --name $TMC_SM_INSTANCE_NAME --host $VCD_HOSTNAME --username $VCD_USERNAME --certificate-file /tmp/vcd.pem --encryption-key ${TMC_SM_ENCRYPTION_KEY} --input-kube-cluster-name=${TMC_SM_KUBE_CLUSTER_NAME} --input-cert-provider=cluster-issuer --input-cert-cluster-issuer-name=selfsigned-ca-clusterissuer --input-dns-zone=${TMC_SM_DNS_ZONE} --input-contour-envoy-load-balancer-ip=${TMC_SM_LOAD_BALANCER_IP} --input-harbor-url=${TMC_SM_HARBOR_URL} --input-harbor-username=${TMC_SM_HARBOR_USERNAME} --accept
INFO [0019] Creating Solution instance entity instance=vmware.vcd-tmc-0.1.0-21897297-tmc
INFO [0019] Triggering action action=hook event=PreCreate
INFO [0020] Run EventPreCreate Hook action=hook event=PreCreate
INFO [0020] Run EventPreCreate Hook successfully action=hook event=PreCreate
INFO [0021] Creating element name=rde
INFO [0021] Creating element name=tmc-admin-global-role
INFO [0022] Creating element name=tmc-member-global-role
INFO [0023] Creating element name=rights-bundle
INFO [0023] Triggering action action=hook event=PostCreate
INFO [0024] Run EventPostCreate Hook action=hook event=PostCreate
INFO [0024] Copy the rights from global roles [Kubernetes Cluster Author] to the global role [tmc:member] action=hook event=PostCreate
INFO [0025] Update rights of global role tmc:member action=hook event=PostCreate
INFO [0025] Copy the rights from global roles [Organization Administrator Kubernetes Cluster Author] to the global role [tmc:admin] action=hook event=PostCreate
INFO [0025] Update rights of global role tmc:admin action=hook event=PostCreate
INFO [0025] Get Solution Org action=hook event=PostCreate
INFO [0025] Solution Org: CSE action=hook event=PostCreate
INFO [0025] Search CSE4 Cluster action=hook event=PostCreate
ERROR [0025] Failed to find any cse cluster in org CSE action=hook event=PostCreate
ERROR [0026] Failed to create instance 'tmc' name=tmc
ERROR [0026] Failed to find any cse cluster in org CSE: exit status 6: failed to execute trigger hook errorCode=5012110011142353

```

Re: Unable remove instance - CSE not found

jeffmace — Wed, 30 Aug 2023 20:46:10 GMT

The TMC-SM for VCD tech preview only has support for CSE 4.0.3. This is the cause of the initial error you had.

The tech preview utilizes an unreleased build of the UI which allows you to inject trusted certificates into the cluster. CSE 4.1 is the first release which allows you to specify certificates to be trusted by the bootstrap VM or cluster. As you've identified, these certificates are now specified at the provider level. This behavior is closer to what the experience will be like when TMC-SM for VCD is released.

Re: TMC Managed K8s Cluster - CLI authentication and Access Roles

returntrip — Tue, 29 Aug 2023 15:54:02 GMT

I could manage the cluster (i.e.: kubectl get nodes, get pods etc)

Re: Unable remove instance - CSE not found

returntrip — Tue, 29 Aug 2023 15:52:26 GMT

I think the answer to the root CA issue is to add teh certificate to "Cluster Certificates (Optional)" in the "CSE Management" window. Will try and see if it works.

Re: TMC Managed K8s Cluster - CLI authentication and Access Roles

jeffmace — Tue, 29 Aug 2023 15:37:02 GMT

Sorry for the delay, I am still looking into this with the engineering team.

Beyond logging in, were you able to view/edit any TMC resources when using the `Cloud Administrator` role?

Re: Unable to delete TMC-SM add-on instance

nazu — Tue, 29 Aug 2023 11:18:09 GMT

Thanks a lot! seems it helped.

Re: Unable remove instance - CSE not found

returntrip — Tue, 29 Aug 2023 08:30:33 GMT

I managed to delete it manually by:

"curl -ks -H "Accept: application/json;version=37.0" -H "Content-Type: application/json" -H "Authorization: Bearer ${VCLOUD_ACCESS_TOKEN}" -X DELETE https://$VCD_HOSTNAME/cloudapi/1.0.0/entities/urn:vcloud:entity:vmware:solutions_add_on_instance:72f202b9-a8a9-46ac-8ebd-9fa4490d0f0b"

The next problem is that the CSI 4.1 Plugin does not have a certificate session during cluster creation. I will need to find a way to add the certificate after cluster creation

Re: Unable to delete TMC-SM add-on instance

jeffmace — Mon, 28 Aug 2023 17:01:57 GMT

Deleting the cluster does not remove entries from the VCD database related to the solution. There are steps on page 32 to delete the solution. Could you try those steps if you haven't already? You may need to follow the steps on page 31 to mark the solution as FAILED.

I will reach out to the engineering team to get some next steps if that doesn't work.

Re: Unable remove instance - CSE not found

jeffmace — Mon, 28 Aug 2023 16:52:06 GMT

I will discuss this with the engineering team and get back to you.

Unable to delete TMC-SM add-on instance

nazu — Mon, 28 Aug 2023 15:08:23 GMT

I am having issues with the installation of TMC-SM.

I got till the page 22 of installation – tried to install TMC-SM add-on instance.

It run for some time, then stopped and remained in ‘In progress’ state.

So now I am not able to either delete it, or create a new one (as only 1 instance is supported).

I tried deleting tmc Kubernetes cluster from VCD UI and the project from harbor and recreated those from the scratch but seems something else needs to be cleaned up.

Attached the log from the tmc instance installation.

Could you help with deleting it?

Unable remove instance - CSE not found

returntrip — Fri, 25 Aug 2023 15:14:56 GMT

I am not sure if "CSE4" is referring to a VM or vApp or if that is just some hardcoded name and of no consequence to the search.

I am asking cause I have just updated CSE to 4.1 and delete the previous vApp/VM (IIRC both called CSE4).

Would be great to have some help with this as I need to remove this instance and reinstall it.

root@PhotonOS-001 [ ~ ]# /mnt/cdrom/linux.run delete instance --name $TMC_SM_INSTANCE_NAME --accept --host $VCD_HOSTNAME --username $VCD_USERNAME --certificate-file /tmp/vcd.pem --encryption-key ${TMC_SM_ENCRYPTION_KEY} --accept --password $VCD_EXT_PASSWORD
INFO [0019] Triggering action action=hook event=PreDelete
INFO [0021] All global roles are ready to delete action=hook event=PreDelete
INFO [0021] cluster:tmc action=hook event=PreDelete
INFO [0021] Get Solution Org action=hook event=PreDelete
INFO [0021] Solution Org: CSE action=hook event=PreDelete
INFO [0021] Search CSE4 Cluster action=hook event=PreDelete
ERROR [0021] Failed to find any cse cluster in org CSE action=hook event=PreDelete
ERROR [0021] Failed to delete instance 'tmc' name=tmc
ERROR [0021] Failed to find any cse cluster in org CSE: exit status 23: failed to execute trigger hook errorCode=5012120012191213

Re: API Error: Failed to list cluster's integrations: Not Implemented

jeffmace — Fri, 18 Aug 2023 00:37:14 GMT

This is a known issue and will be fixed in GA.

Re: Unable to reconcile tanzu-standard repo

jeffmace — Fri, 18 Aug 2023 00:27:47 GMT

I believe this can be solved by adding the root CAs to the kapp-controller pods.

Generate a ca-certificates.crt file with the contents of all CAs to be trusted.

rm -f ca-certificates.crt
cat rootCA.crt >> ca-certificates.crt
# Repeat for all trusted CAs

Load the certificate bundle into Kubernetes and update the kapp-controller deployment to include it in all pods.

kubectl create -n tkg-system configmap kapp-controller-ca-certificates --from-file=ca-certificates.crt

cat <<EOF | kubectl patch -n tkg-system deployment/kapp-controller --patch-file=/dev/stdin
spec:
  template:
    spec:
      containers:
      - name: kapp-controller
        volumeMounts:
        - mountPath: /etc/ssl/certs/ca-certificates.crt
          subPath: ca-certificates.crt
          name: ca-certificates
          readOnly: true
       volumes:
       - configMap:
           name: kapp-controller-ca-certificates
         name: ca-certificates
EOF

The kapp-controller pods will restart with the new configuration and should start working. You can follow the kapp-controller logs for more details.

kubectl -n tkg-system logs -f deployment/kapp-controller

TMC Managed K8s Cluster - CLI authentication and Access Roles

returntrip — Wed, 16 Aug 2023 13:12:00 GMT

Currently, I can login to TMC CLI in the following ways:

1) Using LDAP accountswith `Cloud Administrator` role

2) Using LDAP account with role `tmc:admin`

3) Using local accounts `tmc-amin`, `tmc-member` or any other local accounts with role `tmc:admin` or `tmc:member` assigned to them

I cannot authenticate to TMC CLI from LDAP/local accounts/groups for which I have authentication configured TMC GUI Access section. See screenshot that shows current access policy.

To me, it seems like the `tmc-admin` or `tmc-member` roles are necessary to log ont TMC CLI and subsequentially accesst the K8s API via says kubectl However, having those roles gives automatically admin access to TMC managed K8s clusters which defeats the purpose of RBAC.

Am I missing something?

Unable to reconcile tanzu-standard repo

returntrip — Tue, 08 Aug 2023 11:53:35 GMT

I am unable to reconcile the tanzau-standard repo due to a certificate error. How can I import or trust the authority for the harbor host to overcome this issue?

API Error: Failed to list cluster's integrations: Not Implemented

returntrip — Tue, 08 Aug 2023 08:59:36 GMT

I am getting this error: "API Error: Failed to list cluster's integrations: Not Implemented: please try again later (unimplemented)" as a red banner. Any hints how to solve this? Thanks.