All Posts

Following up on what sakthi2019 said, if you notice the replicas has changed after checking the RDE in entity->spec->capiYaml (you could control/command F and search for replicas if possible as well), then it is possible that the pod responsible for applying updates (rdeprojector) has stopped reconciliation. This is a known issue of the RDEProjector but has been fixed for GA. In order to resolve this you may need to delete the rdeprojector pod and let it restart. To do this, you will need to access the cluster with Kubernetes. 1. Download the Kubernetes config associated to your cluster from the UI. 2. After downloaded Kubernetes config, make sure you know the path of where it is as it needs to be specified when using Kubernetes commands. 3. Get the list of pods running by running 'kubectl get pods -A --kubeconfig=/path/of/kubernetes-config.txt'  4. Look for a pod with a name starting with 'rdeprojector-', and perform a delete command using 'kubectl delete pod -n rdeprojector-system rdeprojectorPodName --kubeconfig=/path/of/kubernetes-config.txt', this should force restart rdeprojector as it will automatically bring up a new pod after, and after sometime the new updates should apply.

Good to know that you have created a cluster.  Resize issue is a known issue for beta. CSE polls the RDE at regular interval to pickup any change. You may check using https://{{base_url}}/cloudapi/1.0.0/entities/types/vmware/capvcdCluster/1.1.0 to see the RDE got updated : entity->spec->capiYaml has got the updated replicas for worker node.      

Thank you for the reply. I can confirm that the "machine" is in pending state. I have also noticed pending task at the load balancer level. I do not know if they are at the origin of this issue or a symptom of it. No member in the pool so maybe they are not there because machine is in pending state, or the machine is in pending state because the pool doesn't have the members.  

capi-kubeadm-boostrap-system/logs.txt has : has the following infra issue. we will check more and provide any troubleshooting "reconciler kind"="KubeadmConfig" "worker count"=10 I0804 13:47:51.777262 1 kubeadmconfig_controller.go:236] controller/kubeadmconfig "msg"="Cluster infrastructure is not ready, waiting" "kind"="Machine" "name"="beta004-worker-pool-1-8554c96b77-2hdm5" "namespace"="beta004-ns" "reconciler group"="bootstrap.cluster.x-k8s.io" "reconciler kind"="KubeadmConfig" "version"="1634" I0804 13:47:51.811648 1 kubeadmconfig_controller.go:236] controller/kubeadmconfig "msg"="Cluster infrastructure is not ready, waiting" "kind"="Machine" "name"="beta004-worker-pool-1-8554c96b77-2hdm5" "namespace"="beta004-ns" "reconciler group"="bootstrap.cluster.x-k8s.io" "reconciler kind"="KubeadmConfig" "version"="1641" I0804 13:47:51.817895 1 kubeadmconfig_controller.go:236] controller/kubeadmconfig "msg"="Cluster infrastructure is not ready, waiting" "kind"="Machine" "name"="beta004-worker-pool-1-8554c96b77-2hdm5" "namespace"="beta004-ns" "reconciler group"="bootstrap.cluster.x-k8s.io" "reconciler kind"="KubeadmConfig" "version"="1641" I0804 13:47:51.838246 1 kubeadmconfig_controller.go:236] controller/kubeadmconfig "msg"="Cluster infrastructure is not ready, waiting" "kind"="Machine" "name"="beta004-worker-pool-1-8554c96b77-2hdm5" "namespace"="beta004-ns" "reconciler group"="bootstrap.cluster.x-k8s.io" "reconciler kind"="KubeadmConfig" "version"="1642" ==== END logs for container manager of pod capi-kubeadm-bootstrap-system/capi-kubeadm-bootstrap-controller-manager-56bdcdf797-skq6h ====

Attached are the logs generated from https://github.com/vmware/cloud-provider-for-cloud-director/blob/main/scripts/generate-k8s-log-bundle.sh  I have stopped the cse service to keep access to the ephemeral VM.

I just noticed from Software requirements: VCD 10.3.3.1 (tested). Will work with VCD 10.3.1 or newer NSX-T 3.1.1 Avi 20.1.3 Does CSE next works with newer version of NSX and AVI as well? (So it is a minimum version and not an exact version requirement) (Theoretically, CSE is only supposed to communicate with Cloud Director and let cloud director communicate with the other.) In our environment which was supported for cse 3.1.3 we are using: NSX-T 3.1.3.4 AVI 21.1.1 And the NSX-T version is managed by the Vcloud Foundation so a downgrade is not an option.

Hello,  Okay, so you're seeing the known issue where some non-TKG OVA vapp templates are being read as TKG OVA vapp templates by UI plugin. The UI Plugin v4.0.102 that I linked to above has that issue fixed, and that issue will be fixed in GA as well. Thank you!

Hello! I can help with the error you're seeing in the TKG OVA datagrid. Your CSE process seems to be running fine. I tried accessing your VCD testbed at 172.21.19.51, but seems it's not accessible.  There's a few things we can try here. Let me know if you'd like to schedule a zoom call to go over this: 1. There is a known issue where non-TKG OVA vapp templates that are visible to the current user are being read by UI plugin as TKG OVAs. This can cause the error you're seeing. To quickly test this, you can either delete all non-TKG OVA vapp templates, or create a user in an org where only the TKG OVA vapp templates are visible (you can verify this with a Postman GET request to "https://{{host}}/api/query?type=vAppTemplate&format=records&page=1&pageSize=20&filterEncoded=true&sortAsc=name&links=true"  2. Alternatively, I have a test UI plugin build v4.0.102 (beta build is v4.0.101) here where this specific known issue is fixed: https://artifactory.eng.vmware.com/ui/native/cloud-director-solutions-generic-local/container-ui-plugin/4479576/ . Can you download the zip file, go to provider VCD -> customize portal -> upload the zip file -> disable container ui plugin v4.0.101 -> enable and publish container ui plugin v4.0.102 -> refresh browser -> try the workflow again to see if the error is gone. Please let me know how it goes or if you'd like to schedule a zoom session