Hi,
we are using VSAN as backend storage for VMware Integrated OpenStack which is used by kubernetes. For each Cinder volume there is a powered off shadow VM created with a VMDK disk. There are some limits that we might hit in the future, beause there can be multiple volumes per kubernetes node.
VMware vSphere:
# Virtual Machines per cluster 8000
# Virtual machines per host 1024
vSAN:
# Virtual machines per host 200
# Virtual machines per cluster 6000
# Components per Virtual SAN host 9000
For each Cinder volume a shadow VM is created with a VMDK attached, so there will be a lot of powered off VMs just to be able to provide Cinder volumes for Kuberntes. Currently we already have ESXi hosts with 140 VMDK's (58 powered on / 72 powered off as docker images). It seems the 'shadow VM' concept has some major downsides.
What is VMware's concept for larger VIO/K8N deployments?
