Feedback on BYOEaaS

[smoonen]

I've reviewed the documentation - thanks - but haven't had a chance to test this yet.

It's interesting to me that this solution exists in a space in between provider and tenant. Unlike the org SSO configuration, there is some responsibility on the provider's part to establish connectivity to the KMS. However, there is still a tenant responsibility to authenticate with the KMS.

In the enterprise context, I can understand why this might be a good arrangement. However, in a cloud context, it is likely that the cloud provider is managing both Director and the KMS. In this case the cloud provider might have means of managing both the network connectivity to KMS as well as the authentication to KMS. As a result:

1. It seems desirable to me for this solution to optionally allow the provider to manage authentication instead of the tenant. In this case, the provider should be able not only to publish the KMS to a tenant org, but there should be a way to ensure that existing and new tenant VDCs are enrolled in the KMS rather than in the default key provider.
2. It seems highly desirable to me for APIs or CLIs to be exposed allowing the provider to automate all of this configuration. The documentation as currently written only shows UI operations.

In fact I had pictured that this feature would be offered by establishing a vCenter KMS connection and selecting a Key Provider per org much like vCenter today allows you to select a Key Provider per cluster. It's interesting to me that you've chosen to implement this as a solution addon instead.

Thanks for the opportunity to review this early!

[rtsanev]

Hi @smoonen ,

Thanks for your feedback and for making the next step in testing the solution add-on!

 > It's interesting to me that this solution exists in a space in between provider and tenant. Unlike the org SSO configuration, there is some responsibility on the provider's part to establish connectivity to the KMS. However, there is still a tenant responsibility to authenticate with the KMS.
[Radostin] Yes, you are right. One of the primary use cases of the solution is Sovereign Cloud where tenants want to own and control their encryption keys. The provider registers the KMS because they have to ensure the connectivity between VC and KMS and carefully review the KMS certificate and decide if they want to trust it. This trust is being stored in the underlying vCenter. When they publish the KMS to their tenants, the tenant admin needs to authenticate with their own credentials because it's their own KMS or their own space in a multi-tenant KMS.

 > In the enterprise context, I can understand why this might be a good arrangement. However, in a cloud context, it is likely that the cloud provider is managing both Director and the KMS. In this case the cloud provider might have means of managing both the network connectivity to KMS as well as the authentication to KMS. As a result:

1. It seems desirable to me for this solution to optionally allow the provider to manage authentication instead of the tenant. In this case, the provider should be able not only to publish the KMS to a tenant org, but there should be a way to ensure that existing and new tenant VDCs are enrolled in the KMS rather than in the default key provider.
[Radostin] It seems the idea is the provider to be able to manage multiple KMS systems and then publish those to their tenants so that they do not use the default key provider. It goes off the initial purpose of the solution which is to allow tenants to control their own encryption keys but it is a good use case and if there is wider need for such we can plan of adding in one of the next releases. Help me understand the use case - why do you want to manage multiple KMS and share KMS1 to tenant1 and KMS2 to tenant2? If you as a provider manage the encryption keys for your tenants and this is transparent to them, do they care in which KMS you store the keys?

 > 2. It seems highly desirable to me for APIs or CLIs to be exposed allowing the provider to automate all of this configuration. The documentation as currently written only shows UI operations.
[Radostin] The solution add-on comes with a full set of APIs which support all of the operations which you see in the UI. Actually the UI is entirely using this API to perform its operations so once you install the add-on which can be done through a CLI, you can use those APIs to build your own experience.

 > In fact I had pictured that this feature would be offered by establishing a vCenter KMS connection and selecting a Key Provider per org much like vCenter today allows you to select a Key Provider per cluster. It's interesting to me that you've chosen to implement this as a solution addon instead.
[Radostin] In VCD, solution add-ons are the way to deliver faster and more solutions / extensions which our providers can monetize on so expect to see more of these in the future 🙂

Thanks for the opportunity to review this early!
[Radostin] We look forward to more of your feedback!

[smoonen]

Thank you, Radostin. Here is an additional question and a few brief replies:

Does the existing VMware KMIP certification process cover this solution? I.e., the solution does not leverage any more KMIP capabilities than are used today by vSphere, vSAN, and vTA encryption?

I understand the sovereign cloud motivation. However, it seems to me that there are many more gaps to close. A vCenter/Director administrator with sufficient privileges can access tenant VM consoles and VM disks today even if they do not have access to the key provider credentials. Do you have a roadmap for closing such gaps? Without such a roadmap it seems premature to name this sovereign cloud.

In our particular case, we operate Director for our tenants and we also operate a multi-tenant key provider for them. It is our vision that we will connect the customer org directly to the customer's key provider instance, including the establishment of credentials, all transparently to the customer. Our desire is that the customer need not manage network connectivity, key provider credentials, or enrollment of particular VDCs in their org. The customer may still revoke either their root key or individual keys within the key provider instance as a means of retaining the right of cryptographic erasure.

Thanks!

[rtsanev]

Does the existing VMware KMIP certification process cover this solution? I.e., the solution does not leverage any more KMIP capabilities than are used today by vSphere, vSAN, and vTA encryption?
[Radostin] Correct. We ask vCenter to handle all operations with KMIP so whatever they support as infrastructure, we also support it.

I understand the sovereign cloud motivation. However, it seems to me that there are many more gaps to close. A vCenter/Director administrator with sufficient privileges can access tenant VM consoles and VM disks today even if they do not have access to the key provider credentials. Do you have a roadmap for closing such gaps? Without such a roadmap it seems premature to name this sovereign cloud.
[Radostin] Manish Arora is the PM for Sovereign Cloud so maybe he can comment on the roadmap there. I understand the concern about the VM consoles but how they can access the encrypted VM disks?

In our particular case, we operate Director for our tenants and we also operate a multi-tenant key provider for them. It is our vision that we will connect the customer org directly to the customer's key provider instance, including the establishment of credentials, all transparently to the customer. Our desire is that the customer need not manage network connectivity, key provider credentials, or enrollment of particular VDCs in their org. The customer may still revoke either their root key or individual keys within the key provider instance as a means of retaining the right of cryptographic erasure.

[Radostin] Let me confirm that I understand you correctly. This basically means that you as a provider manage the KMS and manage the encryption keys on behalf of your tenants. The provider setups the connectivity, authenticates to KMS and also specifies that key1 needs to be used for encryption of the VMs in tenant's Org VDC1 (or the full org for the sake of the example). For your tenant, all of the above would be fully transparent and they would not have to go to BYOE to setup anything. But still, they would be able to login in their KMIP tenant, and observe the keys (key1) which VCD is using there.

If the above is true, what workflows do you expect the tenant to be able to do with the encryption keys in their KMIP?

For example, since you as a provider setup key1 for this customer, then the customer cannot just log in their KMIP tenant and revoke key1 because this would break their VMs until they are recrypted with a valid key. Meaning that the tenant admin needs to call their provider and ask them to replace key1 with key2 and then revoke key1 becuase in this setup the provider manages the keys.

I will be happy to jump on a call and discuss the use cases - I believe it will be much faster and more efficient. If you are OK maybe @jaskaranv can help us arrange it?

Thanks again for your feedback!

[smoonen]

Thanks, Radostin! Some further replies:

For VM disks, I am thinking for example of cloning the VM and decrypting the clone in vCenter console. Alternately we could use a backup provider with crypto admin access to backup decrypted disks and restore them elsewhere.

Your understanding of our setup is correct. Keep in mind that I was using vSphere encryption as my mental model, so that each VM would have its own key issued by the KMS, which ideally would be visible to the tenant in the Director portal as an attribute of the VM, and which would also allow for the rekeying of VMs on an individual basis. It seems to me your setup is more similar to vTA in that a single key issued by the KMS will be used to protect the keys in use for all VMs. So I agree with you that with this approach there is not as much apparent value to the tenant as there would have been in the key-per-VM scenario that I envisioned. It seems like there is some dissonance between what your model wants to achieve and what we want to achieve for our customers.

I'm happy to get on a call. I work regularly with Jon Schulz as well. I have availability this week but am traveling the following two weeks, then will be back in the office the week of the 18th. Thanks!