5 Replies Latest reply on Apr 24, 2019 5:16 PM by TheBobkin

    vSAN Encryption and vCenter lockout scenario.

    SkyVega Lurker

      Seeking clarification. In my lab (in preparation for vSAN Encryption in production) I have a KMS server running outside the vSAN cluster. vCenter runs on the encrypted vSAN datastore. I'm trying to avoid a scenario where vCenter's temporary unavailability locks me out of the cluster. For example, a shutdown of the entire cluster. I'm seeking clarification on the two statements below from 6.7 documentation.

       

      1. vSAN Encryption documentation 6.7 (How vSAN Encryption Works ) states that if a host reboots, the host requests it's KEK from the KMS server. This is ideal, the host is not dependent on vCenter to obtain it's KEK.
      2. VM Encryption documentation 6.7 (How vSphere Virtual Machine Encryption Protects Your Environment ) states that if a host reboots, vCenter on behalf of the host, requests the KEK from the KMS server and then makes it available to the host. This is not ideal. vCenter runs on the encrypted datastore. Encrypted datastore cannot be mounted until host has obtained it's KEK, which it cannot because vCenter is down. Circular dependencies

       

      For a test, I shutdown all 5 hosts including vCenter. KMS server not running on vSAN datastore is still running. I then powered up each host, took them out of maintenance mode and they requested their KEK from the KMS server and mounted their disks and the vSAN datastore. Yay!.

       

      The question I have is why the conflicting information on how the host obtains it's KEK? Unless i'm missing something

       

      Sky

        • 1. Re: vSAN Encryption and vCenter lockout scenario.
          TheBobkin Virtuoso
          vExpertVMware Employees

          Hello SkyVega,

           

          The information is different because they refer to two entirely different Encryption features and mechanisms - VM Encryption is not vSAN Encryption.

           

          It also states clearly in the documentation to not use VM Encryption on vCenter or other management appliances (so as to avoid the circular dependency you noted):

          Virtual Machine Encryption Best Practices

           

          I would advise in general to not run/store vCenter on the vSAN cluster it is managing as it makes troubleshooting more difficult if you encounter issues.

           

           

          Bob

          • 2. Re: vSAN Encryption and vCenter lockout scenario.
            SkyVega Lurker

            Hello Bob,

            Thanks for your response. I understand that the two are different and not to use both at the same time.

             

            My question/confusion is around how the hosts obtain the KEK. Obtaining the KEK is common to both types of encryption. I was curious why VM encryption has the hosts dependent on Vcenter to obtain the keys vs vSAN encryption where the hosts contact the KMS server directly.

            Finally, running Vcenter on an encrypted vSAN datastore is a VMware supported topology.

             

            Thanks again for your time,

            • 3. Re: vSAN Encryption and vCenter lockout scenario.
              TheBobkin Virtuoso
              VMware EmployeesvExpert

              Hello Sky,

               

               

              "Thanks for your response. I understand that the two are different and not to use both at the same time."

              Always happy to help. Actually they are not mutually exclusive and serve different (also somewhat mutually exclusive) purposes - there are caveats to using both simultaneously though such as effectively no dedupe for encrypted VMs if dedupe is enabled (and as an aside, enabling every feature always comes at a performance price).

               

              "I was curious why VM encryption has the hosts dependent on Vcenter to obtain the keys vs vSAN encryption where the hosts contact the KMS server directly."

              This is because of the inherit different properties of how these function based on what they are encrypting - VM encryption is working from vCenter inventory Objects (VMs, vmdk) while vSAN Encryption is encrypting vSAN Disk-Groups.

              It wouldn't make much sense to have a vCenter requirement for vSAN Encryption as vSAN doesn't really care too much if vCenter is not available - sure, vSphere features such as DRS/vSwitch configuration won't be available but the vsanDatastore doesn't go anywhere and VMs continue to run happily (it wouldn't be a very good storage design if this wasn't the case! ).

              Some good overview resources:

              How vSphere Virtual Machine Encryption Protects Your Environment

              How vSAN Encryption Works

              Understanding vSAN Encryption: Booting when vCenter is Unavailable

              Understanding vSAN Encryption - KMS Server Accessibility - Virtual Blocks

               

              "Finally, running Vcenter on an encrypted vSAN datastore is a VMware supported topology."

              Yes it sure is, as vSAN Encryption doesn't require the vCenter to unlock its Disk-Groups and boot the vCenter in the first place - applying VM Encryption to the vCenter is not supported for this reason.

              You noted KMS is running elsewhere, I do hope this the intention in Production as running KMS on vsanDatastore is not supported for pretty much the same reason vCenter with VM Encryption is unsupported in that you can't access what you can't boot because you can't boot what you can't access .

              Do consider also running vCenter where KMS is (or elsewhere) though anyway - this is how the vast majority of our customers are doing it just that it makes it easier when someone sits on a switch.

               

               

              Bob

              • 4. Re: vSAN Encryption and vCenter lockout scenario.
                SkyVega Lurker

                Bob,

                Thanks for your responses and links to resources. Super Helpful and appreciated!

                Sky.

                • 5. Re: vSAN Encryption and vCenter lockout scenario.
                  TheBobkin Virtuoso
                  vExpertVMware Employees

                  Hello Sky,

                   

                   

                  As I said, more than happy to help - if our documentation is ever unclear or too verbose and/or split in multiple directions, this is where persons like myself may be able to help clarify the situation and everyone benefits from it (if they read Communities resources anyway!).

                   

                  I work in vSAN-GSS, so I much prefer discussing this kind of thing now when it is pre-Prod/Test and options can be carefully considered than having someone on a call telling me they have an inaccessible VM encrypted vCSA (unsupported!) with the KMS stored on vsanDatastore (unsupported!) and everything is locked (if we could bypass encryption then it is not encryption, restore from back-ups).

                   

                  For reference of anyone reading this in future - I would urge you to read and understand the relevant documentation relating to any form of encryption (VMware or otherwise) before implementation as getting locked out of access to anything is an implementation fail. Not in any way saying clusters should not consider encryption, just stating if you are going to use a(ny!) tool you should understand the implications of using it.

                   

                   

                  Bob