vSAN Encryption Issues

[Valjean07]

Environment:

ESXi 6.7u3 and vCenter 6.7u3, witness appliance upgraded to ESXi 6.7u3

2 node vSAN cluster with virtual witness appliance

- All drives are HDDs

- 3 disk groups on each node

After encryption, vSphere says that everything is OK health wise.

When verifying on one of the ESXi host, I find that 2 of the 3 disk groups are good. However, I have one disk group, whose drives have an Encryption status of false. In addition, the DiskKeyLoaded is also false.

vdq -iH                                         lists all 3 disk groups

esxcli vsan storage list                 shows CMMDs on all drives, shows all drives mounted

How do I encrypt a single disk group via CLI ? And, what causes this situation to occur?

[TheBobkin]

Hello Valjean07​

Welcome to Communities.

"However, I have one disk group, whose drives have an Encryption status of false. In addition, the DiskKeyLoaded is also false."

"And, what causes this situation to occur?"

The likely explanation is that the KMS was not available when this Disk-Group was created.

"How do I encrypt a single disk group via CLI ?"

There should be no need to use the CLI (but you can of course if you prefer) - validate that the KMS cluster is functional and showing as healthy in vCenter inventory and in vSAN Health checks ('vCenter KMS Status' and 'Host KMS Status'), if these are okay then proceed with recreating the unencrypted Disk-Group - preferably use 'Full Data Migration' option but if you have backups and understand that some data will have reduced-redundancy until resynced then use 'Ensure Accessibility' option.

Recreate a Disk Group

(One can use remove and then add or recreate as above in 6.7 U3)

Then of course validate that Encryption is enabled and DiskKeyLoaded: true.

Bob

[Valjean07]

Thank you for the reply Bob.

I am using a non-HTML5 compliant browser (which cannot be changed) so, I do not see 'Recreate a Disk Group'. I am guessing that it just removes the disk group, grabs a DEK, and then adds the disk group back into the datastore. However, I have already tried removing the disk group and adding the disk group back in without success.

I am able to ping the KMS from the host. And, other disk groups on the same host can be removed and added without issues.

[TheBobkin]

Yes, it basically just removes and recreates the Disk-Group.

"And, other disk groups on the same host can be removed and added without issues."

This could indicate that there is something problematic with that Disk-Group - I say this as if it was a problem with KMS-connection or the Encryption status of that node then you would get another unencrypted Disk-Group when recreating the others.

Is Health Green and vmkernel.log/dmesg 'clean' with regard to any possible disk issues when the Disk-Group is created? Are you able to write data to it?

Though I will say it is in the realm of possibility that it could be a case of bad timing and connection to the KMS is being lost each time you are recreating that Disk-Group - what it the RTT on the ping to the KMS and is it consistently low? (e.g. don't test it with just a few pings do -c 200 and while recreating the Disk-Group)

If it is solely an encryption issue and no issues with the storage devices then there are some good pointers on which logs to check here:

How to troubleshoot encyrpted vSAN cluster- Virtuallysensei.com

Troubleshooting vSAN Encryption and KMS Connectivity | thevirtualpaddy

Bob