I know it is unlikely but let us say host has got owned, ie a hacker has managed to break into the host.
How would they go about breaking into a linux VM that uses full disk encryption?
They can't mess with the .vmdk without damaging it - it is encrypted by the guest.
They can't use vmrun because they do not know the guest passwords.
They can't attach to processes in the guest with debugging tools because they cannot see individual guest processes.
What can they do? And crucially, what can I do as a countermeasure?
If the host has been compromised, a wide variety of attacks are possible. All of guest memory is visible to the host and modifiable from the host. Disk encryption doesn't do any good, because the guest has to be able to decrypt the disk, which means that malware on the host can extract the decryption key from guest memory and decrypt the disk itself.
What if ram is encrypted too with something like the following?
Is it not outside the capabilities of most VPS hosters to find where in ram the key is, as they would have to hire a highly skilled coder and spend a lot of money on them to develop software for this with no guarantee that they will get what they want?
Security of the guest cannot be guaranteed if the host has been compromised. Yes, you can put speed bumps in the way to make it more difficult for hackers to gain access to the guest, but someone with sufficient skill and motivation will be able to circumvent whatever roadblocks you put in the way.
Intel will soon be introducing a technology known as Secure Enclaves (or Software Guard eXtensions) which will provide a mechanism for hiding secrets from an untrusted software platform. A disk encryption system built with SGX could potentially protect the guest from malware on the host, but since the malware can snoop on all I/O performed by the guest, I still wouldn't place much confidence in the integrity of the guest.
What really matters is WHERE you do the encryption. If the encryption is too low, data in the guest appears unencrypted. If it is in the guest, then the keys live in the guest and since SGX is not around at the moment, keys are somewhere in guest memory even for a little bit of time.
So the real question is what are you trying to achieve?
If you are trying to meet encryption at rest requirements then it makes no difference where you encrypt as the data on the disk will be encrypted and without the key no one can decrypt it. Now if you have keys generated within a VM without using DRNGD or some other high quatlity randomness source, then your keys could be predictable and you need to guard against making it easy for a brute force attack.
If you need to encrypt data in motion?
Then you need to consider how the VM is protected itself, how an application interacts with data to determine during 'motion' if someone should not be accessing the data even though they are already supposedly allowed to do so. Keys are in memory, so therefore you need to guard memory access for those keys to only the application in question. This is the hard part, and requires you to think seriously about logging, key management, etc.
So really what are you trying to achieve?
Edward L. Haletky
VMware Communities User Moderator, VMware vExpert 2009-2015
Author of the books 'VMWare ESX and ESXi in the Enterprise: Planning Deployment Virtualization Servers', Copyright 2011 Pearson Education. 'VMware vSphere and Virtual Infrastructure Security: Securing the Virtual Environment', Copyright 2009 Pearson Education.
Just want to add as many obstacles as possible. At any time an exploit may be discovered that allows an infected VM to infect the host, I am sure this is always being searched for. And a while later vmware releases a fix, and a while later another exploit is discovered, and so on. So there is no such thing as perfect security.
How resourceful can you get obstructing malicious intrusions is the question. If almost all ram looks like random bytes, what chances would a committed programmer have in discovering the key in this pile of random bytes? They'd need to reverse engineer the encryption software. Might as well reverse engineer vmware.