Virtual machine disk usage, and allowing enough capacity in your VMFS volume

[vheff]

Hi all,

I've not found much documentation regarding this, but I feel it's an important question surrounding disk space and ensuring you don't leave your ESX servers short of storage! Creating a virtual machine with a 12GB disk doesn't simply use 12GB of disk space, as there is a lot more going on. Firstly each virtual machine has it's own directory, and inside are the VMDK files (disks), Swap file (vswp, which is equal to the RAM), config files (e.g. vmx), logs, and then if you take snapshots you'll get snapshot files (vmsn and -delta.vmdk) which can grow over time. If you are using VI3 with DRS or VMotion, then it is also worth pointing out that when a VM migrates to another host, it creates a copy of the swap file (vswp) during the migration.

Example:

In a particular LUN (VMFS-3 virtual machine store) you have 10 virtual machines. Each VM has 1GB RAM and a 10GB disk. That means that we have 110GB in total (10 x 1GB swap, and 10 x 10GB VMDK). The vmx, nvram and other config files are tiny (around 10k per VM) so I won't add these, and the virtual machine logs files shouldn't take up a lot either, maybe 8MB per VM being generous.

TOTAL = 110GB

Great, so a VMFS-3 volume of 115GB will be more than enough right?

Maybe not...

Now lets say an administrator takes a snapshot of 4 x virtual machines. For each VM, ESX has created a snapshot file (VMSN) of 1GB each (same size as RAM) which equals 4GB (4 x 1GB). If the snapshots are never deleted, as changes occur on each of the virtual machines, a -delta.vmdk file will start to grow. Basically, if you copy a 500MB file to a virtual machine whilst it is in snapshot mode, the delta VMDK will grow by 500MB and a bit. Using that example if each of the four virtual machines have 500MB of changes made, then thats another 6GB of storage required (4 x 1GB for the RAM size, then 4 x 500MB of changes = 6GB)

Now we have a problem, if we had created a volume of 115GB then we would have run out of disk space (110GB + snapshots at 6GB = 116GB). If our administrator also used VMotion to migrate a virtual machine to another host, then it would increase by another 1GB.

In this example, we should have created a volume of 120GB to cover allow enough room for snapshots, VMotion / DRS, etc. But this does raise some questions.

1) Is there a formula to calculating the 'best' volume size for each virtual machine store?

2) Other than snapshots and VMotion / DRS, does anything else significantly increase disk space used?

If anyone has any more thoughts, or maybe disagree with anything I've said then please let me know.

[Randy_B]

I've always tried to stick with EMC's recommendation of ~15 VM's per VMFS volume. We run 500GB luns and keep 10% free space on them for overhead, swap files, etc. Most of our VM's are in the 20-30GB range so the 500GB luns fit well.