Comments please on my VI3 storage design

[array]

Hi everybody,

Please comment on my storage design!

I work on a project where we have bought 39 HP DL460c blades in 3 enclosures. 12 of them are to be used as ESX 3.0.1 servers.

Configuration:

\- dual quad cores (E5345, 2.33 GHz)

\- 16 GB RAM

\- dual channel HBA Mezzanine

\- 2 onboard NICs + quad NIC Mezzanine

\- 2 local 72 GB SAS disks

The blades are connected to an EVA8000 storage array with 144 500 GB FATA disks in one diskgroup (and a few (24) FC disks in a separate diskgroup for sequential use)

Currently, we have 11 DL380G4 servers with around 90 VMs on them. There are no performance issues.

In the new situation we will have initially 150 VMs on these 12 blade servers.

given:

\- 192 VMs, 12 hosts

\- 20 VMs per LUN (~per datastore)

\- max 20 VMs per host (initially max 16 per host)

\- VCB wil be used

\- VM OS volume is 10 G max

\- Average data volume of a VM is 25 GB

\- average VM memory: 1 GB (to be safe)

\- almost all VMs are uniprocessor

\- most of the VMs are light, some normal, no heavy

\- use RDM for VM disks > 150 GB (in case of IO intensive VM: > 50GB)

design:

\- max 8 datastores per host

why:

VMWorld 2006 presentation http://download3.vmware.com/vmworld/2006/tac9516.pdf) states that in an enterprise situation you can use 12 DS per host, in a midrange situation 6 DS per host. Since the EVA8000 is an enterpise array, but the FATA disks are not, I figured that 8 DS / host must be doable

\- seprate VMFS volumes for OS and data disks. This means that per host we can have 4 OS volumes and 4 data volumes. At first I was considering separate pagefile volumes, but that would increase the number of datastores above 8

\- VM OS volume is 10 G max

\- Average data volume of a VM is 25 GB

so: 4 OS volumes + 4 data volumes with 20 VMs per volume = 80 VMs

max 20 VMs per host => 4 hosts, so 3 clusters of 4 hosts total

OS Datastore size:

20 VMs * 10 GB = 200 GB

swapspace (= 0.5 * VM memory) = 20 * 0.5 GB = 10 GB

config & logs (= 10 MB / VM) = 20 * 10 M = 0.2 GB

redo logs (20% of 200 GB) = 40 GB

\----

---

total = 250 GB

Data Datastore size:

20 VMs * 25 GB = 500 GB

redo logs (20%) = 100 GB

\----

---

total = 600 GB

So then we have:

3 clusters with 4 hosts each

every host has 8 datastores (4 x OS of 250 GB, 4 x Data of 600 GB)

Questions:

\- is the 3 cluster setup a good idea?

\- are the sizes for the datastore "right" ?

\- do seperate OS and data datastores make sense and do they count as 2 datastores?

\- is it a better idea to increase the number of VMs per datastore, increase the sizes and have fewer clusters?

\- did I miss something?

Thanks a lot in advance!

Remko

[MR-T]

It looks like you've done your homework with this.

Can I ask why you've decided to create 3 clusters and not 1 single cluster?

The disk sizing looks good.

Do your vm's require 1GB of RAM?

As you've only got 16GB to begin with and the VMkernel will require a portion, your initial 16 vm's per host will already put you in the red for RAM. Although it's unlikely the vm's will use all this, it might be worth lowering this a little to give yourself some room to move.

Keeping the number of vm's on a LUN to 20 or below is a good practice, you should see better performance from doing so.

[wobbly1]

I can see that the 3 clusters is to keep the # of DS's per host down to 8 (4 x o/s & 4 x data)

My only question there is do you have the capacity spare within the clusters to support the level of failover you are looking for - i.e. if using HA do you have the headroom within the cluster to be able to power on the VM's from the failed host onto the remaining nodes?

Other than that, like Mr-T says, you've done your homework and figured it through very well - good luck with the implementation

[array]

Thanks for the replies!

The 3 cluster solution is indeed to keep the number of DS down to 8 but does anybody have experience with this number in combination with an EVA8000 with FATA disks?

And if I split the OS and data to 2 different datastores as I do, does it have the same impact as 2 datastores with OS and data combined?

About the 1 GB of RAM per VM: this is including virtualisation overhead. At least half of the VMs have 512 MB RAM configured.

About 20 VMs/LUN: What is the definition of LUN in this context? Would it make a difference if I use 3 LUNs for 1 VMFS volume? Can I then place 60 VMs on that VMFS volume?

About the number of hosts in a cluster: I'd rather have one big cluster, but then I have to increase the number of DS / hosts. Is that a good idea? Would it be better to make 2 clusters with 6 nodes?

That would look like:

6 hosts / cluster => 6 x 16 VM/host = 96 VM / cluster

20 VM / DS => 5 OS + 5 Data DS = 10 DS / host

Thanks again,

Remko

[JDLangdon]

- VM OS volume is 10 G max

- Average data volume of a VM is 25 GB

I don't know if it is practicle to determine the size our your VM's at this stage in the game. When I started looking at our SAN setup we went with similar numbers but now I'm getting request for VM's that have 25GB OS volumes and 50GB data voulmes.

Jason

[array]

I have to make assumptions, otherwise I won't be able to size the datastores and determine #DS/host and #hosts/DS and the number of hosts per cluster etc.