Design Considerations

[Mouhamad]

Hello,

I'm planing on deploying 5 hosts in a cluster and I'm thinking of the following:

1) Is it recommended that all host are runing the same configuration (Memory, NICs, CPU, etc)?

2) Is it recommended to have one vendor? (HP, or Dell,) or better to mix?

3) Is there any where I can check the best compatible hardware with ESXi 5? An official website?

Can you please provide me with reference links to the answers in addition to your own experience with justification if possible?

Regards,

[VirtuallyMikeB]

Hi Mouhamad,

Very broadly stated, it is recommended that hosts in a cluster have the same configuration.  Now, there are several things to think about here, though.  First and foremost, you want hosts in the same cluster that have processors that come from the same vendor class, i.e. Intel or AMD, for vMotioning (and thus DRS) purposes, and the same processor family.  Slightly different processors may be vMotion compatible if Enhanced vMotion Compatibility (EVC) is enabled at the cluster level.  See page 55 in the vSphere 5 Resource Management guide here:

http://pubs.vmware.com/vsphere-50/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-50-resourc...

You also want to have the same about of physical RAM installed in each server in a cluster.  This will allow you to better plan for HA events and get the most out of your hardware.  We're talking about slot sizes here (specifically the RAM portion of slots) and how many resources are reserved (based on slot sizes) on other hosts in the cluster.  Say, for instance, you configure an HA cluster to tolerate the failure of one host.  You have three hosts with unbalanced amounts of RAM, say 16 GB, 16 GB, and 32 GB.  If you're using close the entire 32 GB of RAM on the one host, then HA will need to reserve enough resources on the other two hosts to start up all the VMs running on the larger host, which amounts to nearly all the RAM in the remaining two hosts - leaving no room for VMs to run on the other two hosts.  So your'e wasting two entire hosts' worth of resources with unbalanced RAM across hosts in a cluster.  Duncan Epping and James Bowling talk about these topics here:

http://www.yellow-bricks.com/2009/08/12/ha-and-slot-sizes/

http://vsential.com/2011/03/should-servers-in-a-cluster-have-the-same-everything/

Page 31 of the vSphere 5 Availability Guide recommends as a best practice setting the Percentage of Cluster Resources Reserved as the admission control policy to 1/N, where N is the number of hosts in the cluster.  http://pubs.vmware.com/vsphere-50/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-50-availab...

It's okay to mix server vendors as long as the processors are similar enough to allow vMotion between the hosts.  As with the above paragraphs, the closer matched each host is in a cluster, the better.

And finally, the best place to look for hardware compatibility is VMware's own HCL.  Many big name vendors will already have their hardware certified and you can probably push the burden on them to prove their hardware is on the HCL.  For everyone else, there's the VMware HCL here

http://www.vmware.com/resources/compatibility/search.php?rct=j&q=&esrc=s&source=web&cd=1&ved=0CCgQFj...

I hope this helps a bit.

Cheers,

Mike

http://VirtuallyMikeBrown.com

https://twitter.com/#!/VirtuallyMikeB

http://LinkedIn.com/in/michaelbbrown

Message was edited by: VirtuallyMikeB

[Mouhamad]

Hey Mike,

Thanks a lot for the information.

One last thing which I'm looking for, mixing hardware from different vendors (recommended or not?).

I'd like to take everyone's opinion if possible.

Thank you,

[EdWilts]

One last thing which I'm looking for, mixing hardware from different vendors (recommended or not?).

My personal preference is to not mix and match although I do have a cluster with a mix of Dell and HP hardware.  The Dells happened to have the same CPUs and memory config as the HP blades or I would have not have done so.

[mcowger · Accepted solution]

Most would recommend against it.  You'll end up with different configs for different types of hosts: more configurations, firmware version, vendors to manage.

I dont see a good reason to mix vendors.

[VirtuallyMikeB]

Absolutely.  If one can help it, don't mix servers from different vendors.  I was talking to a collegue about this this post and he brought up a good point.  If we're just talking compatibility - vMotion, evenly distributed RAM, etc., then is there a problem with mixing vendors if the hardware is compatible?  It's a good point to bring up about having to manage different firmware and BIOS revisions, etc. for different hosts, however, strictly related to the question of, "will it work and is it supported?" then I would venture to say it's okay to mix vendors.

Now, ideally, if one had a choice in the matter, then sure - don't mix server vendors.  But I think it's reasonable to say for all those shops where politics and budget make a difference, one may not have such a choice.  An admin will have to do with what he has.  It that means enabling EVC and moving RAM around to evenly distribute it around the cluster, then fine.

Cheers,

Mike

http://VirtuallyMikeB.com

https://twitter.com/#!/VirtuallyMikeB

http://LinkedIn.com/in/michaelbbrown

[Josh26]

EVC exists solely for the purpose of minimising the impact of different hardware.

VMware certainly took steps to help you in this direction, with the obvious fact that at the end of the day, it's not always plausible to have completely identical hardware.

Of course, "steps to mitigate" a problem shouldn't discourage you from avoiding that problem completely, but consider this:

We purchased five HP blades.

One year later, we went to add two more to our cluster. The original CPUs are EOL, and we need to buy the current model.

Pretty much everyone runs into this at some point.

[Mouhamad]

Thanks guys for your posts.

I have one argunment which supports mixing vendors: If you run into 1 issue on a specific server in your cluster, you don't see it on all your hosts.

What do you think? Do you find it valid?

[VirtuallyMikeB]

Hi Mouhamad,

While that seems like a fair assessment, my opinion is that the ease of managing one type of server - one BIOS revision level, one firmware revision, same parts for swapping out and ordering - outweighs the benefits of multitple vendors and such an argument.

Cheers,

Mike

http://VirtuallyMikeBrown.com

https://twitter.com/#!/michaelbbrown

http://LinkedIn.com/in/michaelbbrown