How many ESX hosts in a cluster?

[thomps01]

I'm sure it's a very much "It depends" answer, but I'm interested to know peoples thoughts.

The cluster limit is 32, but who actually fills their clusters up?

I'm asking the question because as a rule I've always created clusters with between 6 & 8 hosts in them and wanted to see if this was still valid.

If I'm given 50 ESX hosts to work with, rather than create 2 large clusters, I might build 7.

Why? A number of reasons, but usually primarily storage. To keep DRS & VMotion happy, I'll share a number of LUNS between these hosts. If I'm looking to run 20 VM's on a host then I'll split these between 2 LUNS (usually). Multiply this by the number of hosts in the cluster and I've got 14 LUNS.

When configuring DR, I usually do this on a cluster by cluster basis which makes replication of the storage easy to manage as I know exactly where I am. If I've only got to focus on the 14 LUNS, it makes my life a little simpler.

I don't seem to find a use for resource pools, so adding additional CPU, RAM to a cluster isn't really helping me out.

The second reason to keep them small is normally a just to sepearet dev/test from prod.

Also Backups come to mind. Trying to manage hundreds of virtual machine backup schedules can be a pain, so I tend to focus on a cluster at a time. If using VCB, you can align a specifc proxy server to a cluster if you keep the number of hosts in this cluster to a handful.

Am I mad?

Should I throw this idea in the bid and simply create 2 large clusters moving forwards?

[Ken_Cline]

Am I mad?

Not at all.

Should I throw this idea in the bid and simply create 2 large clusters moving forwards?

I would not recommend it. Even though you CAN put 32 nodes in a cluster, it's not something I would suggest. Most storage systems start to experience increased SCSI reservation issues with eight or so hosts (depending on activity). Since you're planning to use VCB, you're going to have active snapshots (which increases SCSI reservation activity), so keeping clusters small (10 or fewer hosts), in my mind, is a good idea.

Ken Cline

Technical Director, Virtualization

Wells Landers

TVAR Solutions, A Wells Landers Group Company

VMware Communities User Moderator

[apatel1]

Another reason to avoid reaching the full 32 node limit is that the HA cluster only designates the first five nodes of your cluster as "primary" nodes, which hold information regarding VM ownership throughout the cluster. I believe that, if you were to lose these 5 nodes (however unlikely that might be) and a failure occurs, you might run into trouble.

Hope that helps! Please help me out by marking my response as "helpful" or "correct" if you feel that it was useful!

-Amit

[weinstein5]

Just an FYI - the number of primary nodes in an HA cluster is dependent on the number of host failures HA will support - so if your HA cluster will support 1 host failure you will only have two primary nodes - the maximum number of failures HA can support is 4 so 5 primary nodes -

[thomps01]

Thanks for the sanity check Ken.

I'll keep with my current design methodolgy then.

[Rodos]

The most important and often forgotten consideration is LUN pathing.

If you are using FC, the maximum number of paths you can have to a LUN is 32. If you have redundant paths in each of the ESX hosts (which you should), you really have a maximum of 16 hosts in the cluster. I am not sure if this number is affected by a path from a VCB server? Anyone know?

The limit can be found in the configuration maximums PDF

Rodos

Considering awarding points if this is of use

[mcowger]

We have 32 hosts in 1 of our clusters.

--Matt

[AllBlack]

I thought the maximum nodes in a cluster for HA was 16 while it is 32 when only using DRS?

[PaulCooper]

I thought the maximum nodes in a cluster for HA was 16 while it is 32 when only using DRS?

No - the limit was increased in 3.5, but never appeared in the release notes - you had to check the 'Configuration Maximums' documents to spot it: 3.0.1/3.0.2 is here and 3.5 is here

[thomps01]

Just to be clear.

In my currenlty planned configuration I am adding 7 ESX hosts to each cluster.

My FC SAN has 2 storage processors and each of my ESX hosts has 2 HBA's giving a total of 4 paths from each host.

So the total number of paths to any LUN in this current configuration is 28?

A total of 8 LUNS are being presented to this cluster from the SAN and each ESX host will have access to them.

The number of hosts sharing a LUN = 7 (VMware's Max=32)

Number of paths to a LUN = 28 (VMware Max = 32)

If my understanding of the above is correct and I continue to present LUNS to all hosts in the cluster, once I add 1 more host, I've reached the 32 maximum paths, is this correct?

[Rodos]

Okay, it took a while but I got some firm confirmation of what the maximum limits mean, because they are a little ambitious.

Only active paths are counted, not passive ones. The 32 limit is per ESX host, not across the cluster (as you would hope and expect).

So in your example the paths to a LUN is based on the host not the cluster. Therefore if you have an active/passive SAN the paths to a LUN is one, if active/active its two.

Rodos

Consider the use of the helpful or correct buttons to award points. Blog: http://rodos.haywood.org/