LUN and VMFS Datastore Limits?

[mrstorey]

Virtualcenter 2.0.1 patch2

ESX Server 3.0.1 Build 47426

Somebody tells me they are certain there is a limit of 256 LUNs and 256 datastores that can be used in any one Virtualcenter instance.

As far as I'm aware, the only limit I'm aware of is the number of LUNs per ESX host. This limit of 256 applies to the host only, although the limitation is that LUN ids must match among any set of clustered esx hosts. This person says that in virtualcenter, as soon as a limit 256 Luns or vmfs datastores is reached (whichever is reached first), further LUNs or datastores cannot be added into the Virtualcenter instance, regardless of which site or SAN hardware the LUNs are presented from.

This sounds wrong to me - but I can't find much material on the web which backs up either argument. Do any of you have experience with SAN LUN and vmfs datastore limits in large vmware environments?

For example, lets say we have two sites (Site1 and Site2), an esx cluster, storage hardware and san fabric on each, all managed by 1 Virtualcenter server.

Site1/Cluster1 has 250 luns and 250 vmfs datastores presented to all nodes, lunID's matching.

Site2/Cluster2 has 150 luns and 150 vmfs datastores presented to all nodes, lunID'd matching

That makes 400 Luns and 400 vmfs datastores in one Virtualcenter / vmware environment. This config will work....won't it?

[vmroyale]

No personal experience on a scale that large, but the "Configuration Maximums for VMware Infrastructure 3" at would seem to support your argument that it is 256 LUNs and-or datastores per host. There are no maximums listed regarding LUNs or datastores for VirtualCenter itself.

[mrstorey]

Thanks - yep I'm seen that. Like you say, I saw that the max luns an ESX host can see is 256 (128 during install), and we all know from experience that LUN id's must match across a cluster....but this hasn't been enough to convince people here that Virtualcenter has a limit of being able to address more than 256 LUNs or datastores!

The more I write and think about this, the more crazy it sounds to me. Can anybody tell me that this is wrong!?

[benny_hauk · Accepted solution]

What in the world makes them so sure that there is a limit since it doesn't sound like they've found documentation saying there IS one? Personally, if they are so sure there's a limit then they'd either better have heard it from VMWare support directly or have observed it themselves because there's no limit documented anywhere. If they say there is a hard limit then I think the burden of proof is on them, not you. I would buy the idea that 256 may be a limit for VC (if it gets confused by having multiple extents with the same vmhba:x:y:z name, for instance) but it doesn't seem to be in any of the documentation.

This won't help much, but here's what I found in the documentation (page 22 of: http://www.vmware.com/pdf/vi3_301_201_installation_guide.pdf):

"A dual‐processor VirtualCenter Server with 3GB RAM can scale to 50 concurrent client connections, 100 ESX Server hosts, and over 2000 virtual machines."

It seems their point here is that the major limitation when it comes to virtualcenter server is the horsepower of the server and maybe just good comon sense. Seriously though, virtualcenter server can scale up to 100 ESX hosts and each host can access 256 LUNs. On my calculator that comes out to a theoretical limit of 25,600 LUNs that a single VirtualCenter Server could see. Worse case: virtualcenter doesn't see all the LUNs and so when you manage LUNs you connect your VI client directly to an ESX server, otherwise you connect like normal to VI server.

Also, page 4 of the Configuration Maximums for VMware Infrastructure 3 makes no mention of a LUN or VMFS-3 datastore limit for the VirtualCenter server itself. The point of that whole document is to point out all maximums through all their VI3 product line and the portion on VirtualCenter Server makes no mention of a LUN-limit.

Probably not as definitive as you are looking for, but if you really needed a definitive answer that will silence your colleagues, then their are probably better ways than using the forums which I doubt will carry much weight with them. If you are that large of an operation I'm sure your sales/tech support team from VMWare will be happy to track down a definitive answer for you- ... I mean for your disbelieving nay-sayers.

One thought: Don't lose too much sleep over it. The worse that can happen is that in the interest of moving your project along you admit there theoretically could be an undocumented ceiling on the number of LUNs virtualcenter will recognize and that their guess that the elusive number is probably 256 could be correct (as correct as any other number over 256 that they pull out of thin air) so you eat a virtual machine to add another virtualcenter server and you eat another VC server license. In a project as large as you are describing I don't think that additional cost will get much scrutiny. Better yet, make the nay-sayers pay for the extra cost and put their money where their mouths (and crystal balls) are.

[mrstorey]

Thanks for your detailed reply - it's pretty much what I thought.

We do have support, but it's provided via a third party and answers do not always come from vmware themselves. I've always found the forums to provide quick responses from other sysadmins - I was very interested to hear your opinions!

Thanks for your help