Craig


vExpert 2009
Malaysia VMware Communities - http://www.malaysiavm.com

Duncan
VMware Communities User Moderator | VCP | VCDX

To be clear - the Symmetrix can absolutely be configured for large multi TB LUNs. What often happens is that the storage team has a "standard" way of configuring LUNs - and one operational model across mainframe and open systems.

This operational model is common in "Enterprise" Storage environments (and therefore shows up with Symmetrix), and less common in "Midrange" environments (that are usually opensystems only).

By open systems, Windows and VMware ESX are two examples.

So... in mainframe environments, generally the storage model are smaller LUNs, that use a LVM to stripe across the LUNs for more parallelism (which is often the source of these "small LUN" default configs from the SAN team).

Now - what does that architecturally look like? Multi-extent VMFS-3 configurations, of course!

There's not the downside that people normally think of ("no performance benefit" = FALSE; "higher risk" = MOSTLY FALSE). The "MOSTLY FALSE" on "higher risk" is basic. There is no higher risk for a "LUN being removed" than in non-multi-extent configurations in the sense that:

1. if you lose the first extent: i) in a single-extent datastores, you lose the datastore - some possibility of VM corruption; ii) in a multi-extent datastore, you lose the datastore - some possibility of VM corruption = SAME.
2. if you lose something other than the first extent: i) in a single-extent datastore, there is only one, ergo you will always lose the datastore, so same as 1; ii) in a multi-extent datastore, the vms on the lost extent will disappear - with some possibility of VM corruption = SAME
3. The only difference - it is possible to have a VM that spans two extents, and it would also disappear if half it's extent was lost. (this would only apply POSSIBLY to one more VM per extent (in otherwords, one, not several could cross an extent boundary).

Net/net? It's unlikely you'll be able to get your SAN team to change their standard. At EMC we are working agressively to integrate VMware/storage mgmt - with the long term goal of making the storage completely invisble (you just create the VMs, and the storage auto-configures).... so they better watch out - they better become more flexible, or they might become obsolete!

I would strongly consider using multiple extent VMFS configurations in your case. Also push your SAN team to use Virtual Provisioning - it's in Enginuity so if they are even CLOSE to current on their array software revs, they have it. If they don't tell them to GET WITH IT, and then just start using vSphere Thin provisioning

My 2 cents!

Chad

In practice - stay lower.

How much lower? Start by thinking about how many VMs per datastore you would like. This is a function of "eggs in one basket" vs. "lower number of datastores".

People get REALLY hung up on this, incorrectly.

Think of it this way - if you have 64 VMs per datastore (not uncommon in VMware View configurations, and how we've done it in the VMware/EMC reference architectures around View and View Composer), if you have 12 datastores, you're talking about 768 VMs per cluster, which is starting to approach "funky limits" of vCenter. Ergo, can you do datastores with hundreds of VMs - sure, but in practice, there's some - but limited upside. LIkewise, often people worry about managing "many datastores" - in the scenario above, we're talking about 12 datastores for an ESX cluster - not a big deal.

So, let's say you settle in on 30 VMs per datastore. If your average VM is 50GB in size, that means that you minimally need 1.5TB. add 25% for vswap, ESX snapshots and other functions. Can those other things conusme more than 25%? YES - but in practice, they are difficult to plan for, and it's easier to make some estimates and move on - remember, you can always svmotion yourself out of trouble, and the new managed datastore options and Storage View Reports in vCenter 4 can help alert and respond as needed. So, you have a datastore now that is 1.5TB + 375GB (25% of 1.5TB).

Then, you look at the storage design to support that. If you control the SAN (which you don't), you can create a single LUN (on most arrays this can be virtually provisioned, expanded, and meta-object linked together in a variety of ways). If you DON'T control the SAN, use multiple extents to get you where you need (in your case 8). The key is that since your SAN is a black box to you (you don't know how those LUNs are laid out), it's possible to "get 8 extents" that are all really on top of each other (as opposed across a wide set of spindles).

If your SAN team doesn't support thin provisioning, I would use the new thin provisioning functions in vSphere (existed to some degree in ESX 3.x, but the key was that thin (thin or zeroedthick) virtual disks always became thick (eagerzeroedthick) when you deployed from a template or cloned a VM (BTW, we're pushing VMware to fix this behavior in ESX 3.5, but we'll see how that goes)

Again - my suggestion - don't overthink. Instead, move forward, monitor, and know how to react. People are so used to the physical configs, where an error or mis-estimation can be fatal. In VMware land, you can respond non-disruptively under many, many (almost all) cases, so the real penalty is in either moving slowly, or over-sizing in fear.

So - take 8 LUNs, create a multi-extent VMFS, and move on. Monitor QUED (queue depth) on the LUNs using ESXtop periodically, and the vSCSI stats on an ongoing basis (easy to automate), and you'll see when QUED starts to get up to high values (16-32) for each LUN backing the multi-extent VMFS. That's your queue to start creating VMs in another datastore. If you see that happening very fast, ask the storage team to look at the "heat map" for those LUNs (which spindles are hot - serving lots of IO, and which are "cold" - serving few). chances are that the LUNs supporting the extents are all on top of one another, so very few actual spindles are being used. The SAN team has lots of tools to fix that non-disruptively.

This is why the whole storage world is moving to "wide striped" (create LUNs across loads of spindles), and "virutally provisioned" (only provision blocks actually used) configurations, and most arrays (certainly all EMC arrays) support this as an option.

As is SO often the case - the technology is moving faster than the people/organization/process.

Hope this helps! Covered this at length in the storage chapter in the upcoming Mastering vSphere 4 book by Scott Lowe (I wrote the storage chapter).