Need Assistance with LUN configuration with EMC SAN

[greggap]

Hi All,

I have a few questions regarding SAN best practices and need some recommendations. We currently have a EMC SAN CX3-20. I am

no SAN expert but I am building out a VMware environment. I have 30 146GB 15K RPM drives to build our datastores with. Here

are my questions..

How many luns should I build with these drives, I don't expect anything with extremely high I/O, should I go with RAID5? How many drives

should I include in each LUN? If I have a raid set with say 4 drives versus one with 8 what percentage performance difference should I see?

Can someone explain to me the difference between LUNS and Raid Groups on the EMC side, should I assign the luns equally to each Storage processor or does that not matter? How do I guarantee the host will use all available HBA paths to the storage?

Looking for some guidance, sorry for all the questions, thanks for all the help/suggestions.

Greg

[chilow]

Raid groups are containers you place disk drives in, so if you have a RAID group named "RAID_Group_100" you can place x amount of physical drives in the group; number of drives will depend on your desired RAID type. You can create multiple LUNs inside a RAID group, so in RAID group named "RAID_Group_100" you can carve out LUNs up to the total usable capacity within the RAID group.

When you say 30 disk drives I assume you are not including the vault (flare) drives and hot spares. I will assume you have 30 data disk drives available. In order to maximize your IO throughput at the disk level you might want to consider using striped meta LUNs. Using RAID5 (4+1) you could build 6 RAID groups which will allow you to build a 6 meta member LUN; make sure to spread out the meta heads across RAID groups. You don't want to have large RAID5 groups, because the larger the group the longer the rebuild will take if a drive failure occurs.

So you could create (6) RAID5 (4+1) RAID groups. After these are built you can start creating equal LUNs across the 6 RAID groups and then form a single striped meta LUN.

Remember to place each ESX server in its own storage group and then place each shared LUN in each ESX server storage group.

Since you are using an active/passive array each LUN can only be active on one SP at a time.

To maximize your frontend throughput you should considering cabling up multiple ports on SPA and SPB. Assuming you have two HBAs in each ESX server you could zone each HBA to multiple SP ports. Example..HBA1 would be zoned to SPA0 and SPA1 and HBA2 would be zoned to SPB0 and SPB1. You could add additional frontend ports as needed, but I would start with 2 per SP incase you have a component failure on a SPA or an entire SP failure. For the ESX load balancing policy you are going to use MRU, unless you want to pay for VMware's ESX Enterprise Plus licensing and EMC's PowerPath VE licensing; both products licensed per physical CPU socket. If MRU is used you can manually set primary paths per ESX server across the primary SP ports.

Hope this helps.

Here is a good link for some EMC VMware info..http://virtualgeek.typepad.com/virtual_geek/2009/09/updated-clariionvmware-applied-technology-guide-and-other-key-docs.html

-C

[greggap]

Great, thanks for the advise.. Can you clarify two things?

make sure to spread out the meta heads across RAID groups? What do you mean by this? Also I am reading about metaluns creating some contention issues and load on the SP, how will I avoid this?

Thanks,

Greg

[joshp]

Based on the information you presented in your question I would configure the CX in the following way (this is a general config without knowing your IO requirements):

1. Create four (4) Raid Groups each containing single 6 disk RAID 5 LUN (5+1). This would give you 4 x 730GB LUNS which would translate to four independent datastores on your ESX host. Make sure two LUNS are using SPA and two are using SPB (you want to spread the ownership across Storage Processors in the CX. Design note: you typically don't want to bind multiple LUNS to a single RAID Group. The best rule is to create a 1-to-1 with each RAID Group containing a single LUN.

2. Assign 1 disk as a hotspare. You should have one hotspare for every 30 physical disks in your EMC array.

3. The remaining 5 disk probably contain the FLARE OS (because you didn't say I am assuming your have a CX with 1 DAE) so don't put anything on them. If the 5 disks do not contain the FLARE OS you can create a four disk RAID 10 or a smaller 5 disk RAID 5 (4+1).

Based on some of your questions I would NOT recommend you build metalun(s) across the 30 disks. Metaluns do provide increase performance in some scenarios but they greatly increase the complexity of your storage configuration and also increase the difficulty in troubleshooting IO related issues.

VCP 3, 4

www.vstable.com

[chilow]

The first device/LUN that is part of a meta is the meta head. So when you build metas you want to spread out the heads across the RAID groups. Example..if you have 6 RAID groups...

RG_10 >>5 disk drives RAID5 (4+1)>> LUNs 10,11,12

RG_20 >>5 disk drives RAID5 (4+1)>> LUNs 20,21,22

RG_30 >>5 disk drives RAID5 (4+1)>> LUNs 30,31,32

RG_40 >>5 disk drives RAID5 (4+1)>> LUNs 40,41,42

RG_50 >>5 disk drives RAID5 (4+1)>> LUNs 50,51,52

RG_60 >>5 disk drives RAID5 (4+1)>> LUNs 60,61,62

When building metas with the LUNs above....

LUN10 would be combined with LUNs 20,30,40,50,60....meta head would be on RG_10

LUN21 would be combined with LUNs 31,41,51,61,11....meta head would be on RG_20

LUN32 would be combined with LUNs 42,52,62,12,22....meta head would be on RG_30

And so on....

There is a performance hit when building and expanding striped metas. With LUNs/meta LUNs you always want to load balance your SPs by assigning an equal amount of LUNs/meta LUNs per SP assuming your IOPS/throughput requirements per LUN are equal. We have 10 Clariions configured this way and haven’t had many performance issues. As your environment grows you will always run into bottlenecks. You may need to add more spindles, front end/backend ports, and cache to fix bottlenecks, but this is just part of life.

Check out this Clariion blog that was put in place by a EMC instructor that teaches a Clarriion performance class http://clariionblogs.blogspot.com/2008/02/metaluns.html . He hasn’t updated it in a while, but there is some good information on the site. Also make sure you sign up for a EMC Powerlink account @ https://powerlink.emc.com and download the EMC Clarriion Best Practices for Performance and Availability based on your flare code.

I am not saying meta LUNs are the way to go in every environment, it is just another option.

Thanks

-C