RAID configuration

[CharlesWhite]

We are doing a base install of ESX 3 on a Dell dual quad-core PowerEdge 2950 with 6 - 400 Gig SAS drives. I am expecting to have a variety of different machines that I might run from this physical machine. I am curious what reccomendations for RAID arrays the group might recommend. I'm concerned with maxmizing performance while providing redundancy. I considered 3 RAID 1 Mirrors and then I could isolate to different RAID sets / physical disks without incurring the write penalty for the pariy information of RAID 5.

Thanks,

Charles

[Chris_S_UK]

How much space will you need for your VMFS partition? And do you mind having multiple VMFS partitions (assuming you don't use Extents)?

Compared with 3 x RAID1 mirrors, a single RAID5 array will give you 800GB more usable space. And do you have an idea whether your VMs will be read intensive or write intensive primarily?

In my experience, although RAID5 might not in theory give the best performance, you are unlikely to notice the difference between this setup and a more elaborate/optimised one unless you are really pounding the disks....especially if the disks are 15K ones.

A "simple" single RAID5 array will minimise 'lost' space too, with only 400gb being 'lost'. But if you can afford to lose more space, then what you propose could work well.

Chris

Message was edited by: Chris_S_UK

[Texiwill]

Hello,

I normally split the OS and VMFS raids apart.... However, you should account for the # of VMs per LUN. specifically the general recomendation is 12-15 VMs. How much space do 12-15VMs require? I would give the OS at least RAID1, That is 400GBs and the OS will take at most 100GB of that leaving 300GB for a VMFS. Why RAID1? Mainly so you can 'bag and tag' the mirror if there is a break in, or for pretty catastrophic results. The leave the other 4 to split up into either a single RAID5 with 1.5TB of space or into 2 RAID1 each with only 400GBs of space.... Do some math on the size and number of VMs and see which one you need to use.

Best regards,

Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education. Available on Rough Cuts at http://safari.informit.com/9780132302074

[rswitzer]

If you have 6 - 400 gig drives, why not just do RAID 10 for all disks? Is there a reason to seperate the OS and VM on different RAID setups?

[Texiwill]

Hello,

No reason RAID 10 can not be used, however I rather use RAID1 for just the OS drives and RAID5 for all VMFS. You could use RAID 50 with 6 drives as well but if you loose one drive you are in a failure mode for the RAID5 aspect. I think a split of 2 and 4 is slightly better and more resilient.

The split of the disks is generally dependent on your needs.

Best regards,

Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education. Available on Rough Cuts at http://safari.informit.com/9780132302074

[CharlesWhite]

No other than not knowing the best practice, or the most common practices. Also trying to be able to isolate disk performance issues even though i have only a single controller. Wanted to know that operation of the virtual machines wouldn't impact disk performance on a specific virtual machine if I kept that single machine assigned to a single raid set. Granted the controller is potentially a limiting factor since the controller will be handling all RAID sets managed. So I might have 3-5 different application type servers one storage group on the raid set of the consoles partitions and have another "dedicated" raid set that would be used only by a set of specific virtual machines that I intend to only run one at a time (development database servers for different applications and processes).

Also I am sort of looking for real world experiences of simple 2 drive RAID-1's v/s 3 drive RAID-5's for moderately intensive short / small database writes. Most of the information I see on the interenet continues to indicate RAID 1 over RAID 5.

Thanks,

Charles

[Texiwill]

Hello,

WIth a local controller most of the IO issues are really gone as its pretty much as fast as it can get. RAID5 is slower than RAID1 but gives better failure capabilities. In RAID 1 if a drive fails your Raid set is trashed and you are stuck using the mirror, at least on RAID5 a drive can fail and you can keep going without much in the way of issues. RAID 1 in all literature will say it is faster. But you do you loose failure modes.

The RAID level you use will depend on your feelings about MTBF on drives. As I stated, I use RAID 5 for all VM data and limit my LUNs to 12-15 VMs per LUN (granted I am using a SAN so for local SCSI that can change to be higher). I would never put just one or two VMs on any given LUN. Nor would I waste so much space making 3 RAID 1 partitions. You can get away with all RAID 1, all RAID 5 with a spare, 2 RAID 1 and 4 RAID 5, etc. There are many options available.

You really want to consider failed drives and how to recover from them. In this RAID 5 with a spare is the best option, that way if a drive fails your spare kicks in and rebuilds the array, then you have to worry about replacing just the spare. Performance on one hand, redundancy on the other, it is always a tough choice. For VM data I tend to fall on the side of redundancy and failure recovery modes.

Best regards,

Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education. Available on Rough Cuts at http://safari.informit.com/9780132302074