VMware Cloud Community
jstraten
Contributor
Contributor
Jump to solution

2x RAID5 vs 1x RAID10

In my system I can either way have two RAID5 arrays or one RAID10 array. Now, my questions are:

1. What is the better option for a VMware ESXi environment?

2. What gives me better performance? I mean I do understand that RAID10 writing is faster than RAID5, but is this statement still true if I am writing against two different RAID5 arrays?

3. How about data safety? From what I can see there are no differences between RAID5 and RAID10. Is that correct?

Thanks,

Jens

Reply
0 Kudos
27 Replies
jstraten
Contributor
Contributor
Jump to solution


So, I am still wondering: Are two RAID5 faster than one RAID5?


Probably but only marginally and only IF you have 2 separate controllers. If you put 2 RAID 5 on the controller that means that one controller controls BOTH RAID groups. So for this it would be slower, if you can stick 2 RAID controllers each managing the RAID 5 separately it might be a little faster, but I doubt it. 2 RAID 5 means that EACH array is still a RAID 5, RAID 10 is the fastest you can get, period (other than RAID 0). This assumes the same number of spindles.

RAID 5 with 6 spindles, 2 RAID 5 with 3 spindles each, or a RAID 10 on 6 spindles. RAID 10 beats RAID 5, RAID 50.

Thanks! This is exactly the information I was looking for.

Basically, in my scenario with one RAID controller, I wouldn't see any performance improvement by having a second RAID array.

It seems that the best of both worlds would be two use RAID50.

How much faster is RAID10 in comparison to RAID50 with a six drive array?

Reply
0 Kudos
golddiggie
Champion
Champion
Jump to solution

This should be required reading...

http://www.adaptec.com/en-US/_common/compatibility/_education/RAID_level_compar_wp.htm

VMware VCP4

Consider awarding points for "helpful" and/or "correct" answers.

Reply
0 Kudos
J1mbo
Virtuoso
Virtuoso
Jump to solution

I'll put some numbers on this. For virtualisation it is random IO that is the metric of interest usually. Sequential MB/s numbers which will be huge with any of these configurations.

My own testing has shown that quality SATA drives like Western Digital's RE series will turn in about 140 IOPS within a reasonably small test space; I tend to test with about 8GB test files.

RAID 10

- 6x 140 IOPS read, 6/2 x 140 IOPS write

- Arbitary 70:30 read:write split should yield about 700 IOPS

RAID 5

- 5x 140 IOPS read, 5/4 x 140 IOPS write

- 70:30 roughly 525 IOPS

- Note writes incur a full additional RPM of latency however due to read-update-write

RAID 6

- 4x 140 IOPS read, 4/6 x 140 IOPS write

- 70:30 roughly 420 IOPS

- As with RAID-5 writes incur a full additional RPM of latency however due to read-update-write

RAID-50

- Probably not a good choice as each "side" of the stripe is only 3-drives, causing a write to absorb all three disks (controllers tend to read-peers then update data + parity with 3-drive sets to save processing overhead of calculating parity twice)

- 4x 140 IOPS read, 4/4 x 140 IOPS write

- 70:30 roughly 435 IOPS

- Write latency per RAID 5 & 6

However all of this might be irrelevant because many array controllers will present only ONE LUN with RAID-x0 configuration, which will obviously be way too big for ESXi which needs each LUN to be just under 2 TB as stated above already. If this controller has this limitation, this immediately forces the configuration to RAID-5 or RAID-6 (which should allow multiple 2 TB LUNSs to be defined across the whole array).

Raid-5 will perform slightly better and give more space. However with 2 TB SATA drives the prosepct of a second failure during a rebuild is a very real prospect, resulting in total data loss, due to the read error rates being not massively higher than the overall capacity. Then again it's a dev environment anyway so speed might be more important.

Raid-6 offers security in the rebuild process but is slightly slower and costs you another disk of capacity.

The battery is essential because ESX will use ONLY write-through caching, as it needs to guarantee to VMs that writes are completed when they think they are. Hence hardware write-back caching is used to hide, to an extent, the effect of the read-update-write process for writes.

HTH Smiley Happy

Please award points to any useful answer.

Reply
0 Kudos
jstraten
Contributor
Contributor
Jump to solution

I'll put some numbers on this. For virtualisation it is random IO that is the metric of interest usually. Sequential MB/s numbers which will be huge with any of these configurations.

My own testing has shown that quality SATA drives like Western Digital's RE series will turn in about 140 IOPS within a reasonably small test space; I tend to test with about 8GB test files.

RAID 10

  • 6x 140 IOPS read, 6/2 x 140 IOPS write

  • Arbitary 70:30 read:write split should yield about 700 IOPS

RAID 5

  • 5x 140 IOPS read, 5/4 x 140 IOPS write

  • 70:30 roughly 525 IOPS

  • Note writes incur a full additional RPM of latency however due to read-update-write

RAID 6

  • 4x 140 IOPS read, 4/6 x 140 IOPS write

  • 70:30 roughly 420 IOPS

  • As with RAID-5 writes incur a full additional RPM of latency however due to read-update-write

RAID-50

  • Probably not a good choice as each "side" of the stripe is only 3-drives, causing a write to absorb all three disks (controllers tend to read-peers then update data + parity with 3-drive sets to save processing overhead of calculating parity twice)

  • 4x 140 IOPS read, 4/4 x 140 IOPS write

  • 70:30 roughly 435 IOPS

  • Write latency per RAID 5 & 6

However all of this might be irrelevant because many array controllers will present only ONE LUN with RAID-x0 configuration, which will obviously be way too big for ESXi which needs each LUN to be just under 2 TB as stated above already. If this controller has this limitation, this immediately forces the configuration to RAID-5 or RAID-6 (which should allow multiple 2 TB LUNSs to be defined across the whole array).

Raid-5 will perform slightly better and give more space. However with 2 TB SATA drives the prosepct of a second failure during a rebuild is a very real prospect, resulting in total data loss, due to the read error rates being not massively higher than the overall capacity. Then again it's a dev environment anyway so speed might be more important.

Raid-6 offers security in the rebuild process but is slightly slower and costs you another disk of capacity.

The battery is essential because ESX will use ONLY write-through caching, as it needs to guarantee to VMs that writes are completed when they think they are. Hence hardware write-back caching is used to hide, to an extent, the effect of the read-update-write process for writes.

HTH Smiley Happy

Please award points to any useful answer.

</div>

The Adaptec 5805 can actuaully build Multi LUNs for RAID10. I haven't tried it for RAID50 yet, but I am guessing that it will work as well.

So, you would recommend to use RAID6 in my scenario if I understand you correctly?

Reply
0 Kudos
J1mbo
Virtuoso
Virtuoso
Jump to solution

If you can afford the space, use RAID-10 and create 3x 2TB LUNs. If you need more space, yes, I'd go for RAID-6 subject to your benchmarks showing sequential write performance is sufficient for your needs with this particular card.

Please award points to any useful answer.

Reply
0 Kudos
golddiggie
Champion
Champion
Jump to solution

From the information provided on the Adaptec site:

Combines multiple RAID 5 sets with RAID 0 (striping). Striping helps

to increase capacity and performance without adding disks to each RAID 5

array (which will decrease data availability and could impact

performance when running in a degraded mode).

RAID 50 comprises RAID 0 striping across lower-level RAID 5 arrays.

The benefits of RAID 5 are gained while the spanned RAID 0 allows the

incorporation of many more disks into a single logical drive. Up to one

drive in each sub-array may fail without loss of data. Also, rebuild

times are substantially less then a single large RAID 5 array.

Usable capacity of RAID 50 is between 67% - 94%, depending on the

number of data drives in the RAID set.

Each RAID 5 array has the striping it contains, then the two are striped together, to increase performance to ~6 drives (if using 😎 or 4 drives (if using only 6). I do see RAID 5/50 losing out if you're going to go to a max of six drives. BUT, if you go to the full eight that the card supports (and the chassis too) then you'll gain there.

I would stay clear of using SATA drives no matter what RAID level you eventually end up with. I think you'll be many times better off using SAS drives, even if they are 7200 rpm models. You can get those in sizes up to 1TB. RAID 10 would be my initial choice of an array as well, but if you cannot afford the cost of drive parity, then I see RAID 50 as the next best thing...

BTW, RAID 10 and 50 perform alike when running normally. RAID 10 will perform better when the array is in a degraded state (when you have drive failures). For that reason, I would always have at least one, or two, 'spare' drives on hand ready to swap in during such a time. You hope you never need to use one of them, but you're prepared in case you do.

VMware VCP4

Consider awarding points for "helpful" and/or "correct" answers.

Reply
0 Kudos
jstraten
Contributor
Contributor
Jump to solution

Thanks Everybody!

I think I will just go with RAID10 for the time being. It seems to be the best option from a performance point of view while also providing me with some safety by mirroring the drives.

Reply
0 Kudos
joshuatownsend
Enthusiast
Enthusiast
Jump to solution

J1mbo provided some great info. I have a series of posts on my site that explain the math a bit more: http://vmtoday.com/2009/12/storage-basics-part-i-intro/


If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".

Please visit http://vmtoday.com for News, Views and Virtualization How-To's

Follow me on Twitter - @joshuatownsend

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful". Please visit http://vmtoday.com for News, Views and Virtualization How-To's Follow me on Twitter - @joshuatownsend
Reply
0 Kudos