1TB SAS 7.2K vs 1TB SATA 7.2K (DELL MD3000i)

[babyorange]

Hi guys,

I'm new to the SAN thing and is hoping someone can shed some light.

I'm toying between using 1TB SAS or 1TB SATA disks for my new DELL MD3000i SAN. Both the disks run on 7.2K RPM. It will be connected to ESX and run "normal" VMs such as Windows File/Print, Exchange, SQL, Linux, etc. Nothing extremely heavy duty.

A. Do you have any inputs on how the performance, reliability, etc would be like between these two types of disks - SAS 7.2K vs SATA 7.2K?

B. With 14 x 1TB 7.2K disks, should I be running as RAID10 or RAID5 to maximise the performance? IIRC, 14 disks in RAID5 will have throughput of 14 spindles, whereas 14 disks in RAID10 will only have throughout of 7 spindles. Capacity is not an issue at this point in time.

C. On a related note, how about 14 x 450GB SAS 15K disks? Will running them in RAID10 be a complete overkill? Or would RAID5 be the sweet spot considering that the disks are already 15K RPMs?

Many thanks!

[AsherN]

It's always more spindles. 7.2K SATA vs SAS comes down to reliability. I would not fret over it. 7.2K vs 15K, the faster drive would theoritically have the advamtage, but I'm not totally convinced that a real world normal mix of VMs would see that much of a difference.

As for RAID levels, it depends on what is done to your data. Most higher end controllers will pretty much take away the write penalty on any RAID level. RAID 10, being mirrored can potentially retrieve data faster. Conventional thinking has always been that in read intensive applications, go mirror. If capacity is not an issue, I'd go RAID 10.

[babyorange]

Hi AsherN,

Thanks for the inputs.

Am I correct to say then that SAS and SATA performance is about the same, only with SAS being more reliable?

While I believe a 26 disk RAID10 (13 spindles) will beat a 14 disk RAID5 (13 spindles) face down, but if comparing between having the same 14 disk in RAID5 vs in RAID10, the number of effective spindles will be 13 (as RAID 5 is N+1?) vs 7 (2 pairs).

Would the RAID10 in this case have poorer performance compared to the RAID5? And will this impact be more pronounced when using 7.2K disks vs 10K or even 15K disks?

Thanks and best regards,

[AsherN]

The rotational speed will have an impact. How great is dictated by the way the data is accessed.

RAID 10 will always beat RAID 5. Today's enterprise RAID controllers usually have enough cache to eliminate the write penalties of RAIDed arays. RAID 10 will distribute your data across spindles as much as RAID 5 will. OTOH, when it comes to reading the data, a RAID 10 array has 2 copies to chose from. In theory, you could be serving 2 read requests simultaneously on a RAID 10 array.

SAS drives are supposed to be more reliable. Look at the MTBF of both drives and ask yourself if it is a real world significant difference. Given that the data is under some protection. I always like to configure a hot spare, and keep an extra drive on a shelf. That way, I'm never waiting for a replacement part to be fully protected.

[AndreTheGiant]

For max performance you have to use always RAID10.

RAID5 with so much disk can be slow during write operation.

Remenber also that if you MD3000i box has 2 controller may be a better solution to create 2 different pool, each with a different controller.

Andrea

[babyorange]

Thanks AndreaMauro & AsherN for all the inputs.

I've been looking at hard disk specs and taking Seagate for example :

The MTBF and other performance characteristics for their 1TB SAS (ST31000640SS) & SATA (ST31000340NS) are almost identical (the SATA model had 32MB cache vs SAS's 16MB), seems like the only difference is the interface. In this instance, because of the additional cache, SATA would be a better buy than SAS?

Am I reading the info correctly or there is something more to the SAS vs SATA?

Regards,

[AndreTheGiant]

Am I reading the info correctly or there is something more to the SAS vs SATA?

Very large SAS disk are not common and probably are very similar to SATA disk.

But if you compare a 300 GB 15k SAS disk with a similar (size) SATA disk you will notice sever difference (first to all the rotation speed).

Andrea

**if you found this or any other answer useful please consider allocating points for helpful or correct answers

[AsherN]

The faster spinnig drives will always win, unless you have more spindles of slower drives. More spindles mean that you are playing the odds that the next data block will be accessible on now on a different spindle as opposed to waiting for the platter to come around.

In theory, assuming a stripped RAID array, twice as many 7.2K should deliver performance roughly equivalent to 15K.

At the end of the day, it's down to 'I can give you capacity, low cost and performance, pick any 2'.

[Lightbulb]

A lot of good stuff has already been said. you did mention Exchange & SQL VMs, these tend to have high I/O utilization . On SQL the Data transaction log and tempdb should be on seperate vmdks and RAID 10 if you can afford it. On the exhange server the mailbox stores and logs should be on separate vmdks (Potentially you would also want the SMTP queue on its own disk depending on usage)

I am not real familiar with the MD3000i if you can present a RAID5 and RAID 10 volume to the hosts this might be best. RAID 5 for the low I/O VMs and you could use the space on RAID 10 for Exchange and SQL high I/O file systems.

[pdamad · Accepted solution]

My understanding is that the SAS interfaced drive will be 10 -15 % faster for random io even though the rotational speed 7.2k is the same. This is the difference capability between the fairly simple SATA controller embedded on the drive and the more advanced dual ported SAS controller. So all things being equal the SAS interfaced drive will be a better choice for general use. For archival purposes for example it would make little difference. As you point out the MTBF figures are similar, the drives are probably identical other than their controller.

As far as raid levels are concerned my view is that one of the deciding factors for raid 10 is the problems that can occur with a raid 5 set is the rebuild time and performance hit.

If you have many vm's running in production and you have a raid 5 rebuild during a business day, is it acceptable to have many of your vm running very slowly!

If it is not then use raid 10 and get better perfomance as well.

[nickchap]

We're running a Dell MD3000i with 14 x 450GB 15K SAS disks in RAID5. We have 3 hosts running ESX3.5 and we're running a similar mix of applications to yours. We don't have any performance problems, but I've not run any benchmarks so can't make any comparisons with other RAID configs or SATA drives. We've been running this for 7 months with no issues.

Nick

[JohnADCO]

The high rpm sas drives are pretty impressive in an MD3000i, we run hard production on them. I've not had any experience with 7.2K SAS drives. Our MD3000i's with 7.2K SATA drives are not so great, I'd lable their performance as adequate at best.

[babyorange]

Hi Nickchap/JohnADCO,

Regarding the MD3000i, I read that it has 2 controllers, each with 2 ports, for a total of 4 NICs in total. Does you model come with dual controllers as well? If so, are you able to make full use of the 4 NICs or is traffic limited to just a subset, like mentioned in other MD3000i/iSCSI threads in this forum?

[AndreTheGiant]

To make (theorically) use of all 4 NICs you need to have:

• Full network topology (like a fabric FC)

• almost 4 LUNs (Volumes), half owned by a controller, and half by the second one

• verify that each datastore use a different path (in multipath)

This is the only way that I know with this kind of storage. No link aggregation, no trunk, just multipath

Andre

**if you found this or any other answer useful please consider allocating points for helpful or correct answers

[JohnADCO]

ESXi here, and we use the EXX software initiators. Limits us to one active path at a time.

We have done some testing and using MS Intiators in the VM itself seems to give link aggragration / load type balancing. But we never put in production that way. In production, the esxi software initiator really seem more than adaquate for us.

[babyorange]

Hi JohnADCO,

How is the loading on your path like? Would using only one activepath put the other 3 paths to waste (assuming you are using dual controllers)?

[JohnADCO]

It's the reason I only use 2 nics for iSCSI on ESXi. Two dual controllers in total on our hosts. I currently have one nic connection that is not doing anything on my hosts. I devote two to iSCSI and one to management / lan connectivity for the host.

With two gig switches on different subnets for your iSCSI network, you still get your 4 failover paths from it. On the one nic connection you still get to see how the controller traffic is split up between the dual controllers, by virtual disk ownership.

[MichaelW007]

You can multipath just fine with ESX's Software iSCSI initiator and get full use of both NIC's on the ESX hosts. Also re the performance of the different types of disk, SAS vs SATA. The single biggest different in the interface is that SAS can do a read and write operation concurrently, and SATA can only do a read OR a write operation. So you can get almost double the performance out of SAS compared to SATA in some cases. This makes a big difference in environments with random IO, i.e. every VMware environment that has more than 1 VM per datastore.

The most likely workload profile in a VMware environment is 100% random IO and normally write biased. The write penalties for RAID5 are still there (although small), especially if you have enough IO going on to fill up the write cache (definite for SAS 7..2k or SATA). RAID 10 will outperform RAID5, especially with read operations, but it will massively outperform RAID5 when there is a failure of a disk.Then you have to consider rebuild times and the likelihood of another disk failing during the rebuild process, which will kill your entire disk subsystem (hence why RAID 6 was invented). Then if you go RAID6 you will have double the parity being written and this will have an additional penalty for write operations.

I'm working in an environment right now with SAS 7.2k RPM 500GB disks (16 spindles per RAID group) in RAID5 configuration and because of the random IO on the ESX Hosts (2 of them in the cluster) the latency is just terrible, averaging above 100ms. Reads are particularly bad, as the spindles have to spin around to pick up the data and they just can't do that fast enough. It's basically at the point where the customer may have to think about throwing out all the 7.2k SAS disks and replacing them with 12k disks at a fairly big expense. Exchange and SQL are especially latency sensitive applications. If you're getting 100ms disk latency on SQL you can forget about getting 40 or 50ms latency on transaction times.

If you were thinking of going SAS 7.2k RAID10 (which would be the only sensible option for performance), then you'd actually be better off going SAS15k with RAID5. SAS15k with RAID5 and have two RAID groups on the MD array (each RAID5 7+1).

Based on experience I've had in production environments of organisations large and small I'd stay well clear of SATA or SAS 7.2k for any random IO workloads. They are fine for sequential read and write, which makes them great for archive and backup type applications. But as soon as you try any random IO you're performance is dead in the water.