VMware Cloud Community
Phatsta
Enthusiast
Enthusiast

Planning a new storage solution

I've got some budget funds to spend, and would really like to upgrade in terms of storage. Today I'm using 3 HP servers with 12 virtual Machines stored in local raid arrays, and backup to a QNAP NAS with gigabit copper connection. It's okay in terms of speed, but I'd like to go beyond okay. I've been looking at building an iSCSI environment to speed up the virtual machine datastore(s), and get a faster backup unit as well. The QNAP performs at about 20MB/s and I'd really like to at least triple that speed.

So first of all a new storage solution and second a better backup solution.

If you had about $5-6000, what would you spend it on? I'd really like to hear!

Tags (3)
30 Replies
vmroyale
Immortal
Immortal

Note: Discussion successfully moved from vSphere Storage Appliance to VMware vSphere™ Storage

Brian Atkinson | vExpert | VMTN Moderator | Author of "VCP5-DCV VMware Certified Professional-Data Center Virtualization on vSphere 5.5 Study Guide: VCP-550" | @vmroyale | http://vmroyale.com
0 Kudos
Atomliu
Enthusiast
Enthusiast

Maybe it is a good solution to build a high available iSCSI  SAN .Check here :KernSafe High-Availability iSCSI SAN for VMWare ESX / ESXi Server

The Enterprise version just needs about $900, so in total it is a perfect solution with $1800

Moderation node: Atomliu is a KernSafe employee. Keep that in mind when making a decision.

Message was edited by: schepp

jwhitehv
Enthusiast
Enthusiast

Wait. The local storage is going to be faster than storage that hits a network. The PCI bus is faster than 10GbE.

Do you mean you'd put a faster array at the other end of the iSCSI path? If you just keep that array local, then you'll have the best of both worlds. Or perhaps you mean you don't want to upgrade the local storage of all three hosts?    

I blog at vJourneyman | http://vjourneyman.com/
0 Kudos
Phatsta
Enthusiast
Enthusiast

Yes, that would be true if the local storage were all hardware raid and running a raid 5 or faster, I guess. But one of the servers are only using single disks since it only supports software raid, and another server has slower SATA drives in a raid 5 array so it's not very fast either. The last server has a raid 5 SAS Array which probably would be faster than an IP SAN.

But it's not the only reason for getting one. Another reason is availability. If all VM's reside on an external storage, it's easy just to add it to any hardware in case of "emergency" (or maintenance). I'd rather depend on a high end SAN than on backups made nightly to a low end NAS Smiley Happy

Also backups will be faster than today since the network speed multiplies by 10.

0 Kudos
Phatsta
Enthusiast
Enthusiast

Atomliu; I'm definitely checking that one out, thanks for the tip!

0 Kudos
Atomliu
Enthusiast
Enthusiast

Hope it can help you!

0 Kudos
jwhitehv
Enthusiast
Enthusiast

Phatsta wrote:

Yes, that would be true if the local storage were all hardware raid and running a raid 5 or faster, I guess. But one of the servers are only using single disks since it only supports software raid, and another server has slower SATA drives in a raid 5 array so it's not very fast either. The last server has a raid 5 SAS Array which probably would be faster than an IP SAN.

Oh, in that case, I'd spend my $5-6K budget on a new server with good local storage controllers and SAS drives. In RAID 10, not parity RAID configurations, assuming you have non-trivial write IOPS requirements. You'd pay for on-site support for a single machine instead of 3, which might allow you to budget for a higher service level (4 hour response or 6 hours to resolution).

You could keep around one of the servers and upgrade the local storage controller/drives to act as a replication target.

But it's not the only reason for getting one. Another reason is availability. If all VM's reside on an external storage, it's easy just to add it to any hardware in case of "emergency" (or maintenance). I'd rather depend on a high end SAN than on backups made nightly to a low end NAS Smiley Happy

I don't think that actually increases your availability. If all your VMs reside on external storage and the external storage goes down, your entire environment is down. You still have one device can fail and bring down your environment. For 12 guests (or is it 12 guests times 3 hosts for 36 guests total?), your baseline config would be a single host with good backups and a good on-site service contract. Stepping up would mean a second host as a replication target. If you go to external storage, you need the kind of multi-chassis device (which would have two complete copies of your data) which is outside of a $6K budget.

I blog at vJourneyman | http://vjourneyman.com/
0 Kudos
jwhitehv
Enthusiast
Enthusiast

Atomliu wrote:

Maybe it is a good solution to build a high available iSCSI  SAN .Check here :KernSafe High-Availability iSCSI SAN for VMWare ESX / ESXi Server

The Enterprise version just needs about $900, so in total it is a perfect solution with $1800

Isn't this a solution built on a Windows operating system? If you're going to buy a new host with this kind of local storage, I think using it as a vSphere host is the best bet.

I blog at vJourneyman | http://vjourneyman.com/
0 Kudos
Anton_Kolomyeyt
Hot Shot
Hot Shot

That's a BAD idea Smiley Sad I'd look strictly @ VMware VSA on your place. And here's why. When it comes to storage there are few things you need to take a look at initially. So, one-by-one:

1) Support. VMware VSA is coming directly from a hypervisor vendor. It's a) supported exceptionally well by VMware staff b) quite popular among the community c) certified and d) allows to avoid fingerpointing phase

as all the components are coming from the single vendor. When you look at the stuff that guy had pointed at you'll see it's a) badly supported (dead forum, one-man-company, mainland China so no English

and no business hours because of the time zone etc) b) has virtually zero people using it so you'll be on your own c) not certified and d) also with c) VMware will simply refuse to support you becase of the

incompatible storage. Keeping in mind a) and b) you'll be the only guy on Earth who'd care about your datastore. VMware is a definite winner here Smiley Happy

2) Price. $1,800 they want from you is just a part of the story. Now it comes to at least a pair for a servers (physical ones) and a pair of Windows server licenses. So you can add easily $2,000 - $4,000 to their fee. This is

more expensive then VMware VSA and VMware Essentials Plus (where you have other nice features these guys can only dream about) being both around $4,000 (or less if you have a good VAR with a good discounts).

And don't forget corresponding network infrastructure like switches, cabling, rack space etc and air conditioning, electricity and "care taking". Because of the two more physical servers you'll have to deploy! So both CapEx

and OpEx are better for VMware as well Smiley Happy

3) Performance. With VMware VSA all VMDK I/O is local and bounded with the hypervisor node. With a network solution any I/O request will hit the wire so is SLOW. This means you'll have more of the IOPS from VMware

then from the referenced stuff. VMware wins again Smiley Happy

At the end of the day... These guys are trying to sell you (yes, he works for that company so that's the answer WHY) something which is a) slow b) expensive and c) has no support. I would not buy it Smiley Happy Also think about

other thing... Good vendors don't desperately try to sell their stuff to anybody because of the refunds and bad word of mouth and tons of support. So if somebody is doing this - he's running out of leads and eventually

money. How long they will stay in business? Who knows... Why do you want to buy the same infrastructure component multiple times?!? VMware from the other point of view is here for a while Smiley Happy Buy VMware VSA and be happy!

Good luck!

P.S. I don't work for VMware, I work for a competitor. But I'm not going to sell you what my company does Smiley Happy

--

Atomliu; I'm definitely checking that one out, thanks for the tip!

0 Kudos
Anton_Kolomyeyt
Hot Shot
Hot Shot

Exactly! Adding to what I've told already having extra hypervisor host will allow to do better balancing. 3 >> 2.

VMware VSA Smiley Happy

--

Isn't this a solution built on a Windows operating system? If you're going to buy a new host with this kind of local storage, I think using it as a vSphere host is the best bet.

0 Kudos
Atomliu
Enthusiast
Enthusiast

I just post a suggestion ,so please take it easy.OK?

Also please make the market regular , please post your own opinion without any personal abuse.

0 Kudos
Phatsta
Enthusiast
Enthusiast

Easy guys, I was just asking for opinions and that's what I got. In the end it's up to me to try and form my own opinion from yours. No harm done yet Smiley Happy

Okay so what I've found is that if you ask 10 people, you'll get 10 different opinions. Of course it all varies depending on how you've built these scenarios and worked with them before. The solution I'm tending to lean towards this far (meaning I haven't really decided yet) is this IP SAN:

QNAP Systems, Inc. - Network Attached Storage (NAS) - Products - Products - Storage - SMB - 8-Bay - ...

And one of these switches for backbone:

High-speed 10 Gigabit Copper switching for small to mid-sized organizations

I'll also need to put in 10gbe cards in all three servers. In total, that puts me at just over $6000. I've been using QNAP for years and never been disappointed about support or support hours, or the quality of their products.

Why I'm leaning towards external storage:

If the external storage goes down I can still revert to backups that resides on the backup NAS. I can even run them from there, although it'll be slow I'll still minimize downtime. If a single unit server goes down though - "no hablo", unless I keep an aging box that won't be able to run 12 simultanious guests anyway. With 3 servers, if one of them goes down the other two can share the increased load until I've ordered a new cheap low-end box. In the end I think I've saved both downtime and money. Just my personal experience / educated guess.

But as I said, I haven't decided anything yet, I'm still evaluating all of your opinions and are trying to do my homework Smiley Happy

0 Kudos
Anton_Kolomyeyt
Hot Shot
Hot Shot

1) This is not a suggestion this is a broke marketing pitch for the company you work for. Quite a different thing Smiley Happy

2) Self-moderation is not allowed from what I know Smiley Happy

3) VMware community is not a marketing vehicle for your company.That's for sure.

--

I just post a suggestion ,so please take it easy.OK?

Also please make the market regular , please post your own opinion without any personal abuse.

0 Kudos
Anton_Kolomyeyt
Hot Shot
Hot Shot

1) 3 10 GbE cards are maybe $400 each so does not sound like a very big investment. If you'd go with an external storage you'll end with a 10 GbE to get comparable performance either way.

2) Backups should be physically separated from a production environment. That's for sure!

--

Okay so what I've found is that if you ask 10 people, you'll get 10 different opinions. Of course it all varies depending on how you've built these scenarios and worked with them before. The solution I'm tending to lean towards this far (meaning I haven't really decided yet) is this IP SAN:

QNAP Systems, Inc. - Network Attached Storage (NAS) - Products - Products - Storage - SMB - 8-Bay - ...

And one of these switches for backbone:

High-speed 10 Gigabit Copper switching for small to mid-sized organizations

I'll also need to put in 10gbe cards in all three servers. In total, that puts me at just over $6000. I've been using QNAP for years and never been disappointed about support or support hours, or the quality of their products.

Why I'm leaning towards external storage:

If the external storage goes down I can still revert to backups that resides on the backup NAS. I can even run them from there, although it'll be slow I'll still minimize downtime. If a single unit server goes down though - "no hablo", unless I keep an aging box that won't be able to run 12 simultanious guests anyway. With 3 servers, if one of them goes down the other two can share the increased load until I've ordered a new cheap low-end box. In the end I think I've saved both downtime and money. Just my personal experience / educated guess.

0 Kudos
Phatsta
Enthusiast
Enthusiast

> 1) 3 10 GbE cards are maybe $400 each so does not sound like a very big investment. If you'd go with an external storage you'll end with a 10 GbE to get comparable performance either way

Yeah that sounds about right. And yes, 10GbE is a must as I see it, if using an external storage.

> 2) Backups should be physically separated from a production environment. That's for sure!

Yes, of course. But as long as you're not talking unique data, only server functions such as AD, PS, Exchange transport servers etc, it's not a big deal if you run the VM from it's backup location during the day, then by night restore it to it's original location. Or you can always setup a temporary backup to backup the backup Smiley Wink If it's not too much data at least. There's an old swedish saying that goes "neediness has no law". As long as you know what you're doing, everything's allowed.

0 Kudos
jwhitehv
Enthusiast
Enthusiast

Phatsta wrote:

Okay so what I've found is that if you ask 10 people, you'll get 10 different opinions. Of course it all varies depending on how you've built these scenarios and worked with them before. The solution I'm tending to lean towards this far (meaning I haven't really decided yet) is this IP SAN:

QNAP Systems, Inc. - Network Attached Storage (NAS) - Products - Products - Storage - SMB - 8-Bay - ...

And one of these switches for backbone:

High-speed 10 Gigabit Copper switching for small to mid-sized organizations

I'll also need to put in 10gbe cards in all three servers. In total, that puts me at just over $6000. I've been using QNAP for years and never been disappointed about support or support hours, or the quality of their products.

As you say, it's up to you to evaluate the technology involved.

Why this option doesn't make sense to me:

A $6K host from a Tier 1 vendor is going to be much more reliable than the QNAP.

The Tier 1 vendor will be able to offer you 4 hour response or 6 hour to resolution support agreements.

If the external storage goes down I can still revert to backups that resides on the backup NAS. I can even run them from there, although it'll be slow I'll still minimize downtime.

If that's the case, then you could do the same if the primary host went down. Run on the backup appliance/host. If that's acceptable for a central storage outage, it should be acceptable for a central host outage, especially since the host is less likely to go down and can be repaired more quickly.    

I blog at vJourneyman | http://vjourneyman.com/
0 Kudos
Phatsta
Enthusiast
Enthusiast

Yes, you might be right. Thanks for sharing your views, it's always good to question every view before making a descision.

Now, to air the views I've heard on the pro-external storage from others;

To run 12 simultanious guests you need either 1 high-end server in the range of 6k, 2 middleclass in the range of 2k or 3 low-end in the 1k range. I'd say that's about where my setup today lies, 2 low-end and 1 middleclass. Depending on how long I can count on the external storage, it might come down to a price issue. Every year I get from the external storage I save roughly 1k in saved investments. I'm not entirely convinced myself as it's kind of an unsure way of counting in my opinion, but still there might be a point there. One of the cons to this idea is that I don't even know how much juice I can squeeze out of a DL360...

Another pro-external storage arguement is simply redundancy. 3 hosts + 2 external storages (if counting in the NAS) = spread risks. Hard to argue. If it's set up correctly that is.

A reason I myself find interesting is the functions that come with the QNAP. Things like Rsync, cross platform support, iSCSI LUN snapshot feature and backup features for smartphones with an app that lets you reach pictures like a cloud service. All in all you get other features that can and will show useful, if using it for more than just external storage for vmware.

At the same time I wouldn't mind having a little more juice in the host, and for 6k you get quite a lot. And as you point out, 4 or 6 hr support, although I'm actually more of a hardware tech myself and usually fix all of those problems myself. Only trouble is when you can't get spare parts within a reasonable amount of time. Then you're screwed either way Smiley Wink

0 Kudos
Phatsta
Enthusiast
Enthusiast

And by the way, another pro-external reason that I just thought of the other day... With local RAID arrays (LUN's) there's no way that I've found to report the status of the array automatically, should it fail. I have to rely on manual physical inspections today. The QNAP has features for this. Would be a nice thing. HP servers flash and mane sounds, but next time I buy a new server I can go with a supermicro server (that doesn't flash and make sounds) and save 2k :smileygrin:

No I'm not working there, just happen to know they are cheap Smiley Wink

0 Kudos
jwhitehv
Enthusiast
Enthusiast

Phatsta wrote:

To run 12 simultanious guests you need either 1 high-end server in the range of 6k, 2 middleclass in the range of 2k or 3 low-end in the 1k range.

Running 12 guests on a single host isn't going to put you in the high-end server range. That's pretty run-of-the-mill consolidation.

Depending on how long I can count on the external storage, it might come down to a price issue. Every year I get from the external storage I save roughly 1k in saved investments. Every year I get from the external storage I save roughly 1k in saved investments. I'm not entirely convinced myself as it's kind of an unsure way of counting in my opinion, but still there might be a point there. One of the cons to this idea is that I don't even know how much juice I can squeeze out of a DL360...

I'm not very clear on the math you're using. Spending $6K on external storage saves $1K a year over a $6K on a host?

A reason I myself find interesting is the functions that come with the QNAP. Things like Rsync, cross platform support, iSCSI LUN snapshot feature and backup features for smartphones with an app that lets you reach pictures like a cloud service. All in all you get other features that can and will show useful, if using it for more than just external storage for vmware.

rsync gives you crash-consistent copies of your workloads, which isn't very attractive. In a DR situation, you don't want to have to cross your fingers and hope OS crash-recovery utilities clean everything up enough to use.

What kind of cross platform support does a QNAP have that a virtual appliance running on the host couldn't provide?

iSCSI LUN snapshots are like rsync: crash-consistent. Without hypervisor integration, you're getting an image of the storage as good as if you'd pulled the plug on the workload the instant you did the snapshot.

Backup features for smartphone with an app that lets you reach pictures like a cloud service? If that's important, get a small, cheap QNAP for that feature. It shouldn't drive the purchase of $6K in external storage for vSphere that you don't need.

At the same time I wouldn't mind having a little more juice in the host, and for 6k you get quite a lot. And as you point out, 4 or 6 hr support, although I'm actually more of a hardware tech myself and usually fix all of those problems myself. Only trouble is when you can't get spare parts within a reasonable amount of time. Then you're screwed either way Smiley Wink

Enterprise support isn't about having cheap labor drive out to your location. It's about a supply chain of spare parts designed to reach you when you have a failure, along with skilled labor that's specifically trained to do the type of repair needed. I've built my own PCs from the ground up many times, but when it comes time to dig around in $5K of gear, I want a technician who's intimately familiar with the brand and it's specific quirks to do the job. With an entire company standing behind him to replace anything else that might happen to go wrong.

I blog at vJourneyman | http://vjourneyman.com/
0 Kudos