NFS (not NetApp) experiences

[bennice]

I'm trying to find a compelling reason to use NFS for ESX/ESXi datastores, and trying to figure out where it makes the most sense. We have ~40-ish ESX servers and around 500 guests running scattered around on DAS and various iSCSI solutions. I've used fiber channel SANs in the past, but we don't leverage that technology for VMware at my current company due to the cost.

Items for consideration:

Some of the guests run apps that are very multicast intensive - it's not uncommon to see 20 Mb/s of sustained multicast traffic on vSwitch0 (the front side network) on any given production ESX server. So dedicated network fabric is a must, and already exists in most cases for iSCSI communication.

Needs to be able to host data coming from 10 ESX/ESXi servers and accommodate ~150 virtual machines hosting various applications.

I NEED fault tolerance and modularity. Single points of failure are not acceptable, and taking the entire solution offline for any reason any amount of time is not an option. Upgrades and maintenance need to be able to happen on the fly without disrupting servers (some degree of performance degradation is acceptable during off-peak hours).

NetApp has priced themselves out of the mix - which is how we ended up with iSCSI in the first place.

So, besides NetApp (whose strength really IS NFS) who else makes an NFS solution that fits the above criteria? In what circumstances and with what applications does NFS really shine?

I've tried playing around with server-based NFS and it was laughably slow, and was a single point of failure.

[bennice]

Wow, nobody has any positive non-NetApp experiences with NFS?

Or maybe everybody is busy working.

[eoporto]

In one case, we've used NFS from a RHEL4 system on an HP DL380 which we mount to dump our vcbMounter images to. They are then backed up to tape later on.

Also good for storing ISO images, since the NetApp space is more expensive and can probably be reserved for more crucial data.

Beware of he who would deny you access to information, for in his heart, he dreams himself your master.

P.S : If you think that the answer is helpful please consider rewarding points.

[dnetz]

We're in the process of deciding on storage for a new infrastructure environment. We're pretty much a netapp shop already and heavy NFS users and met with them yesterday to show off netapp + vmware on nfs and they do have a compelling solution. I do agree about the price though, netapp sure knows how to charge their customers. Our only other alternative right now is Sun Storage 7000, their pricing is 20% of netapp in the 25TB area and they don't charge for specific features (cifs, nfs etc). However their concept is brand new, lacks quite a lot of features, including nice middleware apps like an equivalent of SnapManager. They already have some customers running infrastructure on storage 7000, so if it seems interesting, contact a sales rep and request a demo. They seem pretty willing to lend you a box for tests as well. And they've got an s7000 appliance for download here: http://www.sun.com/storage/disk_systems/unified_storage/resources.jsp

For our environment, it seems we're going with netapp for vmware and storage 7000 for user and application data.

[bennice]

I personally have nothing against NetApp, and have had great success with their products at past jobs. However, the company I work for now has a much tighter hold on the purse strings, so to speak, and as good as NetApp's products are there is no way I can justify the cost...not even to myself let alone the executives. I actually tried to get a demo in-house late last year and they dropped the ball (either NetApp or our CDW sales rep).

We already have various iSCSI solutions in-house, but it's not the right thing for ALL situations. I'm trying to diversify our VMware infrastructure and implement technologies in circumstances where they perform the best. So far, the only place that I know of where NFS would useful is for storage of things like backups (which we don't have many of) and static storage, like ISO images. I'll check out solutions from Sun too...thanks for that link.

Does anyone have ANY other non-NetApp success stories? With NFS being such a popular choice for ESX stores I'd imagine there's got to be something else out there worth trying. I've tried searching these forums, but all I seem to be able to find is NetApp, NetApp and more NetApp.

[hhlodge]

We are currently a NetApp shop with 200-300 VMs running at any time against a FAS3020 using NFS on 5 SATA shelves split up in 4 aggregates, 10 TB with deduplication on. We are dying performance-wise (disk/spindle bound according to perfstats and statit analysis by NetApp) and are looking for a new solution, but want to remain on NFS. NetApp is unable to show us how a move to a newer FAS3160 would resolve our issues and allow us growth out a year or two. And they say we must move to fiber channel shelves to have any chance of having decent performance, so that doubles the investment. So we are strongly leaning toward leaning toward a Sun 7140 cluster with 44TB in a mirrored disk configuration. The NFS numbers posted are impressive. The only downside thus far is my recent discovery that the entire ZFS disk pool needs to be a single RAID configuration scheme, so I can't have some portion RAID10 and non-running, archived VMs on RAID5 or 6. I'd love to hear from anyone using one of these Sun devices in production and hear their experiences; good and bad. Thanks.

[meistermn]

A non Netapp NFS solutions is exanet

Yes at the moment it is not on th hcl of vmware.

[meistermn]

Bluearc is another nfs vendor.

Look at this slide page 27 consolidating netapp to bluearc.

Some performance test based on 32 KB block size , 4K block size missing!!!

Mixed Workload 4K

http://www.bluearc.com/html/library/downloads/bluearc_titan2000_tolly.pdf