VMware Cloud Community
benp23
Contributor
Contributor

NetApp and 3.5: iSCSI or NFS?

Hello,

I know this topic has been much discussed, and I'm reading through the official documentation, but I need some straight advice. :smileygrin: We just got started this past year with VI3, and will have our second ESX host upgraded to 3.5 next week---we of course have already upgraded VirtualCenter to 2.5. There are 4 production vm's running on one host and 5 on the other on local VMFS partitions.

We want to eventually use VMotion, HA, and Consolidated Backup. We are debating whether to use iSCSI or NFS, so which is better? We will eventually be running many hosts/vm's from the NetApp FAS, as we still have 6 Foundation licenses remaining. We'll likely be dividing our FAS into (two?) 6TB aggregates and using 3TB LUNs per ESX HA pair for vm storage (ballpark).

Currently our initial two hosts are Proliant DL 380 G5s using two onboard teamed NICs through one vswitch (service console and vm's on the same switch) to redundant gig switches. I know this isn't ideal for performance but it doesn't matter for these particular vm's. We're planning on adding another dual-port NIC to each host to team for iSCSI and running future vm's off that vswitch. If we use NFS will we need to do this? What are the basic parameters?

I hope this is enough information to begin with.

Thanks,

benp23

0 Kudos
10 Replies
esiebert7625
Immortal
Immortal

It sounds like performance is not that important for you right now but it may be at some point. iSCSI usually outperforms NFS, also iSCSI supports VMFS volumes were NFS does not. Both are network-based storage solutions so having dedicated redundant NIC's in your ESX host is definitely important. I'd recommend reading through some of the below links:

NAS/NFS vs iSCSI for ESX - http://www.vi411.org/2006/10/10/nasnfs-vs-iscsi-for-esx.html

Choosing and Architecting Storage for your Environment - http://download3.vmware.com/vmworld/2006/adc0135.pdf

New Storage Technologies: iSCSI, NAS and VMFS 3.0 - http://www.vmware-tsx.com/download.php?asset_id=20

Comparison of Storage Protocol Performance - http://www.vmware.com/files/pdf/storage_protocol_perf.pdf

VMware over NFS - http://storagefoo.blogspot.com/2007/09/vmware-over-nfs.html

Why VMware over NetApp NFS - http://viroptics.blogspot.com/2007/11/why-vmware-over-netapp-nfs.html

Ethernet-based Storage Configuration - http://www.vmware.com/pdf/iscsi_storage_esx.pdf

Configuring iSCSI in a Vmware 3 environment - http://www.vmware.com/pdf/vi3_iscsi_cfg.pdf

iSCSI Design Considerations and Deployment Guide - http://www.vmware.com/files/pdf/iSCSI_design_deploy.pdf

Eric Siebert

VMware Communities User Moderator

-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=-

Visit my website:

-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=-

stvkpln
Virtuoso
Virtuoso

Well, the reality is that there's no right or wrong answer.. Just what makes sense for you and your environment! I work in a fairly large NetApp-centric environment, and I can say that one of the huge benefits to using NetApp at all are flexvols. They make everything easier, don't they? If performance is secondary to flexibility, I'd possibly say go with NFS. Going down that route, it's a lot easier to add capacity (just grow the flexvol) and it's a lot simpler to maintain overall. With that said, in that configuration for a production environment, I'd plan on having 6 NIC's: 2x for service console/vmkernel (vmotion interface), 2x for vmkernel for NFS connectivity -- yes, I'd keep that seperate from the 'administrative' vmkernel interface, and 2x for vm access. I use NFS datastores on one of our out of use R200's right now and have about 500GB or so of VM's running happily along in a test/dev environment.I will say that you shouldn't sell performance (or iSCSI) short, however. Especially if you're using hardware initiators (I'd recommend it), you will see significant improvement in performance of your VM's, as well as tasks like deploying new VM's, etc.. will seem a lot quicker going across iSCSI. The downside is that you lose some of the flexibility that comes with NFS.

I'd give thought to where you intend your VI to be 18-24 months from now and plan accordingly. As my boss likes to say, take whatever you think it is, double it, and add 20%. You have a unique opportunity now to plan for a to-be state. It never pays in the long run to be short sighted and to only factor in the short term only. Plan as if your decisions WILL come back to haunt you Smiley Wink

-Steve
benp23
Contributor
Contributor

Thanks for your responses.

0 Kudos
mcowger
Immortal
Immortal

Man, if it were anyone besides NetApp, I'd say go iSCSI. But given their crappy iSCSI implementation and their awesome NFS implementation, I'd go NFS.

--Matt

--Matt VCDX #52 blog.cowger.us
0 Kudos
benp23
Contributor
Contributor

Matt,

What do you mean? The other aspects of NetApp's solution (ie basic aggregate/volume/LUN performance) have been solid for us so far? Can you get specific?

Thanks,

benp23

0 Kudos
mcowger
Immortal
Immortal

I love the netapp gear for management, etc, but I found their choice to implement FCP and iSCSI LUNs as simply large files on top of their WAFL to be maddeningly stupid, and subject to terrible fragmentation (which, regardless of what NetApp claims, DOES happen).

If the LUN performance is sufficient for you, go for it. It wasn't for me. I still think their NFS implementation is SO good that I would choose it over their iSCSI.

--Matt

--Matt VCDX #52 blog.cowger.us
0 Kudos
benp23
Contributor
Contributor

Matt,

Thanks much for the advice.

benp23

0 Kudos
RParker
Immortal
Immortal

> SO good that I would choose it over their iSCSI.

....and Fibre?

0 Kudos
mcowger
Immortal
Immortal

The underlying implementation of FC LUNs is the same as for iSCSI on NetApp gear. So yes. Unless I needed a real LUN, I'd be choosing NFS first on NetApp.

--Matt

--Matt VCDX #52 blog.cowger.us
0 Kudos
benp23
Contributor
Contributor

Correct me if I'm wrong here, but NFS does not allow NFS, nor can you P2V.

0 Kudos