ESX 3.01 Cluster with Netapp NFS file system mounts

[ericsowa]

I am looking to move our virtual server storage from a ISCSI to NFS mounts with our two node ESX 3.01 Cluster. Has anyone done this in production envoriment with a netapp fast 270c appliance. We are using the netapp for iscsi currently.

[biekee]

Why would you want that? NFS will give you more overhead i.e. much slower. I would disadvice it.

bk

[ericsowa]

Can anyone help me out?

[ericsowa]

Why is NFS giving more overhead? ISCSI and NFS are both using the dual gig copper backbone.

[paulo_meireles]

As biekee said, it doesn't seem to be such a great decision.

However, you can do it by cold-migrating the VMs. If you have some temporary storage, cold-migrate teh VMs there, reconfigure the NAS and then move them back. If not, you may have to cold-migrate from iSCSI volumes to NFS volumes on the same NAS.

Will you share with us why you're doing that?!

Paulo

[ericsowa]

I am just trying to see what the performance gain or loss with iscsi vrs nfs. We are uisng our Netapp fast 270c with nfs in full production on all our windows hosts and we see great performance. I have read the forums and there is a mix beteen people using NFS and people going ISCSI. I know there is over head but my DL585 G2 servers with quad proc and dual cores should handle the overhead fine.

[paulo_meireles]

Please, let us know your results!

[ericsowa]

Paulo do you see any problem with using NFS as a mount point for virtual machines?

[Mike_Fink]

NFS is going to load the NTAP more then iSCSI. Remember, NFS is a file mode access protocal, iSCSI is block mode.

IMHO, unless you have some overriding reason, you should probably leave it as iSCSI.

If you want to migrate to NFS, the advice given is correct, add the NFS datastore and then migrate the VM to the new drive. Then delete the iSCSI LUN from the NTAP (and the volume). This will free up the disk space for other purposes.

[Mike_Fink]

One other thing.

Make sure you take a look at the utilization on the NTAP, that's where you're going to see the biggest impact.

sysstat -u 1

Watch the CPU. That's where you will see the hit when moving from iSCSI to NFS.

As far as host (ESX) performance, I think you will see a small performance hit; but I expect the NTAP will take the brunt of it. Especially with the 270.

[paulo_meireles]

Paulo do you see any problem with using NFS as a mount point for virtual machines?

Some operations are much faster on a VMFS volume, which can reside on an iSCSI target but cannot reside on an NFS volume. For example, when you clone a VM, space marked as "free" is not copied, which can speed things up. As I said elsewhere, there is no long-term in IT; use what fits your needs today. Of course, account for the costs of change.

Paulo

[ericsowa]

Thanks for all your input...

[paulo_meireles]

You're welcome! Just for the sake of curiosity, please post your results after you've migrated to NFS.

[stvkpln]

NFS is going to load the NTAP more then iSCSI.

Remember, NFS is a file mode access protocal, iSCSI

is block mode.

That's pretty much accurate. Expanding on what Mike said, please keep in mind that NFS and CIFS were both filesystems that were designed for many users to access files via a network protocol. It wasn't designed with multi-gig sized files in mind that you'd constantly maintain access to. That's a big part of why iSCSI was created.. to allow for a network-based protocol for block mode access.

Also factor in that when you use NFS, you're using the underlying filesystem (in this case, it's WAFL presented via NFS) instead of the VMFS formatted filesystem you would use with iSCSI. There's some performance differentials you're going to see in that potentially, depending on how large an environment you have.

As with most things, depending on your situation, your mileage may vary..

[dalepa]

We are running 400 VMs across 13 ESX3 host on one Netapp 3050c via NFS. We keep 3 hourly 3 daily and 2 weekly snapshots. We also do daily snapmirrors to a Netapp R200 for dedundancy and full offline VM backups. We also snapmirror critical VMs to our DR site. We have 200 vm's on Hitachi storage and we are planning to move them all the Netapp. We will be adding another 500 VMs by the end of the year.

The latest version of ontap does deduplication. (Netapp calls this A-SIS) We are getting 80% (5TB of Vm's deduped down to 1TB) deduplication on our R200 A-SYS. Its just a matter of time before we run A-SIS on our primary.

Performance: Netapp NFS has been as fast or much faster in the test that we have run against our other storage devices.

It's just a matter of time before A-SIS (deduplication) is a hot topic with vmware; even Netapp didnt believe the saving in storage we were are to get with A-SIS.