ESX - NAS(freenas) performance

[sveihan]

Hi

I want to use a NAS as storage for iso's, templates and test-vm's.

Currently I'm running FreeNAS on a Dell 2850 with 10k SAS disks (raid5, 1,4TB)

When connecting my ESX's to my NAS using NFS I only get 4-5MB/s (r/w), but when connecting my MAC to the NAS I get 45-50MB/s.

What can cause the poor performance I get between the NAS and my ESX's?

Are there any tuning parameters that will fix this?

(All network connections are Gb)

Svein

[Bisti]

Has NFS been correctly configured (/etc/exports)?

• rw

• no_root_squash

• sync

Is VMkernel port used for NAS storage sharing it`s physical NIC with other kind of network traffic?

[sveihan]

I'm using Freenas livecd (with flash for saving config)

Configuration done in the web-gui:

cat /etc/exports

/mnt/bigdisk/vmware -alldirs -mapall=root -network 128.37.106.0 -mask 255.255.255.128

I also tried to set async i sysctl.conf (using the GUI):

cat /etc/sysctl.conf | grep async

vfs.nfsrv.async=1

I have 4 nics on each server, 2 and 2 nics are teamed on two vswitches.

vswitch0: regular IP traffic (internett)

vswitch1: ISCSI traffic (isolated network with dedicated switches)

The NFS-VMkernel-Port is on vswitch0:

I have two NFS storage boxes connected to the NFS-VMkernel-Port:

DataDomain 510 for VM-backup and Freenas(dell 2850)

I get approx. 40-45 MB/s using NFS with the DD510 and 4-5 MB/s using NFS with the FreeNAS-box.

At the same time I get 45-50 MB/s from my desktop-MAC(Leopard) using NFS with the FreeNAS-box.

So, this seems to be an issue between the ESX and FreeNAS.

[sveinha]

anyone??

I really need to move (smotion) this VM back to my san.

550GB with 3-4MB/s will take a looong time.

Svein

[Bisti]

I`m not really sure what might cause this issue but maybe we`ll work something out.

• What exactly is that you are trying to do? Migrate VM storage from one NAS to another? From FreeNAS to DD510?

• FreeNAS is using NFS version 3 over TCP right?

• Have you tryed testing those speeds with other ESX server in your network?

• Meaby there is some routing issue, have you tryed doing traceroute from your ESX to freenas?

[athlon_crazy]

May sound silly but just trying to figure out your problem.. Your ESX host connecting to FreeNas with gigabit or 100mb speed connection?

System Engineer

Zen Systems Sdn Bhd

Malaysia

www.no-x.org

[sveinha]

Yes, Gb

ESX 2 teamed Gb-nics <-> Cisco 6509 <-> FreeNAS 1 Gb-nic

Svein

[sveinha]

I´m trying to migrate a VM from NFS storage to ISCSI storage

FreeNAS(Dell 2950) -> Cisco6509(switch) -> ESX(vswitch0 -> vswitch1) -> HP2810 -> ISCSI-switch/storage.

I tried change a setting on the vswitch while moving the VM, and then it failed....

Now I´m stuck with:

-rw------- 1 root root 96M Apr 14 16:01 DMotion-scsi0:00_post.hive.no-delta.vmdk

-rw------- 1 root root 315 Apr 14 15:52 DMotion-scsi0:00_post.hive.no.vmdk

-rw------- 1 root root 257M Apr 14 16:01 DMotion-scsi0:01_post.hive.no_1-delta.vmdk

-rw------- 1 root root 320 Apr 14 15:52 DMotion-scsi0:01_post.hive.no_1.vmdk

-rw------- 1 root root 500G Apr 14 15:37 post.hive.no_1-flat.vmdk

-rw------- 1 root root 410 Apr 14 15:47 post.hive.no_1.vmdk

-rw------- 1 root root 50G Apr 14 15:37 post.hive.no-flat.vmdk

-rw------- 1 root root 406 Apr 14 15:47 post.hive.no.vmdk

I have to wait untill tonight to remove the snapshot...

Svein

[athlon_crazy]

have you play with ESX advance configuration for NFS (send / receive buffer size). It's may or may not give you slightly improvement..

System Engineer

Zen Systems Sdn Bhd

Malaysia

www.no-x.org

[BoomBoom17]

Any resolution on this? I am having the same issue. Not sure if anyone has had any luck with performance tuning the free nas to run VMs.

[natewilson]

I'm wondering the same thing.. I just added a FreeNAS box as an NFS NAS. I transferred a few VMs to it, and when copything them via a Linux server which has the old NAS & FreeNAS mounted, I saw ~200-250Mbps throughput. When I moved a VM between the two NAS's via VirtualCenter, I got ~50Mbps throughput.

Is ESX just much slower than Linux when accessing NFS? Or can the performance be improved?

[irvin]

I got exactly the same problem. Does it solved already??

I got over 650Mbps on iSCSI connected ESXi and freeNAS, but NFS mount got only 4-5MB/s...

ESXi has 4 giga nics to connect freeNAS with 2 giga nics...

Please let me know if you got the answer.

Thanks!

[abhr]

Having the same "issue" as well and ended up going iSCSI. FreeNAS is running FreeBSD 7.2, and that has some issues with ZFS and NFS if you are running such a config.

To my knowledge, the NFS performance issue (I am getting the same numbers as natewilson) is caused by ESX always mounting NFS exports sync. Some proprietary solutions (the NetApps and StorageTeks of the world) have quirks in place and performance is good.

FreeNAS 0.7 is in maintenance mode, as the devs have said to be working on a new version with a Django-based UI and FreeBSD 8.0 as an OS. First builds should surface in May or June of 2010, hopefully