VMware Cloud Community
MATTBR
Contributor
Contributor

Slow NFS (VM) performance after upgrading to ESXi 5.1 (799733) - Synology NAS

Upon upgrading from 5.0U1 to 5.1 and doing a fresh vanilla install of 5.1, I've noticed a significant slowdown on Windows VMs (didn't test Linux) running on NFS datastores either using a Standard vSwitch or DVS.  When I boot a VM, I get about 5MB/s throughput over NFS, which causes Windows 2008 to take about 4 minutes to present the Login Screen.  Subsequent disk I/O of the running VM remains slow as well.

What makes this interesting is that when I perform a Storage vMotion or cold migrate on this 5.1 VM (or any NFS-based VM), I get ~100MB/sec throughput for these operations on the same NFS datastores.  If I migrate the 5.1 VM to local storage I get expected VM disk performance, so this eliminates it being a Guest OS or driver issue for me.  So, it appears that when the VM is running using NFS it's slow, but when ESXi is performing a host-based NFS disk operation, the disk I/O is unaffected and performs well.

Lastly, if I re-install ESXi 5.0U1, all NFS (host & VM) performance works as expected (~100MB/sec) as was the case prior to upgrading to 5.1.

I'm not sure if anyone has seen a similar issue, but I thought I would post my findings in case someone else runs into this.

Notes: No Jumbo Frames & Active/Passive 1Gbit NFS vmk0. VMware Tools upgraded to 5.1. 1Gbit NFS SAN working fine with other non-5.1 hosts.  Booting off of 16GB USB2 Thumbdrive. Intel® Xeon® Processor E3-1200v2 Synology DS1812+ NAS w/ DSM 4.1-2636 GA

0 Kudos
10 Replies
microkid
Hot Shot
Hot Shot

Same here. Nothing changed, except the upgrade from ESXi 5.0 to 5.1. NFS traffic is deadly slow, it is crawling at approx 500KB/sec per VM, whereas it would do 80-90MB/sec at ease. Loading a VM takes hours now.... Hope there is a quick fix for this.

Syslog shows the following error:

Warning NFS 3813 Short read for object [.....] requested 0x80000 read 0x20000.

The "requested" number changes, the read number stays at 0x20000 all times.

Running ESXi 5.1 on a whitebox and Synology DS1010+ NAS (4.0-2228) with 1Gbit dedicated network link and switch, jumbo frames enabled.

edit: bug confirmed, see http://www.kendrickcoleman.com/index.php/Tech-Blog/synology-users-do-not-upgrade-to-vsphere-51.html

0 Kudos
Jack_Ryan
Contributor
Contributor

Same problem too on Synology DS1512+

No changes to the NAS only difference upgrading from ESXi 5.0 to 5.1.

Boot up of each virtual machine took minutes including logon.

0 Kudos
mcowger
Immortal
Immortal

Known issue with all Synology products at this time:

http://www.kendrickcoleman.com/index.php/Tech-Blog/synology-users-do-not-upgrade-to-vsphere-51.html

--Matt VCDX #52 blog.cowger.us
0 Kudos
MATTBR
Contributor
Contributor

Got this response from Synology last night:

Hi Matt,

Thank you for your message.

Our engineers are currently on top of this issue trying to reproduce it in our labs. We just received a copy of 5.1 this week and we will first try to reproduce with our xs series platform that is on the VMWare HCL. We will then also try to reproduce it on the Standard Business Class models, such as the DS1812+ that are not certified for 5.0 or 5.1 as of yet.

I will let you know as information becomes available.

0 Kudos
dlund
Enthusiast
Enthusiast

Same thing here, DS211 with DSM 4.0-2198.

After upgrade to 5.1 NFS have been really slow, iSCSI is fine.

0 Kudos
tuomiju
Contributor
Contributor

Same issues here with synology ds212j with newes DSM and esxi5.1.

NFS simultaneous use is totally unusable.

0 Kudos
microkid
Hot Shot
Hot Shot

Update: just got an e-mail from Synology. They have a patch available wich updates the nfds.ko file on the Synology. However, the file is supposedly meant for DSM 4.1-2636 where I run DSM 4.0-2228. So I asked Synology if there is a patch for 4.0 too. They offered to update my NAS remotely using the patch for DSM 4.1 (maybe this nfds.ko file is DSM version independent?). I'd rather patch it myself so I know what has changed. I asked them to send me the instructions to patch my DSM 4.0 myself. Now lets hope and wait Smiley Happy

0 Kudos
SCMHenry
Enthusiast
Enthusiast

Is this patch freely available for download at the Synology website, or did they give any indication how soon a patched version of the DSM would be available??

0 Kudos
tuomiju
Contributor
Contributor

I got private patch from synology yesterday and after that all problems were gone.

I have DS212J and latest DSM.

0 Kudos
orchestrationio
Contributor
Contributor

They released an update today, and one of the fixes is:

Improved the NFS performance when mounted to VMware ESXi 5.1 servers.

0 Kudos