NFS MOUNT PRESENTED TO ESXI KEEPS DROPPING

[fireinthebarrel]

Hello,

I'm running an old dell 3420 workstation as an ESXi host for a home lab. I have an UNRAID server where I am using the cache pool (all flash, raid z1) to present an NFS Datastore to the ESXi host. The storage keeps dropping off. I can't seem to figure out what the issue is and I'm hoping someone will take some time to help.

Thanks!

[vmngo]

How many nics do you have on your esxi host? Are you having all traffic (VM and storage) on the same subnet/vlan? Did you attempt to storage vMotion/run all VMs from the NFS mount?

Looking through the vmkernel logs, i'm seeing this:

2023-08-03T16:45:36.451Z cpu0:1050726 opID=470aa073)WARNING: NFS41: NFS41_VSIMountSet:413: NFS41_FSMount failed: Timeout
2023-08-03T16:45:36.452Z cpu4:1049135)SunRPC: 1092: Destroying world 0x100fb8

Timeouts indicate network connectivity layer issue (try changing network cables) 

2023-08-03T18:45:40.071Z cpu0:1051656 opID=33c5bb13)WARNING: NFS41: NFS41FileDoCreateDir:3981: Failed to process the create directory task: Read only
2023-08-03T18:45:40.071Z cpu0:1051656 opID=33c5bb13)WARNING: NFS41: NFS41FileOpCreateFile:4265: Failed to create file "test" of type 2: Read only

Failed to create tasks sounds like a permission issue on your NFS export in unraid.

Closer to today's date, it looks like logs are showing
2023-08-08T13:51:07.879Z cpu2:1082548)WARNING: NFS: 2581: Failed to get attributes (No connection) 

Did you make some network related changes that doesnt allow your VMK1 interface to talk to the NFS mount?

[fireinthebarrel]

I'm running everything over the single 10Gb nic with an MTU of 1500. I have a Unifi 8 port layer 2 10Gb switch and the host has a single 10GB nic that is pretty busy doing an rsync task...I'm betting thats it. I didn't even think of rate limiting the rsync task. Thats the issue for sure.

[Alfista_PS]

Hi,

I had this problem on some storages and it was disk independent.

Your system with disks doesn't have enough IOPS for operation and the needed amount it can't hold for longer time (only some peaks).

I was searching longer time and tested some storages that I found which can hold for us needed amount of IOPS.

And next problem - go to NFS 3 even its older but its faster (hasn't so many security which slower the access).

Maybe you can try to install TrueNAS on the UNRAID server which can help you get more from the HW as a NAS and it's free. Then you can connect the storage over iSCSI which can help you get more from the storage and from the network.

[ptarnawski]

Hey @fireinthebarrel 

A few things to have a look at: 

• Ensure that the MTU (Maximum Transmission Unit) settings match across all devices involved (ESXi host, UNRAID server, and the network switch). Since you're using 10Gb networking, you might consider increasing the MTU to 9000 or higher for better performance.
• Make sure your network switch is properly configured for 10Gb connections and that it's not encountering any issues.
• Review the NFS export settings on your UNRAID server. Ensure that the export is configured with the correct permissions and that the ESXi host has the necessary read and write access.
• Since you mentioned that the ESXi host's 10Gb NIC is busy with an rsync task, consider scheduling or rate-limiting the rsync task to avoid overwhelming the network link.