Unable to mount NFS datastore to 2 out of 3 hosts

[BattleNerd]

Hello All,

I'm new so please bear with me....

Our infrastructure contains 3 main hosts, we use a Netapp SAN to connect our NFS datastores to, for some reason even when I manually try connecting the datastore to hosts i'm unable to do it on 2 of the 3.

All 3 hosts are on the same network segment, they have a vmkernel interface that connects to the VLAN of our SAN (so all traffic flows correctly, and there is another VMK interface dedicated for management as well) I can connect a datastore from 1 host to all 3 of our SAN controllers, but for some reason the 1 SAN controller thats at our DR site I can't connect to (again on 2 of the 3 hosts)

I went through the VMKERNEL.LOG and noticed this showing up

2013-02-01T16:23:58.453Z cpu14:5066)NFS: 157: Command: (mount) Server: (X.X.X.51) IP: (X.X.X.51) Path: (/vol/pdvideo) Label: (Surveillance_Video) Options: (None)

2013-02-01T16:23:58.457Z cpu19:5066)NFS: 292: Restored connection to the server X.X.X.51 mount point /vol/pdvideo, mounted as b2e55d25-84689502-0000-000000000000 ("Surveillance_Video")

2013-02-01T16:23:58.457Z cpu19:5066)NFS: 217: NFS mount X.X.X.51:/vol/pdvideo status: Success

2013-02-01T16:24:09.468Z cpu7:4103)WARNING: NFS: 217: Got error 13 from mount call

2013-02-01T16:24:09.470Z cpu7:4103)WARNING: NFS: 217: Got error 13 from mount call

2013-02-01T16:24:09.472Z cpu7:4103)WARNING: NFS: 217: Got error 13 from mount call

2013-02-01T16:24:39.475Z cpu6:4102)WARNING: NFS: 217: Got error 13 from mount call

2013-02-01T16:24:39.476Z cpu6:4102)WARNING: NFS: 217: Got error 13 from mount call

2013-02-01T16:24:39.477Z cpu6:4102)WARNING: NFS: 217: Got error 13 from mount call

2013-02-01T16:24:43.097Z cpu18:474641)NFS: 157: Command: (mount) Server: (X.X.X.51) IP: (X.X.X.51) Path: (/vol/pdvideo) Label: (Surveillance_Video) Options: (None)

2013-02-01T16:24:43.101Z cpu18:474641)NFS: 292: Restored connection to the server X.X.X.51 mount point /vol/pdvideo, mounted as b2e55d25-84689502-0000-000000000000 ("Surveillance_Video")

2013-02-01T16:24:43.101Z cpu18:474641)NFS: 217: NFS mount X.X.X.51:/vol/pdvideo status: Success

2013-02-01T16:25:09.480Z cpu8:4104)WARNING: NFS: 217: Got error 13 from mount call

2013-02-01T16:25:09.481Z cpu8:4104)WARNING: NFS: 217: Got error 13 from mount call

2013-02-01T16:25:09.482Z cpu8:4104)WARNING: NFS: 217: Got error 13 from mount call

2013-02-01T16:25:39.484Z cpu8:4104)WARNING: NFS: 217: Got error 13 from mount call

NFS: 190: NFS mount X.X.X.51:/vol/pdvideo failed: The mount request was denied by the NFS server. Check that the export exists and that the client is permitted to mount it.

In this example you will see that it shows a status of Success but then I get all these Error 13's.

I verified all the permissions were set correctly on the /vol and made sure from both the SAN controller and the host that both can communicate with each other. I did a health check through the netapp snapin for vcenter and all show healthy. It pings the SAN and checks connectivity to all datastores, which we have other datastores pointing to this SAN controller as well as the effected host. So it makes no sense why I can't add a new datastore to this host.

I'm a newb so be kind... please haha

Thanks for your help!

[memaad]

Hi,

check this KB out http://kb.vmware.com/kb/2009919 , might be helpful.

Regards

Mohammed

[BattleNerd]

ya didn't work we don't use wildcard for IP. i've done both using the FQDN as wel as the IP address and neither worked.

[BattleNerd]

Also I don't get the:

WARNING: NFS: 959: MOUNT failed with MOUNT status 13 (Permission denied) trying to mount Server

I only get the Got error 13 in the logs.

[memaad]

Hi,

From command line try mounting on ESXi host.

Regards

Mohammed

[BattleNerd]

Mohammed,

I’m somewhat new to the vmware CLI can you help with a support doc that would give me the commands to mount the datastore and I’ll try that right now.

Thanks!

[BattleNerd]

Mohammed

I got this error:

~ # esxcfg-nas -a PDVideo -o X.X.X.51 -s /vol/pdvideo

Connecting to NAS volume: PDVideo

Unable to connect to NAS volume PDVideo: Sysinfo error on operation returned status : The NFS server denied the mount request. Please see the VMkernel log for detailed error information

I’ll look in log now for what the error info is.

[BattleNerd]

Mohammed –

I get the same errors:

Warning: NFS: 217: Got error 13 from mount call

[memaad]

Hi,

what all permission you have given to this ESXI host in netapp nfs server.

Regards

Mohammed

[BattleNerd]

I gave the following:

Allow Root Access

Allow Read Write

This same /vol can be mounted to 1 host, but not on the other two. All 3 hosts are on the same network segment, all using the same SAN VLAN for accessing the SAN. All 3 hosts have a datastore mounted from a /vol on this same controller. Its only new datastores I can not mount to this /vol

[memaad]

Hi,

Can you check this option "The NFS export must be set for either no_root_squashor chmod 1777." on storage array.

Regards

Mohammed

[grasshopper]

Check the best practices for modifying the NetApp config files.  Specifically sometimes you must use wordpad instead of notepad.  A common problem is special characters injected (i.e. carriage return, etc.).  If there is a special character after the first host's IP address, for example, the first host would be allowed access as expected, but the IPs listed after that would be denied.

[BattleNerd]

I've put in a call to netapp support but wanted to get some ideas from VMware guys as well try have all the esx logs so hopefully will get this resolved soon this had been working like I said all 3 hosts have active connections to this SAN on two volumes so I know it's configed right.

Sent from a phone, please excuse any errors.

[esxguide]

I'm getting the same error. I have 22 hosts 5.0.0-build914586 update 2 all mounting correctly, but this one after a reboot, will not mount the NFS Export. In addition, it had been mounted previously.

Now we use 192.1.1.0/24 Read Write and all hosts root access for our permissions, but that is the same on the other 16 NFS Exports. And this one is the only one it is complaining about.

I'm putting in a tech support request right now, but if you got an answer from them on your call, please update here.

Thanks!