VMware Cloud Community
giox
Contributor
Contributor

NFS 4.1 on ESXi 6.5: unable to browse datastore

I mounted a NFS 4.1 share on ESXi 6.5a (single host, no vcenter).

This is the CLI command, but I can also mount it via web interface:

 esxcli storage nfs41 add -a AUTH_SYS -H 192.168.98.4 -v MyLinuxNFS -s /mnt/ds-esxi

My linux /etc/exports is

     /mnt/ds-esxi    192.168.98.0/27(rw,sync,no_subtree_check,anonuid=1002,anongid=1002)

After correctly mounting it, I can see it on the web client:

pastedImage_0.png

But I'm unable to browse MyLinuxFS, the "Datastore browser" stays in the following state forever:

pastedImage_1.png

Permission are correctly set, because I can write files and see the datastore via the CLI:

[root@vm2:~] echo "123test" > /vmfs/volumes/MyLinuxNFS/testfile.txt
[root@vm2:~] ls -la /vmfs/volumes/MyLinuxNFS/
total 24
drwxr-xr-x    4 root     root             5 Feb  5 19:32 .
drwxr-xr-x    1 root     root           512 Feb  5 19:32 ..
drwxr-xr-x    2 root     root            10 Feb  4 17:10 testdir1
drwxr-xr-x    2 root     root             4 Feb  4 22:26 testdir2
-rw-r--r--    1 root     root             8 Feb  5 19:32 testfile.txt

NFSv3 is working fine, but I would like to use NFSv4.1 because I'm experimenting to workaround bug KB2010953‌ in a bigger infrastructure.

I also tried with FreeBSD 11, it seems to support NFSv4.1, but I have the same problem described here: FreeBSD11 NFSv4.1 is not compatible with ESXi 6.5.

Any idea to make NFS 4.1 work on ESXi 6.5 ?

12 Replies
dalong4net
Contributor
Contributor

The datastore is read only in this case. I encountered exactly the same issue when using NFS 4.1 share from FreeBSD 11. I haven' t figured out any solution yet.

If someone can shine some light on this, it will be very helpful. Hopefully VMware can address this soon, since I'm evaluting the vsphere for production now.

Reply
0 Kudos
JelleHissink
Contributor
Contributor

I am having the same issue, have spent a few days now.

My setup uses nfs4 from an ubuntu 16.04.1 server.

The thing is on ssh on the esxi 6.5a server I can read and write files to the nfs server, using workstation 12.5 I am able to browse the datastore.

Using PowerCli reading from the datastore also works.


So the issue seems to be located in the web interface.

Hope this gets fixed soon, it is almost unworkable like this.

Reply
0 Kudos
JelleHissink
Contributor
Contributor

Hmmm, digging some further, not all seems well.

I created a new vm on the datastore:

-rw------- 1 nobody nogroup 2147483648 Feb 10 23:39 test_0-flat.vmdk

-rw------- 1 nobody nogroup        492 Feb 10 23:39 test_0.vmdk

-rw-r--r-- 1 nobody nogroup          0 Jan  2  1980 test.vmsd

-rwxr-xr-- 1 nobody nogroup       2009 Feb 10 23:40 test.vmx

-rw-r--r-- 1 nobody nogroup      52502 Feb 10 23:40 vmware.log

so the files are on the ubuntu server.

Starting the vm however fails, the vmware.log of the vm:

2017-02-10T22:34:48.827Z| vmx| I125: DISK: OPEN scsi0:0 '/vmfs/volumes/18630e34-de98dd50-0000-000000000000/test/test_0.vmdk' persistent R[]

2017-02-10T22:34:48.831Z| vmx| I125: DISKLIB-VMFS  : VmfsExtentCommonOpen: possible extent truncation (?) realSize is 0, size in descriptor 4194304.

2017-02-10T22:34:48.831Z| vmx| I125: DISKLIB-VMFS  : "/vmfs/volumes/18630e34-de98dd50-0000-000000000000/test/test_0-flat.vmdk" : failed to open (The file specified is not a virtual disk): Size of extent in descriptor file larger than real size. Type 3

2017-02-10T22:34:48.831Z| vmx| I125: DISKLIB-LINK  : "/vmfs/volumes/18630e34-de98dd50-0000-000000000000/test/test_0.vmdk" : failed to open (The file specified is not a virtual disk).

2017-02-10T22:34:48.831Z| vmx| I125: DISKLIB-CHAIN : "/vmfs/volumes/18630e34-de98dd50-0000-000000000000/test/test_0.vmdk" : failed to open (The file specified is not a virtual disk).

2017-02-10T22:34:48.831Z| vmx| I125: DISKLIB-LIB   : Failed to open '/vmfs/volumes/18630e34-de98dd50-0000-000000000000/test/test_0.vmdk' with flags 0xa The file specified is not a virtual disk (15).

2017-02-10T22:34:48.831Z| vmx| I125: DISK: Cannot open disk '/vmfs/volumes/18630e34-de98dd50-0000-000000000000/test/test_0.vmdk': The file specified is not a virtual disk (15).

2017-02-10T22:34:48.831Z| vmx| I125: DISK: Opening disks took 5 ms.

2017-02-10T22:34:48.831Z| vmx| I125: Module 'Disk' power on failed.

2017-02-10T22:34:48.831Z| vmx| I125: VMX_PowerOn: ModuleTable_PowerOn = 0

2017-02-10T22:34:48.831Z| vmx| I125: SVMotion_PowerOff: Not running Storage vMotion. Nothing to do

2017-02-10T22:34:48.833Z| vmx| I125: Vix: [90009 mainDispatch.c:1175]: VMAutomationPowerOff: Powering off.

2017-02-10T22:34:48.834Z| vmx| W115: /vmfs/volumes/18630e34-de98dd50-0000-000000000000/test/test.vmx: Cannot remove symlink /var/run/vmware/0/80444906393_90009/configFile: No such file or directory

2017-02-10T22:34:48.834Z| vmx| I125: WORKER: asyncOps=1 maxActiveOps=1 maxPending=0 maxCompleted=0

2017-02-10T22:34:48.839Z| vmx| I125: Vix: [90009 mainDispatch.c:4234]: VMAutomation_ReportPowerOpFinished: statevar=1, newAppState=1873, success=1 additionalError=0

2017-02-10T22:34:48.839Z| vmx| I125:

2017-02-10T22:34:48.839Z| vmx| I125+ Power on failure messages: File system specific implementation of LookupAndOpen[file] failed

2017-02-10T22:34:48.839Z| vmx| I125+ File system specific implementation of LookupAndOpen[file] failed

2017-02-10T22:34:48.839Z| vmx| I125+ File system specific implementation of LookupAndOpen[file] failed

2017-02-10T22:34:48.839Z| vmx| I125+ The file specified is not a virtual disk

2017-02-10T22:34:48.839Z| vmx| I125+ Cannot open the disk '/vmfs/volumes/18630e34-de98dd50-0000-000000000000/test/test_0.vmdk' or one of the snapshot disks it depends on.

2017-02-10T22:34:48.839Z| vmx| I125+ Module 'Disk' power on failed.

2017-02-10T22:34:48.839Z| vmx| I125+ Failed to start the virtual machine.

Reply
0 Kudos
takenalias
Contributor
Contributor

I am having the exact same problem. I can not browse the datastore ect. Would be good to hear from VMware on this issue.

Reply
0 Kudos
rmanning
Contributor
Contributor

I'm simply trying to migrate VMs from an older 5.5 hypervisor to a new 6.5 box and half of them get the same error you are seeing on startup (File system specific implementation of LookupAndOpen[file] failed).  No solution in sight so far, this is ridiculous...

am68
Contributor
Contributor

I encounter exactly the same issue with a NFS 4.1 volume on ESXi 6.5a when trying to browse the NFS v4.1 datastore using the ESXi Web Client (not vCenter).

Everything else works fine (creating and running VMs on the NFS v4.1 datastore, copying files, etc.) - only the datastore browsing in the Web Client seems to be a problem.

However, on a ESXi 6.0 U2 host, the datastore browsing works fine (when using the same NFS v4.1 shared volume).

It also works on both ESXi systems (6.5a and 6.0 U2) when ESXi connects to the NFS volume via NFS v3.

In my case, the NFS server is a CentOS 7.3 system and the NFS connection is using AUTH_SYS (as described here: VMware vSphere 6.5 Documentation Library).

Typically the NAS servers use the no_root_squash, as described here: VMware vSphere 6.5 Documentation Library‌.

The NFS export is defined as follows:

/nfsstorage     *(rw,sync,insecure,no_root_squash,no_subtree_check,no_wdelay).

Just wanted to note that I cannot reproduce JelleHissink's problem with a CentOS 7 based NFS server.

Reply
0 Kudos
ESantelices
Contributor
Contributor

I encountered this issue as well. I'm using an Intel NUC with a Synology in my home lab and ESXi 6.5. I encountered the problem when trying to Register VM's. Switching back to NFS v3 and the issue goes away.

Reply
0 Kudos
jojo
Contributor
Contributor

I have the same problem - esxi host webclient can't browse nfs 4.1 data storages. Version 3 of nfs data storage work fine. If I use the virtual center I can browse the v4.1 nfs storages too.

ESXi Host WebClient

Version 1.8.0

Build 4516221

ESXi

Version 6.5.0

Build 4887370

Regards

    Jochen

Reply
0 Kudos
yanivhs
Contributor
Contributor

I'm having the same problem as well

Using NFSv4.1 share from an EMC Unity 300 storage

Using SSH I can browse it and create files.

Also i was able to deploy VMs from OVF or by copying directly to the datastore using scp.

Any new ideas?

Reply
0 Kudos
enoch85
Contributor
Contributor

I'm also having this exact same issue on (Updated) ESXi-6.5.0-20170304101-standard (VMware, Inc.) + FreeNAS 11.0-U2.


I can read files created on the FreeNAS NFS share, but I can't delete, create or modify from the WebUI. Haven't tried to run a VM from that storage yet, but should be impossible as I can't write to the share.

Any help would be appreciated.

Reply
0 Kudos
solgaeDK
VMware Employee
VMware Employee

I have a similar issue, but a little more insight.

It seems there's an issue when using datastore browser in vSphere Web Client (or HTML5 client). The datastore browser works fine when you attempt to go into file browser (e.g. root directory of datastore), but if you attempt to browse to the VM files that are powered on, it hangs like the thread author described. However, the VMs themselves power on and operate just fine.

It seems when using datastore browser from vSphere Web Client (HTML5 Client), vCenter will use the last host that has access to the datastore (https://kb.vmware.com/s/article/1034573) as proxy for browsing datastores. When you browse to a VM folder that is powered on from that last host, the VM file browsing works fine. If the VM is powered on to any other host, the issue occurs. I'm guessing read requests are getting rejected when using datastore browser from the web client due to file lock.

This isn't exactly deal-breaking issue since the VM operations work fine otherwise, and you can still browse the files via shell access to ESXi host, but it's an annoying issue nonetheless. None of the issue happens on NFS 3.

Reply
0 Kudos
texcor
Contributor
Contributor

For those who still have this problem,

This is a webgui bug fixed in 1.10.0v (see changelog).

Upgrade procedure:

esxcli software vib install -v http://download3.vmware.com/software/vmw-tools/esxui/esxui-signed-latest.vib

Reply
0 Kudos