12 Replies Latest reply on Feb 16, 2012 4:04 PM by zotamus

    NFS Trouble (vSphere 4.0 and Windows Server 2008)

    BenignSage Novice

      Hello everybody. I am at my wits end trying to get a simple NFS  datastore working between vSphere 4.0 (ESX 4.0) and Windows Server 2008.  I was able to set this up in Windows Server 2003 R2 a few months ago  with none of the problems I am experiencing now. I have scoured the  internet for solutions, tried everything and have not been able to get  this to work. Please see below for more details:

       

      Current Set-Up

      Installed Windows Server 2008 File Services

      Installed Services for Network File System

      Created a new folder and shared it for NFS

      • share name: NFS
      • allow anonymous access checked
      • anonymous uid: -2
      • anonymous gid: -2
      • NFS Permissions:
        • [IP Address] Read-Write ANSI Root Access Allowed
        • ALL MACHINES Read-Only ANSI Root Access Disallowed
        • (I have even tried this with all machines root access and read-write allowed)
      • NTFS Security
        • ANONYMOUS LOGON - Full Control
        • SYSTEM - Full Control
        • CEATOR OWNER - Full Control (Special)
        • Administrator - Full Control

      Added datastore to ESX host via GUI

      • Host > Confiuration > Storage > Add Storage
      • Added NFS datastore with name of NFS
      • ERROR: A general system error occurred: Timed-out while waiting to get datastore information from host
      • Datastore appears "normal" in the list of datastores (green check mark, status: normal, capacity and free space correct)

      Tried removing NFS datastore from GUI

      • ERROR: The object has already been deleted or has not been completely created (due to error above)

      After refreshing more datastore objects begin appearing NFS(1), NFS(2), NFS(3), NFS(4), NFS(5) (and continues)

      Tried browsing datastore

      • Datastore appears empty although files exist within
      • Tried creating a folder
        • ERROR: Cannot complete file operation

      Connected to NFS share from another computer

      • Connected successfully
      • Could see files
      • If I clicked on a file it would disappear, a refresh would show it again, multiple refreshes would hide/show the file

      Removed datastore using CLI (success)

      Restarted management services to refresh the GUI (success)

      Datastores are now gone from host

       

      I hope the  above helps, I have tried to be as descriptive as possible. I have  tried so many different "solutions" to try to solve this so I may have  forgotten a few things. I noticed this time around that when I mounted  the datastore I could not see anything. In the past I COULD see files  (browse datastore in the GUI), but if I refreshed they would dissapear,  another refresh would make them appear. I could also (again previous to  this attempt) create a folder, but not delete it (from browse datastore  in the GUI).

       

      I remember when I set this up in Windows  Server 2003 R2 that there was a step where I needed to do something  either with the passwd file or user mapping or something. I am not sure  if this exists or not in 2008 as I do not recall performing this step  nor do I recall the option even being available (but they could have  renamed it like they did pretty much everything else).

       

      If there is any additional information you may need please don't hesitate to ask. I appreciate any assistance anyone can give.

       

      Thanks!

       

      Benign Sage

        • 1. Re: NFS Trouble (vSphere 4.0 and Windows Server 2008)
          kjb007 Guru

          Go into your local security policy, and make sure under Security Settings -> Local Polciies -> Security Options, that yo uhave 'Network access:  Let Everyone permissions apply to anonymous users' set to Enabled.  Reboot your server, and try again.

           

          -KjB

          • 2. Re: NFS Trouble (vSphere 4.0 and Windows Server 2008)
            BenignSage Novice

            KJB,

             

            Thanks for your reply. I followed your advice, changed the setting in the local security policy, rebooted the server, tried to add the datastore again and I am still having the same issues:

             

            Tried to add datastore

            • ERROR: A general system error occurred: Timed-out while waiting to get datastore information from host

            Datastores began multiplying (NFS (1), NFS(2), NFS (3))

            Browsed datastore

            • CAN see files/folders, however
              • folders show up as files
              • refresh many times and folders will show up as folders
              • can NOT delete folders/files
                • ERROR: Object reference not set to an instance of an object

            Used CLI to remove NFS datastore

            Restarted management services

             

            These seem to be some more of the same errors I was receiving the other day. I can only imagine that it is something small that I am missing here.

             

            Any other thoughts?

             

            Thanks!

             

            Benign Sage

            • 3. Re: NFS Trouble (vSphere 4.0 and Windows Server 2008)
              kjb007 Guru

              Did you have the Everyone group in the security settings to allow all permissions?

               

              -KjB

              • 4. Re: NFS Trouble (vSphere 4.0 and Windows Server 2008)
                BenignSage Novice

                KJB,

                 

                I did NOT have the Everyone group in my list of permissions. I configured the Everyone group to have Full Control of the folder, tried the same procedure mentioned previously (adding the datastore, testing it) and am still having the same errors occur.

                 

                Benign Sage

                • 5. Re: NFS Trouble (vSphere 4.0 and Windows Server 2008)
                  BenignSage Novice

                  Just as another note:

                   

                  I have checked my Event Viewer and I am noticing the following warning has begun to show up:

                   

                  Log Name:      Application
                  Source:        Microsoft-Windows-IDMU-PSync
                  Date:          9/22/2011 8:19:45 AM
                  Event ID:      8245
                  Task Category: None
                  Level:         Warning
                  Keywords:      Classic
                  User:          N/A
                  Computer:      STORAGE.mydomain.com
                  Description:
                  Password propagation is not done. Either default encryption key is configured or no UNIX hosts configured to propagate password
                  Event Xml:
                  <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
                    <System>
                      <Provider Name="Microsoft-Windows-IDMU-PSync" Guid="{E8DAF010-F14B-435F-BE59-93EDE773763C}" EventSourceName="NT to UNIX Password Sync Service" />
                      <EventID Qualifiers="49152">8245</EventID>
                      <Version>0</Version>
                      <Level>3</Level>
                      <Task>0</Task>
                      <Opcode>0</Opcode>
                      <Keywords>0x80000000000000</Keywords>
                      <TimeCreated SystemTime="2011-09-22T12:19:45.000Z" />
                      <EventRecordID>47216</EventRecordID>
                      <Correlation />
                      <Execution ProcessID="0" ThreadID="0" />
                      <Channel>Application</Channel>
                      <Computer>STORAGE.mydomain.com</Computer>
                      <Security />
                    </System>
                    <EventData>
                    </EventData>
                  </Event>

                  • 6. Re: NFS Trouble (vSphere 4.0 and Windows Server 2008)
                    kjb007 Guru

                    Are you still limiting your hosts in the NFS permissions?  Do you have multiple vmkernel ports on your ESX host that can reach the storage?

                     

                    -KjB

                    • 7. Re: NFS Trouble (vSphere 4.0 and Windows Server 2008)
                      BenignSage Novice

                      I still had the hosts limited by IP. I took those out and gave ALL MACHINES Read-Write and root access. I had an existing VMKernel port but created another one called NFS just to be sure. Tested connectivity to the file server successfully. Was able to mount the datastore successfully (unless it lies), however when trying to delete folders I received an error.

                       

                      Below are steps I have taken:

                       

                      Removed host-based permissions from the NFS share

                      Granted ALL MACHINES Read-Write and Allow Root Access

                      Created VMKernel port called NFS with a static IP address

                      Using the CLI, tested connectivity to the file server using vmkping

                      • Pinged successfully

                      Created NFS datastore on host

                      • Created successfully
                      • No error as before

                      Browsed datastore

                      • Created folder
                        • Success
                      • Tried to delete folder
                        • ERROR: Object reference not set to an instance of an object.

                      Double-checked NTFS permissions on NFS share

                      • Made sure that Everyone group had Full Control and that permissions were inherited
                      • Noticed the permissions on the folder created that Everyone had Read/Write permissions but not Delete/Full Control
                      • Tried manually assigning full control to the folder and deleting it from Browse Datastore, same error

                       

                      When I am browsing the datastore sometimes the folder I create does not show up, I have to refresh multiple times for it to show up, sometimes other folders disappear, sometimes they have a folder icon, sometimes it's a file icon.

                       

                      Getting really close now! Almost there!

                       

                      Benign Sage

                      • 8. Re: NFS Trouble (vSphere 4.0 and Windows Server 2008)
                        kjb007 Guru

                        Can you turn off permission inheritance on the NFS folder, so that the new folders take their permissions from what you set?  I'm running out of ideas at this point.

                         

                        -KjB

                        • 9. Re: NFS Trouble (vSphere 4.0 and Windows Server 2008)
                          BenignSage Novice

                          KJB,

                           

                          Thanks for all your input, it has helped to get me closer to solving the problem.

                           

                          I have turned off inheritance, double checked all permissions, Everyone, Anonymous Users both have Full Access. Tried deleting and recreating the datastore. Even tried it all from the command line. Mounts the datastore but can not modify/delete files.

                           

                          Benign Sage

                          • 10. Re: NFS Trouble (vSphere 4.0 and Windows Server 2008)
                            kjb007 Guru

                            Are you using a plain vanilla Windows 2K8 build, or is it a custom / hardened os?  Maybe there's additional security processes at work here?

                             

                            The service seems to be working, at this point, you are definitely running into a permission problem somewhere, I just can't see it.

                             

                            -KjB

                            • 11. Re: NFS Trouble (vSphere 4.0 and Windows Server 2008)
                              BenignSage Novice

                              I am wondering as well. I have a suspicion... one of the tutorials I went through for a "solution" suggested that the server needs to be a domain controller. I was hesitant, but after trying everything else made this server into a domain controller. The only thing I can think of is that being a domain controller has tightened security and caused these issues. The next chance I get I am going to revert all the changes I made (uninstall NFS services, demote it as a domain controller, etc.) and then try setting up NFS again. It's so close to working, I can only help but think it's something small causing these issues, it should not be this hard to set up. I appreciate all your input and will be referring back to it when I try to set up NFS again on this box.

                               

                              Thanks so much for your help!

                               

                              Benign Sage

                              • 12. Re: NFS Trouble (vSphere 4.0 and Windows Server 2008)
                                zotamus Novice

                                I don't know if this will help, but I found this article on troubleshooting NFS:

                                 

                                http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003967

                                 

                                Also this information might be what you are looking for specifically:

                                To make sure that you can create and manage datastores from your host, the NFS administrator must turn off the root squash feature or add the VMware Server host’s physical network adapter to the list of trusted hosts.

                                Syntax:

                                http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1005948

                                 

                                Other info:

                                http://www.vmware.com/files/pdf/VMware_NFS_BestPractices_WP_EN.pdf - page 11

                                 

                                Cheers,

                                 

                                Zot