VMware Cloud Community
RColbert
Enthusiast
Enthusiast

NFS Export from ESX 3 COS

I am trying to find a way to export an IDE partition from the console of an ESX 3.0 server. I can export it and have access via a UDP nfs session but for reliability, I want a TCP connection. Does anyone know how to make the console nfsd export via TCP? When I do an rpcinfo -p, it only shows UDP listeners for nfs.

Reply
0 Kudos
61 Replies
bbutler
Contributor
Contributor

I'm so close on this...

I've completed all the steps above, and have it "working".

I have successfully shared out /datastor on the my main partition.

So, then I've added a second 500GB IDE drive to my system (/dev/hdb1); and fdisk it to be one single partition.

I then created the filesystem on it.

I then issue the mount /dev/hdb1 /vmdata

Modified the /etc/exports to read as follows

/datastor *(rrw,no_root_squash,sync)

/vmdata *(rw,no_root_squash,sync)

And everything works... But, when I rebooted the system. The /vmdata mount no longer exists.

So... I edited the /etc/fstab file adding the line below

/dev/hdb1 /vmdata ext3 default 1 1

But, the same problem happens. If I go to the console, and re-mount the system by hand, and restart nfs. It works.

Any ideas?

Reply
0 Kudos
jjamieson
Enthusiast
Enthusiast

I did all of this, but then it said the server doesn't support NFS version 3 over TCP, which is required. Other then that, everything looks good - module loads,

What am I missing?

Reply
0 Kudos
jjamieson
Enthusiast
Enthusiast

Okay I guess the "built-in" nfsd.o was loaded, so that's why I got the error. Upon reboot, the problem dissapeared.

So, this works. However, for me, whenever I try to use the NFSd as my datastore, the service continually becomes unresponsive. It will completely lock up; this includes all networking and even any keyboard control on the console of the machine. Suddenly, it will become responsive again, and work for another 10 seconds, then hang again. This only happens when I start a VM that's located on the NFS store.

So, who knows. It all sucks. I can't use NFS nor iSCSI remotely, either. It's just a test box, and local SCSI works okay, so I guess that's what I'll have to live with. You can get refurb 180GB SCA disks for $89 on pricewatch.

Reply
0 Kudos
magohl
Contributor
Contributor

Anybody who has got this to work on 3.01? I have followed the suggestion exactly by using copy/paste but i can not get it to work.

I get the infamous: nfsd.o -- unresolved dependency issues

Any ideas? Thanks in advance!!

Reply
0 Kudos
jjamieson
Enthusiast
Enthusiast

I haven't upgraded to the new 3.0.1 patch, so I don't know if there is an update to the service console kernel. If there was, it's hit or miss whether or not you'll have problems.

Basically, it's like this with Linux kernels:

\- The linux kernel is monolithic. This means that all kernel functions and all device control software (drivers) are integrated into one large kernel. Kernel Modules were created so that you can load and unload parts of the kernel on-demand, as needed. Kernel modules are generally compiled with the kernel that you'll be running.

\- Different kernel options may provide different functions. If a kernel module is compiled against a kernel with support for certain things, the module may make use of those functions. If they are missing from the kernel, it may not. The problem this creates is that if you build a module that is compiled against a kernel not configured in the same way, there's a chance that the functions the module needs are not supported in the kernel. This can cause an unresolved dependency if you try to run it.

\- Different kernel versions can cause all sorts of problems even if they are configured the same way.

The version of the service console linux kernel is: (from uname -a)

2.4.21-37.0.2.ELvmnix #1 Tue Jun 13 01:24:07 PDT 2006 i686 i686 i386 GNU/Linux

If they updated the kernel in 3.0.1, but not kernel sources, this could be a problem for you.

If not, you just need to make sure you follow the instructions in bringing in the kernel configuration options, update the custom version, make dep and make modules, as specified in the guide.

Copy the nfsd.o into /lib/modules/2.4.21-37.0.2.ELvmnix/kernel/fs/nfsd, overwriting the one that's there.

Then do a depmod -a. This will update all the dependency information about the kernel modules.

Make sure the old nfsd.o isn't loaded.

If you do an lsmod, you'll see a list of loaded modules:

Module Size Used by Tainted: P

nfsd 86192 0 (unused)

lockd 63024 0 \[nfsd]

sunrpc 90964 0 \[nfsd lockd]

sg 36612 0 (autoclean)

loop 12432 0 (autoclean)

ipt_REJECT 5152 1 (autoclean)

ipt_conntrack 2144 1 (autoclean)

ipt_state 1536 26 (autoclean)

ip_conntrack 30272 2 (autoclean) \[ipt_conntrack ipt_state]

ipt_LOG 4576 1 (autoclean)

iptable_filter 2848 1 (autoclean)

ip_tables 16416 5 \[ipt_REJECT ipt_conntrack ipt_state ipt_LOG iptable_filter]

vmxnet_console 21940 1

keybdev 3360 0 (unused)

mousedev 5876 1

hid 23328 0 (unused)

input 6848 0 \[keybdev mousedev hid]

ehci-hcd 20576 0 (unused)

usb-uhci 26308 0 (unused)

usbcore 81248 1 \[hid ehci-hcd usb-uhci]

vmnixmod 391532 18 \[vmxnet_console]

If you see nfsd, do an rmmod nfsd. It should be removed from the list. Then do a modprobe nfsd. See if it loads - if you get no error, that's a good sign. Do another lsmod and see if it's there.

If it doesn't work still, reboot. It's possible another module is loaded causing the issue, and will clear out on a fresh reboot after the depmod -a and new nfsd.o.

Other things to check are to be sure your kernel configuration has been properly set up. After you import the configuration as outlined in RColbert's post, do a make config and make sure nfsd is enabled, with TCP and V3 support. I find that make menuconfig is a lot faster to use since you don't have to skip past a few hundred other prompts first. (You need ncurses-devel for this, and you can use the CentOS 3.7 version HERE: http://centos.osmirror.nl/3.7/os/i386/RedHat/RPMS/ncurses-devel-5.3-9.4.i386.rpm)

NFS is in File Systems > Network File Systems. You can see if the nfsd is enabled with TCP support. It's listed as EXPERIMENTAL.

Check all these things and I'm sure you can get a good compile. Once you do, I would be curious to see if you run into problems like I did - with terrible performance and system "pauses" while using the exported NFS folders. It was basically unusable for me.

For testing, I found that a second box running Microsoft's NFS (part of SFU 3.5) was the easiest to get running with very good performance. There's a guide for this out there too, on that RTFM website. It's very easy, and runs on XP.

Note that you can use the same method of creating the nfsd.o for creating other modules you want the service console to support. Not all of them will work, of course, since it's vmware and the service console is all in league with the vmkernel. I wouldn't expect a lot of hardware drivers to work. However, you can add support for things like read-only NTFS for if you want to import files or vm disks from a USB hard drive formatted with NTFS.

Reply
0 Kudos
VMboxes
Contributor
Contributor

There is a typo on:

10. On console - exported the nfsshare

nano /etc/exports

/nfsroot 192.168.1.200/255.255.255.0(rw, sync, no_root_squash)

it should be

/nfsroot 192.168.1.200/255.255.255.0(rw,sync,no_root_squash)

there should't be any spaces

Reply
0 Kudos
rtrotman
Contributor
Contributor

Can I get a copy of nfsd.o send to rtrotman@unfcu.com

thanks

Reply
0 Kudos
lirazoo
Contributor
Contributor

please , i need the compiled nfsd.o too.

please mail to : tamir.bar-hai@mailpoalim.co.il

Thanks!!!!

Reply
0 Kudos
xAyiDe
Enthusiast
Enthusiast

Can someone mail me the system.map and nfsd.o ?

Reply
0 Kudos
xAyiDe
Enthusiast
Enthusiast

Btw did anyone get this to load on any laptop?

Reply
0 Kudos
RColbert
Enthusiast
Enthusiast

Fully patched (with all system patches up to 2/1), the VMNIX_BUILD number is 35804.

This number can be found in the VI3 console next to the server version number. i.e. VMware ESX Server, 3.0.1, 35804.

The VMNIX_BUILD definition in ./includes/linux/version.h is important, so don't leave it out.

Reply
0 Kudos
devaholik
Contributor
Contributor

can anyone email me the system.map and the nfsd.o; no matter what i do i can't get the daemon to start and it has resolve issues.

devaholik@yahoo.com

Reply
0 Kudos
devisez
Contributor
Contributor

Hi Madvenu,

Is there any chance of you sending me the nfsd.o and map file please?

At the same time, has anything major changed since all this was being discussed...

Thanks

R Lowery

Reply
0 Kudos
Paul_Lalonde
Commander
Commander

Check out my post in the ESX 3 Configure forum. Nice streamlined way to set up NFS with no dependency issues.

Paul

Reply
0 Kudos
utkinpol
Contributor
Contributor

Hi,

could anybody PLEASE forward working nfsd.o to utkinpol@yahoo.com email and please explain how did you get it compliled properly on 3.0.1 32039 ESX?

I am not new to linux and do everything correctly but still get this:

depmod: *** Unresolved symbols in /lib/modules/2.4.21-37.0.2.ELvmnix/kernel/fs/nfsd/nfsd.o

depmod: nlmsvc_ops

depmod: lockd_down

depmod: xdr_decode_string_inplace

depmod: svc_reserve

depmod: xdr_decode_string

depmod: svc_exit_thread

depmod: svc_proc_unregister

depmod: nlmsvc_invalidate_client

depmod: lockd_up

depmod: xdr_encode_array

depmod: svc_makesock

depmod: svc_destroy

depmod: svc_create_thread

depmod: svc_recv

depmod: svc_process

depmod: svc_create

depmod: nfsd_debug

depmod: svc_proc_register

depmod: svc_proc_read

Reply
0 Kudos
utkinpol
Contributor
Contributor

As I can see generally unresolved line differ this way:

U vfs_mknod_Rvmnix_6beee001

in original nfsd.o file

U vfs_mknod

in recompiled file.

same applies to rest of unidentified symbols - they all lack "\_Rvmnix_" suffix and number at the end. I cannot recall or may never knew how exactly to fix such things. Does anybody here know about this?

Reply
0 Kudos
joliveira
Contributor
Contributor

Hello,

Can you email me the nfsd.o ?

I was starting investigating this and have an "old" machine work as a test ESX3.

Please e-mail me to jorge.maroliv@sapo.pt (if you have updated instructions, please add tehm)

Tks in advance and best rgds

Jorge

Reply
0 Kudos
MR-Z
VMware Employee
VMware Employee

wow, the first I tried the procedure it failed. SO I ran it again, with the extra "make mrproper" and it worked!

I did not copy any system.map. In fact, I could not find that file outside of /boot.

Reply
0 Kudos
Erik_Zandboer
Expert
Expert

Hi,

I have been running this setup for almost half a year now on my testbox at home. I had an ESX 2.5 server with SCSI, and needed more cheap storage. When upgrading to 3.0, I mounted a 250GB IDE disk. recompiled NFS to support TCP, and installed the iSCSI target "iSCSI Enterprise Target 0.3.8". You have to use THIS version because later version only support 2.6.

Just put the VMkernel and the console on the subnet on the same vswitch, and physical network will not even matter Smiley Happy

I must say iSCSI works, but is slow as hell (<1MBYTE/sec). NFS on the other hand, is very much useable (at least for me), it measures about 40MBYTES/sec using hdparm, I get around 4-8 MBYTES per second out of a single 7200rpm ATA disk when I copy files from or to it. I had to manually enable DMA on the /dev/hda device though.

Credits do not go to me, but to Reinout Wijnveen. I used his manual to get there:

http://www.vmug.nl/downloads/VMWare_NFS_en_iSCSI.pdf

Unfortunately it is in dutch, but I think you could manage Smiley Happy

Message was edited by:

Erik Zandboer

Visit my blog at http://www.vmdamentals.com
Reply
0 Kudos
SHBECKER
Contributor
Contributor

Hi,

I've done this all and NFS is working fine on ESX 3.0.2 for some minutes. But suddenly the NFS-share becomes always inactive. The message I can see in /var/log/vmkernel is:

WARNING: NFS: RPC error 13 (RPC was aborted due to timeout) trying to get port for Mount Program (100005) Version (3) Protocol (TCP) on server (my IP-Address).

Do you have an idea? The output of rpcinfo -p:

100000 2 tcp 111 portmapper

100000 2 udp 111 portmapper

100003 2 udp 2049 nfs

100003 3 udp 2049 nfs

100003 2 tcp 2049 nfs

100003 3 tcp 2049 nfs

100021 1 udp 32770 nlockmgr

100021 3 udp 32770 nlockmgr

100021 4 udp 32770 nlockmgr

100021 1 tcp 32777 nlockmgr

100021 3 tcp 32777 nlockmgr

100021 4 tcp 32777 nlockmgr

100005 1 udp 808 mountd

100005 1 tcp 808 mountd

100005 2 udp 808 mountd

100005 2 tcp 808 mountd

100005 3 udp 808 mountd

100005 3 tcp 808 mountd

Thanks for your help!

simon

Reply
0 Kudos