VMware Cloud Community
wmarques
Contributor
Contributor
Jump to solution

FTP on ESXi 3.5??

I have been bangin my head against our VM server for a while now trying to enable FTP so that we can back up the virtual machine files to another location in case the ESXi server were to meet an untimely end. I do not have the consolidated back up option. I have read multiple documents about removing the # comment from the inetd.conf file and then restarting the services or rebooting the server. I have done all of this and still can not get the ftp to work. I have managed to connect to the console using SSH, but no luck with ftp. When I try to FTP it returns the following...... -ash: ftp: not found

Anyone have this problem?

0 Kudos
114 Replies
Dave_Mishchenko
Immortal
Immortal
Jump to solution

The oem.tgz file here has been updated - http://www.vm-help.com/esx40i/customize_oem_tgz.php.

0 Kudos
SMAI
Contributor
Contributor
Jump to solution

FTP works great on my ESXi 3.5 U4.

I search a "modified" ghettoVCB script.

wget is a ftp download tool, right ?!

wput is a ftp upload tool, right ?!

Has anyone replaced the nfs option in a ftp option in the ghettoVCB.sh and wput installed ? So you can backup the VM andupload directly to the ftp store. Thats a nice thing.

When not, i will change the ghettoVCB script.

0 Kudos
birnenschnitzel
Contributor
Contributor
Jump to solution

Maybe the attached wput will help.

Best regards

0 Kudos
SMAI
Contributor
Contributor
Jump to solution

Hello,

thanks for the fast answer.

0 Kudos
Dave_Mishchenko
Immortal
Immortal
Jump to solution

Attachements aren't sent by email. If you look at the thread on the VMware Communities page there is an attachment there.

0 Kudos
IT_Architect
Enthusiast
Enthusiast
Jump to solution

I will be staging another ESXi machine in the next week or two. With regard to Ghetto, would I be better off going with ESXi 3.5u3 as I have been using, or use ESXi 4.0 that I have no experience with, but may have some advantages that I'm not aware of such as network speeds? Has anyone here worked enough with 4 to know what the pros and cons are?

Thanks!

0 Kudos
IT_Architect
Enthusiast
Enthusiast
Jump to solution

I hate to be the bearer of bad tidings, but I discovered one big con of upgrading to ESXi 4.0. Not only are your wings clipped as compared to 3.5U3, SH_FTP_oem_1.3.tgz doesn't work with at least the latest ESXi 4.0, 164009. Everything seems to work fine with the exception that the inetd.conf does not get replaced at boot. Of course that is key to getting proftpd loaded on boot. If I manually edit the inetd.conf and then HUP inetd, it works fine. The tgz looks perfect. It is a clean install of 4.0 build 164009 from an ISO to disk.

One machine is a SuperMicro X8DTU for ESXi and the processor is a Nehalem X5570 with 6 gigs. The interfaces are IPMI/KVM, and a couple Intel NICs. The controller is an Adaptec 5405 SAS with Intel SLCs in a RAID-0. At first I wondered if it had something to do with the new extensions in the Nehalem (Some pertain to security), or a timing issue.(Solid state drives) However, I tried it again today with a another more conventional machine. This one was a SuperMicro Harpertown 5430 with 4 gigs, and a SCSI array. Both are ESXi compatables.

One thing different about the inetd.conf situation is it is the only file that overwrites another during boot. It also has a sticky bit set, but I wouldn't think that any process that deploys the oem.gz boot would be short on rights. What is almost certain is the current SSH_FTP_oem_1.3.tgz will no longer add ftp capabilities to the current version of ESXi 4.0 without manually changing the inetd.conf after each boot.

A request: If you make up a new version the works around the problem. add wput to the mix. That would be very useful for backup scripts to automate transfering backup directories to another machine on the private VLAN.

Thanks!

PS: Solved! I found this little "zinger" on the web site that stores oem.tgzs.

"Note: if you have previously updated inetd.conf and then added one of the oem.tgz files, then that copy of inetd.conf in state.tgz (or local.tgz for ESXi Embedded) will overwrite the copy in oem.tgz. If that is your case, then you'll need to manually edit inetd.conf and restart the host."

Of course that would be about everybody since you would enable ssh. Thus the cure is to copy the contents in the inetd.conf inside of SSH_FTP_oem_1.3.tgz and paste it into the one in /etc and it will survive reboots.

0 Kudos
MarkW2009
Contributor
Contributor
Jump to solution

Does anybody have any advice on getting ftp to work with ESXi 3.5 U4 Embedded ? (I've got ssh to work fine)

I scp-ed the 'proftpd' package from and followed the instructions. No luck - though it does create an empty /var/run/proftpd_trace file.

Thinking it might be a problem with the proftpd part, I took the simpler option of just using the tcpd part. I set this line in /etc/inetd.conf ...

ftp stream tcp nowait root /usr/sbin/tcpd in.ftpd

Then I bounced inetd with ...

kill -HUP <pid>

No matter what I try, I always get this at the ftp client:

prompt> ftp 192.168.0.200

Connected to 192.168.0.200.

421 Service not available, remote server has closed connection

I do notice that 'find / -name 'ftpd' does not return anything other than the 'proftpd' stuff - is this a problem?

To what does 'in.ftpd' refer?

0 Kudos
MarkW2009
Contributor
Contributor
Jump to solution

Follow up to previous post ...

The ftp issue might be related to the fact that none of my Linux machines can resolve machine names on the LAN (their own names included). This includes the ESXi host, which fails the final step in the "Test Management Network" for this reason. I notice when I run 'proftpd' at command line that it complains about not being able to resolve the ESXi machine name - it cannot find the IP address - then it gives an error that it cannot find any servers, then an error processing the proftpd.conf file. But it might all be because it cannot resolve the ESXi machine name.

My Windows machines (even virtual machines) can resolve local machine names (but not with the DNS suffix attached). I'm not running a domain - just a 'workgroup' (whatever that is).

If anybody knows how I might get Linux machines to resolve local machine names - please let me know.

Oh well ... back to Veeam ...

0 Kudos
MarkW2009
Contributor
Contributor
Jump to solution

OK - got it working.

It just needed an entry in the /etc/hosts file so the ESXi host could find itself without relying on a DNS server.

0 Kudos
IT_Architect
Enthusiast
Enthusiast
Jump to solution

Thanks huge for sharing MarkW2009

I'm going down that road right now and can benefit from any advice you have or post a little How-To.

Thanks!

0 Kudos
MarkW2009
Contributor
Contributor
Jump to solution

Happy to help if I can, IT_Architect. I had to fight with this for a day or so. The following just reflects my current understanding - usual caveats apply.

The 'what' was pretty easy; the 'why' was a little more interesting.

Basically, I downloaded and used the 'proftpd' package from (11th post in this thread) but I needed to do a bit extra to get it working in my environment. I'm using ESXi 3.5 U4 Embedded (the free version) in a home office with no local DNS. We use the ISP's DNS servers for internet lookups, but so far as I know we cannot resolve our own hostnames on their DNS servers. For Windows machines in our Workgroup that's OK because they use WINS, with NetBIOS lookups on our gateway (i.e. our LAN router - e.g. 192.168.0.1). But for Linux it's different - now there may be a way to deploy WINS on Linux to get it to do NetBIOS lookups as well, but I didn't want to risk messing with ESXi too much. So for example, the ESXi server has a <hostname> and it grabs the <DNS suffix> from the ISP's DNS servers, but when I do "Test Management Network" and it trys to resolve <hostname>.<DNS suffix> this step always fails.

Here's what I did:

1) Download the 'proftpd' package, unzip it and read the instructions from the post referred to above.

2) Get into the ESXi Tech Support Mode (it's Linux command-line). Edit (vi) the /etc/inetd.conf to remove the comment from '#ssh' to enable ssh and scp, and include the 'ftp' line for the above package:

ftp stream tcp nowait root /usr/sbin/tcpd proftpd

3) THIS IS THE EXTRA BIT: On the ESXi server, edit (vi) the /etc/hosts file to include the line:

127.0.0.1 <hostname>.<DNS suffix>

Make sure the <DNS suffix> is exactly what you see on the ESXi Management Console (including any '.' at the end).

4) Deploy the 'proftpd' package to the ESXi server. I edited out the anonymous login information at the end of the proftpd.conf file. I use a Linux client to connect to ESXi and deploy with the following script:

#!/bin/sh

echo copying proftpd files...

scp tcpd proftpd proftpd.conf root@<ip-address>:

echo deploying proftpd files...

ssh root@<ip-address> << eof

chmod 644 proftpd.conf

mv proftpd.conf /etc

mv tcpd proftpd /sbin

ps | grep inetd | awk '{ print \$1 }' | xargs kill -HUP

eof

echo done

5) Use 'ftp' clients to connect to the ESXi server.

Note: If you're using 'andLinux' like me, you'll need to 'set passive' in your ftp session because the 'andLinux' connection looks like it's coming from it's Windows host and the ESXi server cannot make the ftp 'data' connection back to 'andLinux'.

Note: The edits to /etc/inetd.conf and /etc/hosts get remembered through ESXi restarts. So it's just necessary to redo step 4).

Here's why I did it:

a) The infamous 'i/o errors' trying to do VI Client datastore downloads of large VMDK files.

b) The RCLI vifs.pl uses HTTP GET, but eventually gave me 500 read errors.

c) Internet Explorer on the '<ip address>/folder' could not download VMDK files -> bad HTTP error

d) Veeam Backup and FastSCP is fast and can work fine, but for me, it too had it's moments - probably because my Windows XP machine is so flakey (it's a cow).

e) The ftp server daemon must be able to resolve it's own hostname. Luckily for me, ftp appears to use 'nslookup' to resolve this.

Test this by doing 'nslookup <hostname>.<DNS suffix>' from the ESXi server - it should be able to find itself now.

Note that my ESXi "Test Management Network" still cannot resolve <hostname>.<DNS suffix> even though /etc/host.conf says 'order hosts,bind' - so I assume that it's trying to bind at the ISP's DNS server regardless.

f) I had read on the internet about uncommenting the existing '#ftp' line in the /etc/inetd.conf file, but this didn't work for me. I'm not entirely surprised because I cannot see what 'in.ftpd' refers to - maybe I'm missing something. I am a bit curious about this though...

g) I get ftp speeds similar to Veeam Backup and FastSCP - about 11.5 MBps. Good enough for me.

Hope this helps somebody.

0 Kudos
IT_Architect
Enthusiast
Enthusiast
Jump to solution

I get ftp speeds similar to Veeam Backup and FastSCP - about 11.5 MBps. Good enough for me.

I'll download it and read over what you posted. Something that surprises me in your post though is 11.5 MB/s. When I copy between servers using Veeam FastSCP, it get 22MB/s, or 31 minutes to transfer a 40 gig VM. The Windows 2003 machine where FastSCP is running from is a VM on the same machine. I would think FTP and WPUT, it would be quite a bit faster. I switched to from ESXi 3.5u3 to 4i. Maybe that's the difference.

It surprises me you had to maintain the host file. From within the VI Client, you should be able to go to Configuration / DNS and Routing, or whatever you put in the ESXi console window during setup should have gotten written to the hosts file.

Thanks!

0 Kudos
MarkW2009
Contributor
Contributor
Jump to solution

Hi,

The ESXi server machine has two GigE NICs.

I use the ftp client and the Veeam FastSCP from a separate machine running Windows XP. It has firewire-connected external storage that I use for VMX/VMDK file backups.

The NIC in the XP machine is an Intel PRO/100 VE Network Connection. This NIC should be capable of 100 Mbps.

The ESXi server and the XP machine are both connected at a Linksys 10/100 5-Port Workgroup Switch. This switch claims to auto-detect the port speeds independently, either 10 or 100 Mbps. But I've no idea how it does this. My understaning is that the switch should pass IP traffic directly from one connected port to another. I just hope it's not putting everything through the LAN router.

Feel free to let me know if you see anything in this set-up that would cause the 11.5 Mbps top speed.

Thanks,

Mark

0 Kudos
IT_Architect
Enthusiast
Enthusiast
Jump to solution

At the moment I'm a bit puzzled. I setup ftp and I'm transfering a flat.vmdk between servers with wput as we speak. The hardware is a Harpertown 5430 with battery backed up SCSI array on one end and the top dog Nehalem on the receiving end with solid state raid-0 on the other. HOWEVER, they are actually copying from one backup sata to the other's backup sata. They both have GIG-E adapters and a gig private network between them. I'm getting 15MB/s with ftp, and 22 with Veeam FastSCP. That doesn't make sense unless Veeam FastSCP uses some sort of on-the-fly compression. There is a boatload of space on the receiving drive as well. I don't believe this. I gotta try it again with FastFCP. Even if they worked out to be the same, FastSCP pre-allocates to prevent fragmentation. That would also be a lot easier on the receiving ESXi host. However, I wouldn't THINK that would be the bottleneck, but I'm open to being wrong. I was hoping for wput to be the answer because it's scriptable and I could throw it ghettoVCB. It isn't working out this way for everyone because I read some getting 33, which is what I fully anticipated I would get. I'm not getting half that at the moment.

Update:

I followd up immediately with a FastSCP copy and it's doing 11MBs now. I'm not sure why that would be. It might be telling me that ftp is indeed faster, but I think when I figure out why 22 one time and 11 the next, I'll get my 30 to 33MBs too. These are production servers. The Nehalem does 66 php pages a second with database reads, and 1/4 of the traffic is inbound when it assembles data for people and creates highly optimized png maps. Still, we're talking copying files here and on a different interface, and it take a lot more than this to stress a Nehalem. The Harpertown is not in the same league as a Nehalem, but the Harpertown isn't doing anything. The only thing the sites use the private interface for is rapid access to resolvers. It's got to be something else. The loads on the machines are also a little lower with FastFCP probably because it is tranferring slower. I'm going to have to think about this for awhile. I'll try a SCSI array to Solid State drive later but it's tough to imagine a drive or cpu not keeping up with a NIC. I may have copied in the other direction before and that was sata to sata as well. I'll try that too. I'm still at a loss.

Update:

The same file going from the Nehalem to the Harpertown via wput gives 25.8. I have no idea why. This is most likely the direction that I used earlier with FastSCP. FastSCP does use some form of compression. Neither solution doesn't look so hot. I'm going to look at rsync next so I can use deltas instead of passing the whole file. Something else I'm not fond of is that, unlike with vmkfstools, it ends up thick at the destination.

Update:

It doesn't look as though I'll be able to use this technology. I set up automatic backups to have VMs send their control panel backups to the volumes on the opposite server across the local VLAN. The VMs on both servers become almost completely unresponsive during the backup even when only one control panel is backing up at a time. The only difference I see is that I'm using ESXi 4. I'm going to look at iSCSI. That way I get thin provisioning for my backups as well.

0 Kudos