VMware Cloud Community
formulator
Enthusiast
Enthusiast

People using FreeBSD on ESX 3

Just some questions for those doing this:

1. what version of freebsd?

2. vm in production? if so how busy is the vm and how long has it been up for on ESX 3.

3. any issues so far?

4. any special configuration mods to make it work?

Thanks

0 Kudos
37 Replies
DavidDPD
Contributor
Contributor

First try last night at Installing FreeBSD 6.1 (32bit) in a VM on ESX 3 (full I3 install) on a Dell PE 850 with CERC 6ch SATA board, dual 250gig SATA HD, pentD 820.

FreeBSD install CD booted, however it would not detect the VM SCSI drive. It did dectect the controller, but not the drive. Adding an IDE Vitural disk was not an option that was allowed. FreeBSD 6.x works just fine in VMware Server 1.0 (free download) on both Windows and Linux with no tinkering.

0 Kudos
jlanders
VMware Employee
VMware Employee

You'll need FreeBSD-6.1-STABLE, not FreeBSD-6.1-RELEASE.

There were changes checked into STABLE in July that work

around an issue with our mpt SCSI emulation on ESX and

should be available in future FreeBSD releases as well.

Thanks,

Joe

0 Kudos
formulator
Enthusiast
Enthusiast

Thanks for the replies. Anyone know how I can get the specifics on these changes? I'm working on migrating from ESX 2 to 3 but we have a number of 4.10-RELEASE-p16 guests. I guess this means I need to upgrade these guests as part of the migration unless I can get the new mpt driver for 4.10-RELEASE-p16.

0 Kudos
formulator
Enthusiast
Enthusiast

I'm currently testing the below configuration:

guest: freebsd 4.10-RELEASE-p16 using Buslogic SCSI controller

host: ESX 3.0.1

Will run a disk io stress test overnight and post results.

0 Kudos
jlanders
VMware Employee
VMware Employee

I was able to run a 4.9-RELEASE guest with both buslogic and lsilogic on ESX 3.

You can browse the FreeBSD CVS tree online. Look at src/sys/dev/mpt/mpt.c.

Revision 1.3.2.5 was the update for RELENG_4.

I don't have a 4.10-RELEASE-p16 guest handy.

But, please let us know your results.

Thanks,

Joe

0 Kudos
formulator
Enthusiast
Enthusiast

I've been running stress 0.18.8 with the following options:

--io 7 --hdd 7

inside this FreeBSD 4.10-RELEASE-p16 guest with Buslogic SCSI adapter for about 23 hours now. VM is averaging 82% CPU usage, 60536KBps of disk usage and 91% memory usage over the last 24 hours according to VC 2.0.1. Everything seems to be fine. I'm probably going to run it for another day or so before considering it stable for migration. Can you think of any other tests I should run before migrating this production VM? Any issues that you've heard about with FreeBSD guests and network IO (lnc driver)?

Any input would be greatly appreciated. Thanks.

0 Kudos
jlanders
VMware Employee
VMware Employee

The FreeBSD's lnc driver tends to drop packets under load. You may want to

download VMware Server (it's free), extract the FreeBSD tools and use the

vmxnet driver in your guest (ethernet0.virtualDev = "vmxnet" in the config file,

vxn in FreeBSD). You could also try the e1000 (ethernet0.virtualDev = "e1000"

in the config file, em in FreeBSD).

Keep in mind that ESX 3.0 doesn't officially support FreeBSD.

Joe

0 Kudos
formulator
Enthusiast
Enthusiast

I did notice that this FreeBSD guest on ESX 3 hasn't logged the classic "/kernel: lnc0: Missed packet -- no receive buffer" message like it does on ESX 2 but I only did some very basic network IO tests. I'll look into that further and maybe try out the em driver, I've had problems with using vmxnet before which is why I'm currently using lnc.

Thanks again.

0 Kudos
jlanders
VMware Employee
VMware Employee

Well, if you do get a chance to try the tools from VMware Server on your ESX

FreeBSD guest, please let us know the results. We've fixed some bugs and

the tools from VMware Server will be the more recent than the ones from ESX 3.

Joe

0 Kudos
formulator
Enthusiast
Enthusiast

I've finished my testing. In the end I let the disk IO stress tool run for about 35 hours and had no issues. After that I went on to do some network IO tests. I started with the lnc driver and downloaded a 500MB file over the Gbit network and averaged about 30MB/sec over 4 or 5 tries, I did notice the standard "no receive buffer" error being logged after these test runs. After that I switched to the em driver and did the same tests this time I averaged 54MB/sec and received no errors.

So, the testing seems to have gone much better than I had expected, it seems that these VMs actually perform better on ESX 3. I decided to migrate one of our production guests over, it's been running since last night and hasn't had any issues so far.

I haven't tried the vmware server version of tools because the version that came with ESX 3.0.1 worked so well. The version that comes with the latest vmware Server is newer than the ESX 3.0.1 version? Is it possible to get a list of what changes/improvements were made in the newer tools version? I may be able to dedicate a couple of test VMs to testing the new version if it sounds like it's worth it to test the improvements. However the current version does seem to work well.

Thanks,

Colin

0 Kudos
jlanders
VMware Employee
VMware Employee

That's good to hear.

The ESX 3 tools are from an older code-base than the VMware Server tools.

There haven't been many changes, just some bug fixes and adding support for

FreeBSD 6.x.

If you're happy with the performance results, I don't see a reason to change.

Thanks,

Joe

0 Kudos
andrew_hald
Hot Shot
Hot Shot

This needs to change. There are enough of us out here that need this. We have >100 FreeBSD VMs in our production environment. The migration to 3.0 makes us officially unsupported now. What are the current prospects of getting FreeBSD 6 support in the next point release?

0 Kudos
jlanders
VMware Employee
VMware Employee

Yes, I agree and understand. Unfortunately, I can't say for certain when it

might be supported again. Generating an SR will help. So will contacting

your sales rep.

Joe

0 Kudos
darkuncle
Contributor
Contributor

Andrew's right ... official FreeBSD support would really help to offset some of the scaling problems we've had deploying our new production VI3 environment (ESX3 scales just fine; VirtualCenter is another story altogether.)

0 Kudos
andrew_hald
Hot Shot
Hot Shot

Both have been done. We are going big with our FreeBSD deployment and need VMware to back us as we do. Thanks. Smiley Happy

0 Kudos
oreeh
Immortal
Immortal

FYI: the VMware server tools (especially the vmxnet driver under heavy network load) are not stable with FreeBSD 6.x.

0 Kudos
jlanders
VMware Employee
VMware Employee

Do you have any better luck using the e1000 (em) driver in your FreeBSD guest?

Joe

0 Kudos
oreeh
Immortal
Immortal

the e1000 has some problems too with heavy network loads (at least with ESX 3.0.0)

there seems to be a fix in ESX 3.0.1 but I didn't had the time to test it

at the moment I prefer the lnc0 with all its known problems

0 Kudos
jlanders
VMware Employee
VMware Employee

Yes, some e1000 problems were fixed for ESX 3.0.1, so please give it a try.

Joe

0 Kudos