VMware Cloud Community
nick_couchman
Immortal
Immortal

GPL Components - Build ESX Drivers?

I recently downloaded the GPL components from the VMware VI3 download site. One of the tarballs in that file is the Open Source pieces of the ESX drivers. Along with this is a script called build-vmkdrivers.sh. I've tried running this script, and it results in many, many errors and absolutely no correct builds. I'm running it on a CentOS 3.9 system, similar to the RHEL3 base that is used for the Service Console, and I've tried using GCC 3.2.3 (included in CentOS3) and GCC 3.3.6, to no avail. Here's a very small sample of the types of errors I get:

    • Compiling drivers/block/cciss/cciss.c

In file included from src/include/linux/module.h:11,

from src/drivers/block/cciss/cciss.c:24:

src/include/linux/spinlock.h:124: error: syntax error before '*' token

src/include/linux/spinlock.h: In function `vmk_spin_is_locked_by_my_cpu':

src/include/linux/spinlock.h:126: error: `lock' undeclared (first use in this function)

src/include/linux/spinlock.h:126: error: (Each undeclared identifier is reported only once

src/include/linux/spinlock.h:126: error: for each function it appears in.)

In file included from src/include/linux/module.h:11,

from src/drivers/block/cciss/cciss.c:24:

src/include/linux/spinlock.h: At top level:

src/include/linux/spinlock.h:230: error: syntax error before "spinlock_t"

In file included from src/include/linux/module.h:11,

from src/drivers/block/cciss/cciss.c:24:

src/include/linux/spinlock.h:237: error: syntax error before "spinlock_t"

src/include/linux/spinlock.h:239: error: syntax error before '}' token

In file included from src/include/linux/fs.h:12,

from src/include/linux/capability.h:17,

from src/include/linux/binfmts.h:4,

from src/include/linux/sched.h:10,

from src/include/linux/mm.h:22,

from src/include/linux/slab.h:14,

from src/drivers/block/cciss/cciss.c:29:

src/include/linux/wait.h:90: error: syntax error before "spinlock_t"

src/include/linux/wait.h:102: error: syntax error before '}' token

src/include/linux/wait.h: In function `init_waitqueue_head':

src/include/linux/wait.h:194: error: dereferencing pointer to incomplete type

src/include/linux/wait.h:199: error: dereferencing pointer to incomplete type

src/include/linux/wait.h:199: error: dereferencing pointer to incomplete type

src/include/linux/wait.h:199: error: dereferencing pointer to incomplete type

src/include/linux/wait.h:199: error: dereferencing pointer to incomplete type

src/include/linux/wait.h:205: error: dereferencing pointer to incomplete type

src/include/linux/wait.h: In function `waitqueue_active':

src/include/linux/wait.h:253: error: dereferencing pointer to incomplete type

src/include/linux/wait.h: In function `__add_wait_queue':

src/include/linux/wait.h:266: error: dereferencing pointer to incomplete type

src/include/linux/wait.h: In function `__add_wait_queue_tail':

src/include/linux/wait.h:283: error: dereferencing pointer to incomplete type

I've also tried building these in the service console on one of my ESX hosts, but I get similar errors:

    • Compiling drivers/block/cciss/cciss.c

In file included from src/include/linux/module.h:11,

from src/drivers/block/cciss/cciss.c:24:

src/include/linux/spinlock.h:124: syntax error before '*' token

src/include/linux/spinlock.h: In function `vmk_spin_is_locked_by_my_cpu':

src/include/linux/spinlock.h:126: `lock' undeclared (first use in this function)

src/include/linux/spinlock.h:126: (Each undeclared identifier is reported only once

src/include/linux/spinlock.h:126: for each function it appears in.)

In file included from src/include/linux/module.h:11,

from src/drivers/block/cciss/cciss.c:24:

src/include/linux/spinlock.h: At top level:

src/include/linux/spinlock.h:230: syntax error before "spinlock_t"

In file included from src/include/linux/module.h:11,

from src/drivers/block/cciss/cciss.c:24:

src/include/linux/spinlock.h:237: syntax error before "spinlock_t"

src/include/linux/spinlock.h:239: syntax error before '}' token

In file included from src/include/linux/fs.h:12,

from src/include/linux/capability.h:17,

from src/include/linux/binfmts.h:4,

from src/include/linux/sched.h:10,

from src/include/linux/mm.h:22,

from src/include/linux/slab.h:14,

from src/drivers/block/cciss/cciss.c:29:

src/include/linux/wait.h:90: syntax error before "spinlock_t"

src/include/linux/wait.h:102: syntax error before '}' token

src/include/linux/wait.h: In function `init_waitqueue_head':

src/include/linux/wait.h:194: dereferencing pointer to incomplete type

src/include/linux/wait.h:199: dereferencing pointer to incomplete type

src/include/linux/wait.h:199: dereferencing pointer to incomplete type

src/include/linux/wait.h:199: dereferencing pointer to incomplete type

src/include/linux/wait.h:199: dereferencing pointer to incomplete type

src/include/linux/wait.h:205: dereferencing pointer to incomplete type

src/include/linux/wait.h: In function `waitqueue_active':

src/include/linux/wait.h:253: dereferencing pointer to incomplete type

src/include/linux/wait.h: In function `__add_wait_queue':

src/include/linux/wait.h:266: dereferencing pointer to incomplete type

src/include/linux/wait.h: In function `__add_wait_queue_tail':

src/include/linux/wait.h:283: dereferencing pointer to incomplete type

I was wondering if anyone has successfully built these drivers from the provided source, or if you're even supposed to be able to build from these sources, and how, or if providing this is just VMware complying with a formality and you're not really supposed to be able to do anything with those sources?

Thanks - Nick

20 Replies
Texiwill
Leadership
Leadership

Hello,

You will most likely also need the VMware DDK. Access to this however is limited.... Unfortunately, GPL does not require you to provide something 'buildable' just the GPL portions of the code in source form. If the GPL code depends on licensed code that does not need to be provided.... GPL with licensed code is such a PITA at times.


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll

Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
nick_couchman
Immortal
Immortal

Thanks, Edward, for the info. Actually, I figured it out. Here's what you have to do...

1) Download the GPL Sources for ESX

2) Unpack the VMware-eesx-drivers package.

3) Unpack the VMware-eesx-server package.

4) Go into the VMware-esx-drivers folder, then into the src/include directory.

5) Make a symbolic link to the include/lkhdrs/asm-i386 folder in the VMware-esx-public-source folder (from the VMware-eesx-server package).

After this, the build scripts should work. I've got in running on my CentOS 3.9 system, building the drivers without a problem!

Getting ahold of the DDK requires, among other things a Select TAP agreement, which runs about $5K per year. Too bad...

Reply
0 Kudos
Rumple
Virtuoso
Virtuoso

The big question being though, does this do anything for you other then give you the ability to load drivers for the service console, which is mainly just an effort in futility since its the vmkernel that you want to have the expanded driver support for.

Reply
0 Kudos
nick_couchman
Immortal
Immortal

Well, it took me about three weeks to come up with the answer, but the answer is yes - I can successfully modify and build drivers not originally included with the ESX(i) distribution, load them, and use them for VM-type activities. I've just finished doing this with the Realtek 8169 Gigabit Ethernet controller driver - I've modified it, compiled it, and loaded it into ESXi. I'm then able to create a vSwitch in ESXi with this as the physical interface and connect both virtual machines and VMKernel interfaces to the controller. Now I just need to clean up the code, figure out a couple more issues, and then get a good compile script created and I should be able to reproduce this for other drivers.

I'd like to start some sort of community project to port drivers over to ESX/ESXi...

Reply
0 Kudos
Texiwill
Leadership
Leadership

Hello,

Sounds like a plan. I would help with this. If we get a good build script it should be pretty straight forward and would help quite a bit. Have you tried your driver in ESX yet?


Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.
Blue Gears and SearchVMware Pro Blogs -- Top Virtualization Security Links -- Virtualization Security Round Table Podcast

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
Reply
0 Kudos
nick_couchman
Immortal
Immortal

I haven't loaded it in ESX, yet - I don't have any ESX test machines around here - all of my full-blown ESX machines are in production mode. I only have ESXi test machines, which is where I've loaded and tested the driver.

As far as a community project goes, do you have any feel for whether or not it would be acceptable to create a SourceForge project called "ESX Community Drivers" or something like that? I guess I'm just unsure as to whether or not we could use the name "ESX" in the project name, or if VMware would not like that. Or if we make sure we attribute "ESX" to VMware, maybe it would be okay?

Reply
0 Kudos
Texiwill
Leadership
Leadership

Hello,

I will ask.


Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.
Blue Gears and SearchVMware Pro Blogs -- Top Virtualization Security Links -- Virtualization Security Round Table Podcast

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
Reply
0 Kudos
admin
Immortal
Immortal

I am not a lawyer, but I would stay away from ESX in the name. Trademark law is a lot about avoiding confusion, and VMware has a lot of open source projects on its own, so I would be confused if these were 'ESX Community Drivers'. Just give it a name. SexyDrivers? Backseat Driver Project? OpenHyperDrivers?

From looking at other products that work with ESX, you can certainly mention that they're for ESX in the description.

I'll ping our open source folks as well, as soon as I figure out who that is.

Reply
0 Kudos
nick_couchman
Immortal
Immortal

Thanks, John - I appreciate the input! Let me know when you hear from your legal/open source folks, too - I'm very interested to hear what they have to say!

Reply
0 Kudos
Texiwill
Leadership
Leadership

Hello,

I like the Open 'visor Drivers myself.... But that could be confused with Xen and KVM as wel

Perhaps just:

Open Virtualization Drivers


Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.
Blue Gears and SearchVMware Pro Blogs -- Top Virtualization Security Links -- Virtualization Security Round Table Podcast

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
Reply
0 Kudos
nick_couchman
Immortal
Immortal

Reply
0 Kudos
jasonjm
Contributor
Contributor

I would to jump onboard (no pun intended) with this sf proj.

My team and I have been looking at the new e1000e and realtek sources for sometime.

(current e1000e use an older source base)

Reply
0 Kudos
admin
Immortal
Immortal

I spoke to some folks internally and we don't have any objections to the project or experimenting with the open source drivers. Keep in mind that they wouldn't be supported, of course, and there's no open api or ddk.

My personal thought is -- more power to you! You might even be able to expand the HCL, and this kind of thing can be very beneficial for the greater community. So have fun.

I didn't think it was necessary to talk to legal if you weren't going to name the project ESX. The licenses are pretty standard.

John

Reply
0 Kudos
nick_couchman
Immortal
Immortal

Sounds good - Thanks, John! And I definitely understand the point about them being unsupported by VMware - makes perfect sense. I'll make sure the project page includes something about that...

Reply
0 Kudos
danielibarnes
Contributor
Contributor

A thousand thanks for your build tip. I am desperately trying to compile the drivers for the NIC (8086:10cd) and SATA interface (8086:3a20 and 8086:3a26) in a Dell XPS Studio 435mt. It looks like ESX uses 2.4.37-rc1 or later and I need to patch the ata_piix driver to 2.4.37 for SATA support.

Reply
0 Kudos
nick_couchman
Immortal
Immortal

There's probably a bit more to it than just patching for sata support. First, the ESX Linux emulation layer is actually based more on the 2.4.21 kernel:

Linux <hostname> 2.4.21-57.ELvmnix #1 Thu Nov 20 13:06:23 PST 2008 i686 i686 i386 GNU/Linux

Second, while I haven't done much with the storage drivers, in my meandering through the network driver code I've noticed quite a few things that have been added/removed for VMKernel support. For example, in the network interface code, when a module is compiled for VMKernel support the occurrences of synchronize_irq() are replaced with vmk_synchronize_irq(). There a few things like this, and in some cases there are a lot more detailed things that need to take place, as well. One of the things we'll be working on in the SF.net project we've just started will be documenting these sorts of changes so that there's a clearer picture of what has to be changed.

Good luck!

Reply
0 Kudos
aka_MAC
Contributor
Contributor

Hello!

I have the same problem. I need to create driver disk for Realtek 8168 network adapter just like this. (for new ESX/ESXi installation). All I have is linux driver source.

How can I build this CD?

Thanks a lot!

Reply
0 Kudos
akshunj
Enthusiast
Enthusiast

Could anyone try porting this driver?

http://sourceforge.net/projects/et131x/

Reply
0 Kudos
Robert321
Contributor
Contributor

This is a general comment more geared towards VMware, but I figured this thread was an excellent place to voice my thoughts and rant at what I see as a BIG business mistake.

VMware,

As a company with share holders, I find it absolutely DUMB decision to NOT embrace customized drivers for the ESXi OS's. The easier you make the process the larger the user base becomes. A laymen can do the math and figure out more users later translates into higher demands for your software. This later turns into REVENUE or DOLLARS for those who cannot figure out the complex R word (sarcasm intentional).

The decision to NOT embrace and support customized drivers is akin to Microsoft not buying your company long ago. They failed to embrace Linux, and they ultimately paid the price of not hugging Linux and loving it. Your obviously moving away from Powershell with the new Linux based vCenter appliance, probably due to your fear of Powershell and VMM 2011 (no fears, it stinks).

Seems your making a few mistakes of your own there. Hug and embrace Microsoft, but do what you do best, make a better product that works closely with the enemy. Perhaps your Google friends are pushing your hands, perhaps not, who knows besides insiders...

Nuff said on that topic, back to the driver issues.

Point is the more people using ESXI 5.0, the more likely those people become future customers. Make the process of compiling and creating drivers EASY as pushing a button just makes good business sense.

Perhaps you should POLL the MOB's of THOUSANDS!!!!

MOB, REPLY HERE PLEASE.:smileymischief: