VMware Communities
COREstorm
Contributor
Contributor

Workstation 9 - No 3D Acceleration to Guest - Ubuntu 12.04 Host - Nvidia Card

Ok, I've been going at this for some time.

Current Installation: Fresh install of Ubuntu 12.04

Video card: Nvidia Optimus GT540M

VMware: Workstation 9. (Unbuntu 12.04 is the HOST)

(Brief history: tried to get 3D support working under Workstation 8, using numerous nvidia drivers (every step you can find online), upgraded to Workstation 9 and retried all the steps. Reinstalled Ubuntu.);

Current status: Currently using Bumblebee. 3D WORKS on Ubuntu. I have glxinfo returning YES for 3D support and glxspheres working.

Since No nvidia drivers are installed, Vmware Workstation WILL NOT recognize the video card, and thus will not enable 3D support for the Windows 7 VM. - This is the problem we need to solve.

I have tried to uninstall Bumblebee and install Nvidia drivers and suffered from the 640x480 issue, with the Nvidia Settings not detecting a proper x configuration.

So the issue is currently: VMware will not recognize the video card properly since Bumblebee is installed. The 3D support will ONLY (so far) work using Bumblebee.

Let the delimma begin. This is currently a Vmware issue, although could quickly become another Ubuntu 12.04 issue if I have to go down the path of many drivers.

(Yes, I spent 9 hours trying to install every version of the Nvidia drivers everyone "suggested" with zero luck and many issues.)

112 Replies
dskinner
Contributor
Contributor

Just tried again, intel only laptop, still fails.

...

Reply
0 Kudos
AlanaA
Enthusiast
Enthusiast

The release notes for Workstation 8.0.5 finally got posted: http://www.vmware.com/support/ws80/doc/releasenotes_workstation_805.html

It says:

The following operating systems are now supported host and guest operating systems:
     Ubuntu 12.04 and 12.10
     RHEL 5.8 and 6.3
     CentOS 5.8 and 6.3
     OpenSUSE 12.1 and 12.2
     Fedora 17 is supported as a guest only.

and

Workstation 8.0.5 accepts a Workstation 9 license key. Customers who  have reasons for not upgrading immediately can purchase Workstation 9  and continue to use Workstation 8 as necessary.

So maybe people can go back to Workstation 8 for now if 8.0.5 doesn't break the 3D support.

Reply
0 Kudos
alfito2012
Contributor
Contributor

Hi,

I wonder if someone has tried this workaround for running vmware guest through optirun.

https://github.com/allquixotic/vmxwrapper

My first approach has been unsuccesfull, but looks it could be a solution. I am waiting for the developer for some help. Anyway, would be interesting if someone else could try this.

Reply
0 Kudos
ninjacoding
Contributor
Contributor

alfito2012 wrote:

Hi,

I wonder if someone has tried this workaround for running vmware guest through optirun.

https://github.com/allquixotic/vmxwrapper

My first approach has been unsuccesfull, but looks it could be a solution. I am waiting for the developer for some help. Anyway, would be interesting if someone else could try this.

Has anyone tried this?

I find it hard to believe that it works. If it does work, how weren't vmplayer engineers unable to see this solution?

I'm unable to try it since my WS9 eval period has expired.

Also, I got a bit traumatized some weeks ago by so much time spent around vmware with uninstall and install and trying all sorts of thread (for those new, read the whole thread and you see).

With google I found this post:

http://askubuntu.com/questions/221560/has-anyone-gotten-vmware-workstation-9-0-1-working-with-vmxwra...

Reply
0 Kudos
rashm2k
Contributor
Contributor

I tried it - it didn't work for me.

However rather than saying that mesa driver is detected - it shows that the correct graphics was detected but it was not compaitble for some reason.

Reply
0 Kudos
dskinner
Contributor
Contributor

it's not *the* solution anyway, it's *a* solution for people that happen to have nvidia along with intel to make use of nvidia (assuming it works). I have an intel HD4000 only laptop and have reverted to workstation 8 since the latest update allows the use of WS9 keys.

There's really two issues here and the major one is that acceleration on intel cards in WS9 completely flops. Not trying to be offensive here, but i'd be downright bummed if the official fix from vmware came down and was "here, now you can sidestep the intel issues" and leave intel in a broken state.

I certainly hope both issues are resolved.

Reply
0 Kudos
goomba000
Contributor
Contributor

Hello everybody.  I got 3D acceleration working using the nvidia card with this configuration:

Dell XPS 15 L501X
Optimus Intel HD Graphics + nVidia GeForce GT 420M
VMWare Workstation 9.0.0 build 812388
Host: Ubuntu 9.10 64-bit
Guest: Win XP SP3 VM with DirectX 9.0c
NVidia drivers in host: 304.43
VMWare tools installed
The problem is that the executable used to launch the VM is vmware-vmx NOT vmware. Therefore by running optirun vmware you're only running the vmware GUI inside VGL using your nvidia card.  What you must do is wrap the "vmware-vmx" binary inside a script that will launch it with optirun.  It is located by default inside /usr/lib/vmware/bin/.  You can check if vmware-vmx is launched by optirun by running "ps ax | grep vmware" after launching the VM.
Also, for me optirun has trouble loading nvidia's libGL.so so I had to add symlinks in /usr/lib/x86_64-linux-gnu/ pointing to libnvidia-tls.so and libnvidia-glcore.so

The steps:

Symlinks:

sudo ln -s /usr/lib/nvidia-current/tls/libnvidia-tls.so.304.43 /usr/lib/x86_64-linux-gnu/

sudo ln -s /usr/lib/nvidia-current/libnvidia-glcore.so.304.43 /usr/lib/x86_64-linux-gnu/
sudo ldconfig
Notes:
- Change the name of the libs accordingly.
- If running a 32-bit linux, symlink to /usr/lib/ instead, or if using 32-bit VMWare, symlink to /usr/lib/i386-linux-gnu/.
Wrapper script:

sudo /etc/init.d/vmware stop

sudo mv /usr/lib/vmware/bin/vmware-vmx /usr/lib/vmware/bin/vmware-vmx.orig
Create the /usr/lib/vmware/bin/vmware-vmx script containing this:
#!/bin/bash
VGL_GLLIB=/usr/lib/nvidia-current/libGL.so.1 exec optirun /usr/lib/vmware/bin/vmware-vmx.orig "$@"
Set permissions (vmware-vmx.orig is setuid):
sudo chmod u+s,a+x /usr/lib/vmware/bin/vmware-vmx
sudo /etc/init.d/vmware start
Although 3D "works", for me the VM crashes shortly after starting the Direct3D test in dxdiag.  Further testing needed.
To get some debugging information from the GL driver and from VirtualGL, start vmware from the command-line and make the wrapper script this instead:
#!/bin/bash
VGL_GLLIB=/usr/lib/nvidia-current/libGL.so.1 LIBGL_DEBUG=verbose VGL_VERBOSE=1 exec optirun /usr/lib/vmware/bin/vmware-vmx.orig "$@"
This information was provided by amonakov of the Bumblebee project to make 3D work in Virtualbox, although in that case a wrapper file is not necessary since you can launch the VM directly from the VirtualBox executable with the -startvm switch

These steps are similar to the steps to wrap vmware-vmx in alsa-oss for OSS audio mixing:
(To do that if you're interested, just install alsa-oss and add LD_PRELOAD=libaoss.so to the script)

I'd love to know how it runs for other people.

Reply
0 Kudos
goomba000
Contributor
Contributor

I just read the posts about vmxwrapper.  It's not necessary, vmware-vmx can be wrapped in optirun in a shell script, I did it and it works.  Also, that program doesn't really do anything.  It just seems to launch the original vmware-vmx executable with the same parameters + 1 more.  I think the author forgot to put something in his code, I emailed him about it. 

But in any case, it's not necessary, you can wrap vmware-vmx in a bash script like I did.

Reply
0 Kudos
dskinner
Contributor
Contributor

lthough 3D "works", for me the VM crashes shortly after starting the Direct3D test in dxdiag. 

Hi, as I've stated before, intel only here. I was also able to get the system to boot with 3D enabled but essentially the same issue. Once starting tests in dxdiag (for xp) or running system rating (win7), the system crashes. I was under the impression that nvidia only systems could work which would lead me to believe that getting WS9 to use nvidia in an nv/intel combo would also work.

I mention my case in-case perhaps the intel video is still playing some role in your setup.

Reply
0 Kudos
goomba000
Contributor
Contributor

dskinner wrote:

I mention my case in-case perhaps the intel video is still playing some role in your setup.

I don't think so.  I've checked vmware.log to make sure the nvidia driver was loaded not the intel one.  When I use the intel driver (with mks.gl.allowBlacklistedDrivers = "TRUE"),  the Direct3d tests in dxdiag run (all 3 tests, D3D 7,8,9), but there's no texture on the spinning cube.  The VM also crashed shortly after running the tests last time I tried too.  When I use the nvidia driver with optirun, I can start the Direct3D 7 test and I see the spinning cube, textured this time, but I was never able to finish that test before the VM crashed.

I wonder if people with an nvidia card only setup have had more luck with 3D in vmware 9.

Reply
0 Kudos
rashm2k
Contributor
Contributor

goomba000 I'm not so sure what you mention is anything new.

There are two problems we are trying to address in this thread:

1) The Intel Driver and Graphics does not work with VMware 9 - forcing this by editing the config files causes crashes in VMware, even though glxinfo in the guest shows it's enabled (same on windows)

2) optirun also does not work as it fails to recognise the nvidia graphics and like above crashes - again showing 3d acceleration as enabled but crashing when you attempt to use any 3d features.

There is no solution so far for problem 1 or 2. I have observed the following:

1) Intel graphics does not work - forcing vmware causes vmware to crash.

2) Using optirun initially made no difference - however using vmxwrapper optirun now works with vmware (still crashes VM though!).

The problem with all the solutions presented is that they cause the VM to crash.

However there is ONE final solution which I have not tried yet. I noticed in my previous attempts that adding the nomodeset boot parameter in my attempt to force use of the nvidia graphics my vm did NOT crash. I cannot test this until later in the evening.

Those that are able to test I suggest you do the following

Get optirun working properly and with vmware as well (vmwrapper or the script above) - how can you tell if it working? Well if you are using the Intel driver your vmware.log will say something like 'detected Mesa driver... disabling... use mks.gl.allowBlacklistedDrivers = "TRUE" to override' and when using the nvidia graphics (in my case) it says something '...unable to....try upgrading your nvidia drivers....' (sorry can't remember the exact phrase).

Try it with optirun, without optirun then with nomodeset and also with REMOVING the override in the .vmx file!

Reply
0 Kudos
rashm2k
Contributor
Contributor

A quick google for hypervisor and graphics turned up some pretty interested results!

Why bother with VMware when you can get Xen to use PCI/VGA pass through!

http://superuser.com/questions/454296/what-hypervisor-gpu-can-be-used-for-gaming-under-a-virtualized...

http://www.phoronix.com/scan.php?page=news_item&px=MTA3NjA

http://hardforum.com/showthread.php?t=1689500

Given that vmware doesn't give a sh!t maybe we should do the same and move to something else, since this really only affects those of us that are using Linux anyway.

A
Reply
0 Kudos
cmillersp
Contributor
Contributor

Hi. Getting all of this working has been a major issue for me.  I had previously done what you did, but had linked the libnvidia-tls file directly from the nvidia folder, as opposed to from the tls folder. Fixing that allowed me to start the vm using the nvidia card.

Unfortunately, the VM does crash, and crash hard.  Not to mention (in Windows 7 at least) it doesn't redraw correctly.  Junk is basically left on the screen.

Personally, when I enable the intel driver and run it like that, I was able to run 3dmark, windows experience index, and all sorts of other stuff with somewhat poor scores, but without crashes... although after a few hours it would sometimes hang, but it wasn't necessarily directly related to using the graphics, as far as I could tell.

I'm wondering if there would be some way to get it running using primusrun.  I don't know if you're familiar:

https://launchpad.net/~zhurikhin/+archive/primus

Apparently, many programs that don't work well with regular bumblebee do alright when ran with primusrun.  The only issue I have is, whatever kind of script I use, I can't get vmware to use the nvidia libs like I can with optirun.  If there's anything you might know that would perhaps bypass this or get it running with primusrun, we may have not only nvidia acceleration, but as-needed nvidia acceleration.

I should note as well that vmxwrapper does actually work if you specify the gl lib and just optirun vmware itself.  This way, you don't have to edit a script to choose whether or not to use bumblebee, you simply run optirun vmware or vmware.

Some other things I'd like to try after work:

virtualgl transport method change.

compare result to other versions of vmware (has anyone done this yet with optirun?)

One other thing to note.  Although it totally stinks in terms of usability, hybrid-windump is a working option.  You would need to set something up to make the mouse/keyboard work across servers, like Synergy.

We would, I'm sure, however, all prefer to be able to run it without crashing with our choice of nvidia or intel chips by use of bumblebee.

Reply
0 Kudos
goomba000
Contributor
Contributor

rashm2k wrote:

Get optirun working properly and with vmware as well (vmwrapper or the script above) - how can you tell if it working? Well if you are using the Intel driver your vmware.log will say something like 'detected Mesa driver... disabling... use mks.gl.allowBlacklistedDrivers = "TRUE" to override' and when using the nvidia graphics (in my case) it says something '...unable to....try upgrading your nvidia drivers....' (sorry can't remember the exact phrase).

That is what I did, and I checked vmware.log like I said in the post before yours.

2) optirun also does not work as it fails to recognise the nvidia graphics and like above crashes - again showing 3d acceleration as enabled but crashing when you attempt to use any 3d features.

Why do you say optirun does not work?  It surely does as the VM clearly sees the nvidia card like I mentioned. vmware.log:

2012-11-27T21:30:38.647-05:00| vmx| I120: [0x7FA064675000-0x7FA06473EBE8): /usr/lib/nvidia-current/libGL.so.1
2012-11-27T21:30:38.647-05:00| vmx| I120: [0x7FA064472000-0x7FA064474600): /usr/lib/x86_64-linux-gnu/libnvidia-tls.so.304.43
2012-11-27T21:30:38.647-05:00| vmx| I120: [0x7FA062089000-0x7FA063A57C90): /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.304.43
2012-11-27T21:30:38.647-05:00| vmx| I120: [0x7FA0613CD000-0x7FA0613CD928): /lib/x86_64-linux-gnu/libaio.so.1
2012-11-27T21:30:38.647-05:00| vmx| I120: End printing loaded objects
2012-11-27T21:30:38.821-05:00| mks| I120: OpenGL Version: "3.2.0 NVIDIA 304.43" (3.2.0)
2012-11-27T21:30:38.821-05:00| mks| I120: GLSL Version: "1.50 NVIDIA via Cg compiler" (1.50.0)
2012-11-27T21:30:38.821-05:00| mks| I120: OpenGL Vendor: "NVIDIA Corporation"
2012-11-27T21:30:38.821-05:00| mks| I120: OpenGL Renderer: "GeForce GT 420M/PCIe/SSE2"

Also, I CAN run 3D in the VM.  Like I said, it doesn't crash right when I start the D3D test, but after a short while (I can SEE the 3D graphics, rendered correctly as opposed to the Intel card), which lead me to believe that the bug is in VMWare.  Optirun just makes the app output to a different X server where the nvidia driver is used, but render it to the same display, so why would it be responsible for a VM crash?  But yes to be sure, it would be nice to have somebody test on a computer with only an nvidia card.

1) Intel graphics does not work - forcing vmware causes vmware to crash.

That is to be expected.  VMWare 9 probably uses OpenGL extensions that the Intel/Mesa drivers don't support.  Can't complain when we override it and crash the VM.  The layer of translation D3D -> OpenGL is probably enormously complex, so I can't say I'm surprised if they decided not to support Intel cards (or other integrated graphics) to make their lives easier.

Reply
0 Kudos
goomba000
Contributor
Contributor

cmillersp wrote:

I'm wondering if there would be some way to get it running using primusrun.  I don't know if you're familiar:

https://launchpad.net/~zhurikhin/+archive/primus

Worth a try.  I might fiddle with it later.

One other thing to note.  Although it totally stinks in terms of usability, hybrid-windump is a working option.  You would need to set something up to make the mouse/keyboard work across servers, like Synergy.

Yeah that would really suck.  I used that for apps that require VDPAU (vlc, etc).  That's just the worst hack I've ever seen, no control over the window at all.

But then again, all of what we're trying to do is ugly hacks.  I see only 2 ways to do things properly in the future:

1- Proper support of Optimus through DMA-BUF.  Not likely to happen because of licensing issues. See http://lists.freedesktop.org/archives/dri-devel/2012-October/028846.html

2- VGA passthrough of the nvidia card using the IOMMU.  That would be kickass, no translation required, the guest uses the card directly.  That is in the hands of VMWare, and passthrough for graphics card is much harder than for other PCI cards, so that won't happen soon either.

Reply
0 Kudos
cmillersp
Contributor
Contributor

I don't know how that would work, though. Even with xen, if I'm not mistaken, when you use passthrough to use another card, you have to hook an additional monitor up to that card to use it.  We'd still need some ugly hack along the lines of hybrid-windump, even with passthrough to windows, but it would seem even more complicated.

I've been considering giving up, running windows natively, and virtualizing linux, but that's not very fun, and then I lose performance on the OS I actually use for most things.  It might be more fun to just go buy a notebook with AMD graphics.  I also need to see what I can get out of VirtualBox.

In the meantime, if you can figure out a way to use primusrun, since it isn't VGL, it might work somehow.  I wouldn't be too optimistic, though, because it supports even less opengl functions than virtualgl does.  But we'll have to see!  I just can't seem to make it load primusrun's version OR nvidia's version of libGL.so.1, so let me know if you can!

Reply
0 Kudos
goomba000
Contributor
Contributor

cmillersp wrote:

I don't know how that would work, though. Even with xen, if I'm not mistaken, when you use passthrough to use another card, you have to hook an additional monitor up to that card to use it.  We'd still need some ugly hack along the lines of hybrid-windump, even with passthrough to windows, but it would seem even more complicated.

Not necessarily.  We use hybrid-windump because we want to run something on a different X server but on the same display (say, in a sort of "window" overlayed on top which is near impossible to move).  However, for a virtual machine, I wouldn't mind running it in a different virtual console where the X server has exclusive access to the nvidia card so you switch to the that display the usual way (ctrl+alt+F8).  Of course then you don't have the benefit of drag-n-drop with vmtools or unity (which I never used), and also you can't run anything on the nvidia card (with optirun) on the first display, but I think those inconveniences are offset by the massive advantage of direct access to the card by the VM and full DirectX functionality, with no speed penalty.

In the meantime, if you can figure out a way to use primusrun, since it isn't VGL, it might work somehow.  I wouldn't be too optimistic, though, because it supports even less opengl functions than virtualgl does.  But we'll have to see!  I just can't seem to make it load primusrun's version OR nvidia's version of libGL.so.1, so let me know if you can!

I'm gonna try later tonight.  I haven't thought at lenght of the problems that might arise using VGL instead of X directly, I think it might be worth a try to disable the intel chip completely and run X on the nvidia card if that's possible.  I don't hold much hope of something functional either way with VMWare though.  In any case, for me one of the reasons I wanted to try it is that I would have liked to be able to play old win9x games that require 3D, but I realized a few days ago that it only works for XP+ (NT/2000? haven't tried), which is too bad.  Another area where VGA passthrough might be useful.  And I really don't want to dual-boot such an OS (and find drivers/etc).

Reply
0 Kudos
cmillersp
Contributor
Contributor

As much as I hate to say this, running the nVidia card only is not possible without a hardware mux.  The nVidia hardware is not and can never be connected directly to the display, with the exception of laptops with a hardware mux (Thinkpad W series for instance).  With optimus, the only thing which can directly display is the intel card, which the nVidia card must use to display whatever it rendered.

This is also the reason that even if we get PCI-passthrough on a VM, nothing will display on the screen.  The intel card still has to somehow display what the nVidia card rendered.  We can do this now with vnc, for example, and running a second X session on the nVidia card.  The problem with that is, performance is bad, hence the existance of hybrid-windump, which, I think, should be worked on and perfected so that we can get native nVidia features.

Reply
0 Kudos
goomba000
Contributor
Contributor

Hmmm you're right.  Well that is an unfortunate situation.  I had hoped that it would have been possible to still dedicate the nvidia card to the guest while using the intel display buffer transparently through some voodoo hackery.   Curse the day I bought this laptop...  Lower power consumption be damned, it isn't worth the trouble.

Reply
0 Kudos
cmillersp
Contributor
Contributor

An update:

I have 3d from nvidia card working with no crashing... in VMWare 8.0.5, though.  But I got over 8000 3dmarks in 3dmark 03, and I can confirm it's working, using primusrun.  afaik, it needs to be run as root.

I'm currently looking into trying this on Vmware WS 9.0.1 and will let everyone know when I have something to share.  Worst case scenario, we will at least have fully working nvidia-accel on VMWare 8, which, afaik, we haven't had yet.

Edit:

Yes, it works on VMWare 9 as well.  I will come up with a tutorial tonight or tomorrow when I have time, but it works!  If you don't want to wait for the tutorial, your vmware-vmx file should look like this,

#!/bin/bash

LD_PRELOAD=/usr/lib/x86_64-linux-gnu/primus/libGL.so.1 exec primusrun /usr/lib/vmware/bin/vmware-vmx.real "$@"

You need to have primus installed from its ppa, and you need to run sudo vmware.  It will give some errors when it runs, such as

Xlib:  extension "NV-GLX" missing on display ":0".

but... there has been no crashing so far.  Will report later if any.

I think 8.0.5 may be better in terms of 3d performance in this case, as I can't get more than 6500 in 3dmark 03 on vmware 9.

Reply
0 Kudos