I'm getting a GL_OUT_OF_MEMORY error when trying to allocate textures that consume ~100MB of memory, I have set the VM to use ~2GB of VRAM so it shouldn't be the case that there is not enough memory, just for whatever reason the driver won't let me allocate larger than the 100MB. When I use my host environment I can allocate a texture size up to ~4GB (max size of my VRAM) without an error.
The textures that I'm trying to upload are ~128MB so if there's some way to instruct the driver on the VM to increase its maximum allowable allocation that would be great 
edit: This is caused by a call to glTexImage2D and is found in vm3dgl64.dll, is there a way to get the source for this dll to rebuild it myself or is that proprietary in the VM?
edit 2: My system is a Windows 7 Pro running VMWare Workstation 12 with a Windows 7 Pro guest with an AMD FirePro v9800, my coworkers machine would not be relevant as he's not running VMware. However my boss is running a MacBook with VMWare Workstation 12 and a Windows 7 guest and has the same problem. I've attached the log, but I don't believe the error gets logged, this is a problem that only appears when running my application in a VM. Note that running natively in Windows host the application is fine.
Hi,
side info: A coworker of mine is using a VM but has a dedicated graphics card with driver installed to access the hardware directly and does not get this error.
What product is your coworker using?
AFAIK VMware Workstation does not support PCIe pass-through for dedicated graphics cards
Btw, you are not mentioning what VMware Workstation version you are using, what your host OS is, what your guest OS is -windows something I can guess after your edit- or what graphics adapter you have in your host.
Is it possible that your graphics card triggers the error?
It might be a good idea to attach a vmware.log file from your VM, preferably a copy which has the error you are seeing in it.
edit: Unless I"m severely mistaken, the source for the windows graphics drivers is not available.
--
Wil
