VMware Cloud Community
taylorjonl
Enthusiast
Enthusiast
‎12-23-2010 12:10 PM

VMDirectPath and ATI Radeon

I am using VMware ESXi and I am trying to setup a guest that is Windows 7 that will have an ATI Radeon video card passed through to it.  I actually had this working on a previous system but I had to reinstall.  Now when I do this the guest fails to start and I get the following:

Error message from localhost.XXXXXXXXXXX:
PCIPassthru 004:00.0: Guest tried to (null)map
32 device pages (with base address of 0xb5d20)
to a range occupied by main memory. This is
outside of the PCI Hole. Add pciHole.start =
"2909" to the configuration file and then power
on the VM.
error
12/23/2010 1:04:36 PM
media
User

When I do as it asks, the guest now starts but gets an immediate BSOD concerning memory management.  Any ideas on why this is occuring and why it worked at one point but now it fails?

Reply
814 Replies
ciuly
Contributor
Contributor
‎10-22-2012 02:55 AM

sounds to me like your passthrough is not working or drivers not installed. Normally, as already explained by shlomiassaf, you will get your VM display on your monitor after a little while, IF the passthrough works. Otherwise, either your VM has not installed the drivers and thus still using the VMware SVGA, or your VM will go into BSOD orPSOD depending on what OS you are running on it, and your screen will remain stuck at the cnic_register.

So your first option is to connect either remotely or with vspher client and see what is the status of the VM. if it requires driveres for the newly found hardware video card, install them. If it's crashed ... you cna start trying a few settings that are posted on this forum and maybe you cna get it running.

VMDirectPath with ATI GPU document https://docs.google.com/spreadsheet/ccc?key=0Aqp_xYBwP_Y7dE5EclhtaDdIV09lNWxfODd1alRUTlE
Reply
0 Kudos
Bhargava_Shriva
Contributor
Contributor
‎10-22-2012 03:15 AM

Yes you’re absolutely right

passthrough is not working or drivers not installed.

“either your VM has not installed the drivers and thus still using the VMware SVGA” – I can try once pass through work but manually I have edited .vmx file and I had given 800 MB video memory, but I wanted to know where does this memory taken form exactly from GPU because there is no onboard video memory there is only dedicated quadro graphic card on the workstation.

“So your first option is to connect either remotely or with vspher client and see what is the status of the VM. if it requires driveres for the newly found hardware video card, install them.” Which one your referring to Guest OS or esx box if its ESX box where do I find that in vSphere kindly let me know.

Guest OS is windows 7 Ent.

Reply
0 Kudos
Bhargava_Shriva
Contributor
Contributor
‎10-22-2012 03:17 AM

this may give some more clarity

http://communities.vmware.com/message/2134186#2134186

Reply
0 Kudos
ciuly
Contributor
Contributor
‎10-22-2012 03:37 AM

It's not clear form what you wrote. Is it working if you manually edit the vmx?

after you have configured your card for passthrough, the host configuration is done. Everything else you do on your VM, except for the BIOS configurations that are suggested on this thread, which obviously you do on host.

Although now that I'm thinking about it, it's been literally ages since I last looked at a VMWare bios and never on one from an ESX VM...

VMDirectPath with ATI GPU document https://docs.google.com/spreadsheet/ccc?key=0Aqp_xYBwP_Y7dE5EclhtaDdIV09lNWxfODd1alRUTlE
Reply
0 Kudos
Bhargava_Shriva
Contributor
Contributor
‎10-22-2012 03:52 AM

Let me explain,

We have manually edited vmx file to increase video memory form 128 MB (Max that can be set using vSpehre) to 800 MB that will reflect as 1 GB vSGA (Virtual Shared Graphics Acceleration) and working fine.

But for more robust performance we are trying to bypass GPU and wanted to have dedicated graphic card   on individual VM.

Reply
0 Kudos
KRA77
Contributor
Contributor
‎10-22-2012 04:36 AM

When you set up your gfx card for passthrough it is normal your display will freeze right after cnic_register. Just wait a little longer ang connect remotly with vsphere client. Once you start up your VM with the gfx card setup your display will most likely go dark, ant it will not initialise until it's drivers are loaded. So you have to run your admin and installation from vsphere.

Reply
0 Kudos
Bhargava_Shriva
Contributor
Contributor
‎10-22-2012 04:40 AM

K but we have waited for 2 hours now also let me check. How long it may take?

Reply
0 Kudos
ciuly
Contributor
Contributor
‎10-22-2012 05:12 AM

we all say that you need to connect with vsphere client to your esx and check the console for that VM. Did you do that? If not, do it now. You can connect to the esx and then vm console in about 1 min after cnic_register

In the VM Console you will then see either BSOD, or vm operating (boot, window loading or window loaded and idle)

VMDirectPath with ATI GPU document https://docs.google.com/spreadsheet/ccc?key=0Aqp_xYBwP_Y7dE5EclhtaDdIV09lNWxfODd1alRUTlE
Reply
0 Kudos
Bhargava_Shriva
Contributor
Contributor
‎10-22-2012 05:49 AM

Thanks guys for all ur replies

But no luck it’s almost 6 hours my ESX 5.1 is hung in cnic_register. Kindly give solution for this cnic_register J

Here is the link where there is no solution here as well http://communities.vmware.com/message/2134186#2134186

Reply
0 Kudos
KRA77
Contributor
Contributor
‎10-22-2012 05:59 AM

Please double check that you are doing it correctly, the behaviour you describe is perfect normal with gfx passthrough. Once cnic_register is loading display will freeze and stay that way until you start a VM that access the gfx card, at that point the screen will normally go dark, and stay that way until gfx drivers are loaded. Text display is output via console in vSphere.

So your remote vSphere client fails to connect to the esxi server ? Are you sure you are connecting to the right IP adress with vSphere ?

Reply
0 Kudos
ciuly
Contributor
Contributor
‎10-22-2012 06:08 AM

I see you are doging my question. if you want any more help, please answer ALL the following questions:

Do you know what the vsphere client is?

Are you using it?

Can you connect with it to your ESX that has the passthrough GPU?

Can you see your VM that us configured for that passed through GPU in the vsphere client?

Can you see anything on the CONSOLE of that VM in the vpshere client? If so, what is it?

Thank you.

VMDirectPath with ATI GPU document https://docs.google.com/spreadsheet/ccc?key=0Aqp_xYBwP_Y7dE5EclhtaDdIV09lNWxfODd1alRUTlE
Reply
0 Kudos
Bhargava_Shriva
Contributor
Contributor
‎10-22-2012 06:33 AM

Guys well here is the answer

Do you know what the vsphere client is?

yes I do and i am using it

Are you using it?

Yes without tat how can I bypass graphic card.

Can you connect with it to your ESX that has the passthrough GPU?

It works fine until I enable bypass for graphic card once i bypass GPU and restart ESX, ESX host will hung when it comes to cnic_register loading successfull

Can you see your VM that us configured for that passed through GPU in the vsphere client?

No after bypassing GPU esx is not coming up it comes to cnic_register loading successful and hangs

Can you see anything on the CONSOLE of that VM in the vpshere client? If so, what is it?

No

Reply
0 Kudos
Bhargava_Shriva
Contributor
Contributor
‎10-22-2012 06:40 AM

And more over I have tried when vm’s are present in ESX and more than 4 times by doing fresh esx installation and trying to bypass GPU. Kindly help me out.

Reply
0 Kudos
KRA77
Contributor
Contributor
‎10-22-2012 06:51 AM

Try different pci-e slot if possible, try different gfx card (ATi).

Reply
0 Kudos
Bhargava_Shriva
Contributor
Contributor
‎10-22-2012 08:07 AM

Sorry we don’t have ATi GPU in our environment I again tried using

HP Z400 workstation with

  • 24 GB RAM
  • 500 GB HDD
  • 2 GB Quadro 4000

Guys its simple Quadro 4000 will support virtualization positively and all Nvidia quadro and tesla series. Kindly help me out.

Reply
0 Kudos
ciuly
Contributor
Contributor
‎10-22-2012 09:13 AM

I think you have a terible miss-understanding of what the vshpere client is, what the host is, what the VM console is.

What you see on the monitor with cnic_register that is the esx host, not the VM console. YOu are not looking at the monitor to see the VM Console, you are looking in the vsphere client to see the VM Console.

The vsphere client runs on another computer then the ESX computer. The ESX computer is named the host. You tipically install the vsphere client on a machine that is not inside the ESX (download it from here https://my.vmware.com/web/vmware/info/slug/datacenter_cloud_infrastructure/vmware_vsphere/5_1?rct=j&... )

When you start the vsphere client, it will ask you the ip address of the esx host, the root user and password. Once inside the vsphsere client you can see all the virtual machines that are on that esx host.

If you then click on a virtual machine from that list, you will see to the right a tabbed something, and one of those tabs is the VM CONSOLE. That is the place where you will see what the VM is displaying on its default VMWare adapter.

If all you did was to enable passthrough for the graphics card and assign it to the VM then you will always get display showing on this VM CONSOLE and NOT on the monitor. For that, you first need to go on the VM CONSOLE in the vsphere client, install the drivers, maybe enable the adapter and THEN, if you are lucky, you will get the monitor to display something.

Until you've bene in hte VM CONSOLE AND installed the drivers, you have no business looking at the monitor. You won't see anything else than what you've been seeing for the past many hours and htat's the cnic_register.

To overcome this cnic_register, add a secondary card, make that the default adapter for the ESX host so that the esx will display its stuff on that one, connect tit to a second monitor or KVM, and then you will see the esx stuff just fine without being interrupted by the passthrough.

Regarding your setup, I now see it's NVidia. I didn't pay attnetion to that detail since we're talking ATI here. We had no success with vmware and NVidia cards, however they are reported working under XEN if I recall correctly.

So you either have to change the hypervisor from vmware to something else OR, change the GPU to an ATI one.

You can chek the following document for a working setup in ESX

https://docs.google.com/spreadsheet/ccc?key=0Aqp_xYBwP_Y7dE5EclhtaDdIV09lNWxfODd1alRUTlE#gid=0

however keep in mind that to this date, our experience says that the mainboard, bios version, processor and GPU all play a role in getting a success story. So even if you have the right GPU, the MB or CPU might kill the game for you, same for the MB and so on.

VMDirectPath with ATI GPU document https://docs.google.com/spreadsheet/ccc?key=0Aqp_xYBwP_Y7dE5EclhtaDdIV09lNWxfODd1alRUTlE
Reply
0 Kudos
Bhargava_Shriva
Contributor
Contributor
‎10-22-2012 11:00 AM

Friend,

Don’t complicate your life making me understand the VM basics I may not be advanced but aware of basics.

I want achieve GPU bypass as you ppl have achieved in ATI, but using Nvidia because Nvidia has professional series of GPU like Quadro and Tesla these are suitable for graphic hungry application. GPU what u ppl have tested all are almost gamers graphic card. Tesla and quadro is meant for servers and virtualization environment. Tesla comes with 6 GB GPU where as in ATI we can’t find the same.

Reply
0 Kudos
KRA77
Contributor
Contributor
‎10-22-2012 11:05 AM

Like he said, there are no success story's with esxi and nvidia, besides this is a thread for ATi. If you plan on continue to try this and expect help I would suggest creating a thread called "VMDirectPath and Nvidia". But if you don't need vmware, go for Xen who is confirmed to be working with nvidia.

Reply
0 Kudos
derickso
Contributor
Contributor
‎10-22-2012 11:13 AM

The current combination of ESX and Nvidia just does not work for GPU pass through properly. From everything I have read, nobody has been able to get it to work with ESX, and only a couple people have made it work with Xen, and that was with major effort. I suggest you either switch to a Radeon based card, or give up for now.

Reply
0 Kudos