What about VMDirectPath?

I am trying to combine a Linux PVR backend, and 3 frontends that have Nvidia cards with HDMI out ( all on one shelf with HDMI out to rooms )

So if I could run a vm for backend recording via USB, and 3 frontends that had a dedicated card for HDMI out, that would save the number of machines and power I use.

Thoughts?