Shouldn't be a host starving problem.  If you're using unity or full screen mode, does it happen when in windowed mode?

One other thing - make sure you set 'prevent host from sleeping when screen blanks' in system preferences/energy, and set it to 'never' sleep.  Sleeping a host while a guest is running is something that doesn't always work (sometimes it does, but others it doesn't).