I think your best option would be the VMCI API. It was in experimental stage a while ago, but with ESX 4.0 onwards, it is built in. I think it is now called
VMCI sockets 1.0.
Here is the documentation: http://www.vmware.com/support/developer/vmci-sdk/