Check out the Firewall configuration section (pg 32) of the View Administrator's Guide (http://www.vmware.com/pdf/viewmanager3_admin_guide.pdf).
You could firewall the ports on the IP addy's you don't want the View Administrator to use. That will block clients from connecting to the 'bad' IP addy's on those ports.
Port information can be found on Pg 34.
I'm sure there's a fancier way to do it, but I'm just a QA Monkey, not a full blown sysadmin 😛
-Lazlo...