No, if you REMOVE the hosts then you may lose your VM's. One thing you should do before you begin is to change the IP in Virtual Management Server Configuration under Administration in the VIC. The IP should match the IP of your new VC.
Then you can simply reconnect a host. It says it will move them (because the program is set to bring up certain dialogs) but it won't. As long as your new VC points to the same database, your settings are fine.
Reconnecting your hosts may prompt for root credentials of the host, but everything should be fine.