I am assuming that as you removed the VMs from VC, VCAC will not know that you've done that, and there will be other artifacts in that are out of sync. There is a way of safely removing these artifacts but it's not documented, and GSS VMware support may be able to help you.
Bit of background: There is a workflow that cleans up the static IP address no longer assigned to VM in a network profile. It is called ReclaimDestroyedStaticIPAddresses. Have a look in the workflow history under the infrastructure tab and search for ReclaimDestroyedStaticIPAddresses. You should see this running every 5 minutes and is should be reclaiming static IP addresses no longer assigned to a VM for you automatically. Also, have a look at this link - Increasing/Decreasing the workflow execution timing in vCAC 5.2 | Cloud Relevant. It should still be valid for 6 I believe, but it's not supported by VMware. I guess if you need to increase the frequency this workflow runs, then change the values in the DB. You are able to manually reclaim any addresses in between this workflow running if there are address that have not been reclaimed by the ReclaimDestroyedStaticIPAddresses can are in Destroyed state.
However, as you have removed the VMs from VC, the ReclaimDestroyedStaticIPAddresses workflow is likely to be running and you still have allocated IP addresses for the VMs you deleted in VC, then you might want to look at the table in the DB, StaticIPv4Address and see if you can figure out which machines are taking up the IP address and remove them, but again this is unsupported. Take a back up first too 
.
You could also first try to destroy the VMs within the managed machines tab under infrastructure. Might be your safest bet depending what state they are in. If this is a production system, log a ticket with GSS and they can safely remove the VMs from both CAFE and IAAS.
Oli