Hi,
Not sure if you solved this already, but just in case...
I was having the same problem. Then I found out that the vcenter hostname is case sensitive for the VRMS appliance, but not for the SRM server, for example, if your vcenter hostname is "vCenter01.test.com" and you have registered "vcenter01.test.com" while installing SRM server, it will lead to this problem. SRM sites pairing will not complain, but VRMS pairing will. No need to reinstall anything, you just have to reconfigure the SRM connection on the remote SRM site and pair the SRMs again, after that, try to pair the VRMS appliance again, it will work.
Hope it still helps,
Regards