It's not clear what do you mean by "single Active/Passive VCSA"?
Do you whant to manager all resources with one vcenter and make it high available?
Yes, that's correct.
is that possible?
There are multiple things we need to consider in your scenario.
1. you want to have a single vCenter instance, then the site B vCenter will not be used anymore.
2. you can not convert the site B 5.5 vCenter as a Sita-A's passive vCenter. Since the inventory DB all these are different
3. The only option is to delete the Site B vCenter and add all the host to Site A vCenter
4. Before deleting Site B vCenter you have to migrate all the hosts and VMs to a standard switch.
5. Make sure you SiteB ESXi hosts are not having any NAT to reach site A network.
Hope the above points will give some thoughts.
Minimal downtime is expected for the VMs when you flip the port group from VDS a to standard switch. also check the feasiblity for other integrated products with vCenter like SRM, vRA, vROPs, NSX, backup etc..
Agreed, if your backup software is integrated with site-B vCenter. then you have to reconfigure the backup software with site-A vCenter after the consolidation