I was able to make some progress on this issue finally over the past week or so. Thanks to the Horizon API's in PowerCLI 6.5. The following applied to my environment, and is probably limited to a very specific scenario, but hopefully will be useful to other people too.
Using the Horizon API's that are in PowerCLI 6.5, i found the issue was NOT vcenter directly, but rather one of my four connection servers had an issue with its connection to vCenter. It appears that if any conn server has an issue connecting to vcenter, view admin flags vCenter as red and not the problematic connection server. View Admin and its lack of logging or good error messages didn't help aim me in the right direction at all (hint hint VMware, improve the logging and error messages for view admin server statuses please). Once i identified the conn server that had issues, restarting its connection server service resolved the issue for the time being. I still dont know why that one server has been having issues, but at least now i know it's not a vCenter issue and I can now start looking in the proper place to hopefully resolve this permanently. FWIW, this issue has not returned since restarting that service on that conn server.
1. Install powerCLI 6.5 and be sure the Horizon View API's are included in the installation wizard.
2. open PowerCLI and type connect-HVServer "servername" where servername is FQDN of one of your connection servers. You'll need to do this from a computer/server that can connect to your connection server. in my case, i did this on a connection server directly.
3. run this command, which queries the status of each connection server in relation to it's connection to vCenter: $Global:DefaultHVServers.extensiondata.VirtualCenterHealth.VirtualCenterHealth_LIST().ConnectionServerData.Status This command will return the status of all connection servers, not just the one you're running PowerCLI from.
4. if any of the servers show the STATUS field as STATUS_DOWN, this will cause vCenter in view admin to show as red even though the issue is actually with the connection server.
5. restart the connection server service on the connection server(s) showing STATUS_DOWN and in a couple of minutes vCenter should move to "green" in view admin
again, this is just one scenario that can cause vCenter to show as red in view admin but hopefully it can help others make some progress troubleshooting their environment.
shout out to this blog for getting me started with using the API
http://www.virtu-al.net/2016/12/08/getting-started-powercli-6-5-horizon-view/
and VMwares horizon API documentation for helping me find what i needed to in the API to find this fix
VMware API Explorer - VMware {code}
