I have NSX 6.2.3 install on 6 UCS rack servers that are in a cluster, each servers have two NIC that is connected to two Cisco 9k, one nice on a server going to 9K A and next NIC going to 9k B. I have iBGP running between the ESG and the 9K. I found a problem that ping and BGP peering not working on some of the ESG, until I remove one half of the connections going to any of the 9K and just have the Exi host connect to one 9K.
Can someone tell me what I am not doing? And how can I have all EXi host connect at the same time to the 9K and have iBGP working on all the uplinks. 9k ports are config as trunk ports.
Below is the blog from Brad Hedlund explaining why there should be no dynamic routing over vPC peer links. The same applies in the NSX environment as well.
Make sure you are not using a vPC link to peer the ESGs with the upstream 9Ks as Cisco does NOT support this topology on the Nexus 9Ks.
Here is a link to Cisco's latest vPC design guide: http://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices...
Paul A. Mancuso VCDX | VCI | CCNP Data Center | CISSP
VMware NSBU; Technical Product Manager