Hello,
I was thinking about scenarios like this, discussing them with our network guys and came up only with operational "pinning" of particular subnet to particular site.
So you either configure DRS rules (in case of stretched cluster) and/or write some operational procedures to your staff (maybe even automate VM provision to reduce probability of human error).
For example, you have subnet 192.168.aaa.0/24 (VNI 5aaa) and 192.168.bbb.0/24 (VNI 5bbb) and decide that VMs, connected to VNI 5aaa, should run in site A, and VMs, connected to VNI 5bbb - in site B.
Then you must announce those networks to the external world following way (using OSPF or BGP or anything else):
- preferred route to 192.168.aaa.0/24 is via site A border router, non-preferred route is via site B
- preferred route to 192.168.bbb.0/24 is via site B border router, non-preferred route is via site A
This way you have routing resilience in case of site outage, but if your operational "pinning" is somehow broken (for example, VM, connected to VNI 5aaa, ended up in site B), you have asymmetrical trafic which most likely will be dropped by your ESGs (as they are firewalls).
I guess, there is just no "easy and clean" solution for this, you have to compromise somehow (maybe leave firewalling on DFW level only, then traffic can be as asymmetric as it happens).
Maybe you should ask yourself first, what is more important in this scenario: traffic locality, ability to freely move VMs between sites or something else.
Another possibility is described in this great blog post http://www.routetocloud.com/2016/02/nsx-dual-activeactive-datacenters-bcdr. Look up "Active/Active Datacenter with mirrored ESGs" section. But there is no traffic locality in this solution.
And something about local ingress :smileylaugh: I've stumpled upon another great post some time ago: http://https://networkinferno.net/ingress-optimisation-with-nsx-for-vsphere It is not supported of course, but I hope we will see functionality like this in short time.
