vApp lan is not working for few VMs

[srl101]

In my cloud, I have deployed many vApps from the same template. VMs within a vApp use vApp lan to communicate.. Since the vApp LAN is not connected to outside, all the vApps use the same IPs for vApp LAN. I have not seen any problems till now. But now, few VMs are not able to communicate over the vApp Network. Ex, If I have 5 VMs in the vApp, 3 VMs are able to communicate with each other but other two are not. There are no events/alarms are reported related to networks in vCNS/vCloud/vCenter. I am facing this issue multiple times now. \

I am using VCNI backed network pool.

What could be the problem here?

[cfor]

First thing I would attempt for troubleshooting is to move all the VMs in a broken vApp to the same host, also move the vse-edge VM (if one exists for the vApp network).

Then test... If something is broken still, then it is not VCNI, but something else, maybe host firewall.  However (and what I expect to see) is that all the VMs will work.  If this is the case then take one of the VMs that has been broken, and move it to a different host (and each host in your cluster) ; see if it still works.  That will tell you if one or more of the hosts is having issues talking on the VCNI VLAN.

All that said a second thing to check is to clear the ARP tables of the upstream routers, VCNI has some issues where the tables on some switches can get corrupt, a clear will tell you if that is the case.  (I know for some switches we used in the past we had to tell the apr tables to clear often). 

When all done take a look at if VxLan is an option for you, it really works much better.

[_morpheus_]

If only one instance of this vApp is running, does the problem go away, and then problem comes back when more instances of the vApps are running?

[srl101]

Issue found , we have two enclosures and the interconnect cable had some problem. Since the problem was with the interconnect, VMs which are present on hosts in one enclosure are not able to communicate with the VMs present in the other enclosure and the VMs which are present in the same enclosure were able to communicate among themselves.

Thanks for the reply..