Im dumb, somehow I thought it was OK to have the VSAN's configured as DHCP. I do not know why exactly resetting the fw/router broke VSAN 2 switches away, but thats the way she goes.
To make matters more difficult my Photon machine was in that VSAN.
I have since re-ip'd the VSAN's on each host. I can ping each vmnic, but they just are not communicating. I have looked for relevant logs but dont see anything.
This is a flat network, no vlans, fw's open for lan.
Attached are what I believe to be relevant pictures.
Thanks for your time
Hello fRactIalz00m
Is this a vSAN 6.5 0d or later cluster?
If so, check the Unicast agent list and repopulate it if necessary if the IPs have changed (or are incorrect)
# esxcli vsan cluster unicast list
This should only contain the information of all other hosts in the cluster, not the host you are checking this on.
Bob
This brought me to the right direction and im back in business.
Thank you Bobkin!