Well, a problem that I see is when you upgrade the witness-appliance. It' a rip-and-replace procedure. You delete the old appliance, build a new one by deploying a newer appliance-version etc. and all that time, both nodes are going bezerk because the witness is gone.
Don't forget that while the witness is unavailable, you cannot produce new objects. You can forget about making snapshots or powering on a VM etc. as all such actions produce new objects (snapshot-object, swap-object etc.)
And what if you simply reboot te witness. Not a problem as it's quite fast but both nodes will freak out.
Something tells me this is a bad idea... 😉
Read this: Isolation Address in a 2-node direct connect vSAN environment? - Yellow Bricks
Or if the buildings are quite far apart and have their own physical uplinks to site 3, it's actually becoming a regular stretched cluster. Then this best practise applies: vSphere HA heartbeat datastores, the isolation address and vSAN - Yellow Bricks
