I have the same exact lab (esxi 6.7u2+vc 6.7)with same network setup exactly and also same issue
Did you find solution for this issue ?
Hey, yeah I've managed to fix it. I just kept it simple and used a portgroup with no physical up-links, worked fine that way. Maybe my issue was originating from using MTU 9000 on my physical switch. You can still set it to 9000 but on the isolated port-group (distributed switch actually).