Hi all! OP here, a few more notes that I don't think I included above--
pciPassthru0.msiEnabled (where 0 is replaced with whichever device is your GPU) must be set to FALSE in advanced vm settings for the GPU closest to the CPU, otherwise you get constant crashing of the video driver (often with immediate restart and no bluescreen, looks like a slow computer with a screen that occasionally blinks). Beware, because if you change or delete the device and then re-add, you need to redo this.
Very important post on changing the esxi host configuration to prevent random core memory hopping. This was on reddit and made a substantial difference, reducing or eliminating microstutter on all the VM's. I also no longer needed to pin them to specific ccx's. This is in the reddit post titled AMD EPYC on ESXi 6.5-6.7 NUMA issues: Mostly Resolved
Using 6.7 U1 now, can use EFI in VM's instead of BIOS (used to hang if USB passhtru).
Ran into a problem where any VM would slam to 100% disk usage under heavy load (like say a file download) and become almost unresponsive for > 10 minutes after download stopped. Eventually tracked it down to meta corruption on the underlying VMFS. VOMA doesn't like vmfs 6 (didn't last week) so I migrated to a new vmfs, problem solved.
Other notes:
The motherboard appears to retrain its BIOS (and renumber PCI devices ) if you take any out or put any in, potentially breaking more vm's until you re-activate for passthru. You also need to reset lots of bios settings.
Very interestingly, I discovered my system overheating from a dying Enermax Liqtech TR4 AIO. This caused really "interesting" features like the system abruptly (failing to POST) after changing memory settings, even though it had been working fine a minute before. Even worse, if it DID memory training when hot, it would set to super low speeds. This sounds easy, but with not way to check CPU temps outside of BIOS, it was slightly less straightforward (anyone know another way to check CPU temp under esxi with a threadripper? or any of the vm's?)
