We tried the following configuration and the issue did not appear:
VMware Tools 12.0
Installed the 9.17.1.2 SVGA driver as part of the VMware Tools install
Horizon Agent 8.3 // Horizon Agent 8.2
Did not install the VMVideo option
Confirmed the Display Adapter was still using the VMware Tools SVGA driver
Teradici Firmware 21.01.3 // 21.01.2 // 21.03
The above configuration works without issue. Substituting out the Horizon 8.3 (or 8.2) Agent w/ Horizon 8.4 Agent (same install script) and using the same Teradici firmware will result in the latency and artifact issue on the rotated display. When using the Horizon Client (from physical machines) to connect to the virtual desktop, and using a rotated portrait display, we do not see the issue.