okay, one more important insight: I think it has to do with the console view - if I disable it, GUI goes smooth even with 2 RDP connections and all 4 VM's running.
Therefor I assume it has to do with colour settings or graphical effects being transmitted from guest's OS, GUI console view, RDP, RDP and so on ... or maybe mouse movement etc. which is being forwarded / transmitted from RDP to RDP to CUI to guest.
Something like this. Thankfully I don't need the GUI and doing RDP to the guests directly works fine too ... so, juest disable console view and things should be okay I guess.