By "stable build" of Fusion. are you referring to Fusion 13 on the same M1 Mac?

From what I'm seeing on a LG Ultrawide (non-retina) monitor, if I set the resolution to a custom non-native value, the Tools within the VM up that value as my resolution when going full screen. It doesn't go to native resolution of the display.  

It almost seems like it's thinking that since you have a custom resolution on a Retina display, it assumes that your custom 1680x1050 resolution is a Retina/HIDPI setting and doubling it to get the 3360x2100 when you uncheck the "Use full resolution for Retina display".

I'm not sure if that's intentional or not.