You could download the 11.2.3 full installer from the App Store, run it to overwrite your 11.3b1 install, then try the update to 11.3b4. It appears that's unlikely to work, but may be worth trying.
Otherwise, wait for the 11.3 release and hope it works. An update to VMware Fusion that attempts to fix this might also help.
My only Big Sur VM was created from scratch, so I can't test that easily.
