@IlDavo and @orthohin, thanks very much for this. 

I was having similar problems with a Windows 10 VM guest on VMware Fusion 12.0.0 on MacOS Catalina 10.15.7. It seems that it was a video driver DLL hanging around from previous VMware Tools installations. Following @IlDavo's procedure sorted it out and now everything is working properly.