OK, I found that if I enable 3D Acceleration in the VM settings, it works.
But that is not a solution right now, because enabling 3D acceleration causes random full system crashes (in the Windows guest) as described here:
http://communities.vmware.com/message/2160012#2160012
So I guess Retina support is not ready for prime time. (It is also half-baked in Windows itself; if you set the UI settings in Windows to 150% or 200%, things look good, but various old legacy apps don't support those settings, and (astoundingly) neither does any of the new freak-show Windows 8 UI.)