Odd. I'm running Slackware 11 in a VM running on a Slackware 11 host and X only consumes 0.3% CPU using top on a bash screen in KDE 3.5.4 when idling.
Are you running on a desktop or laptop system?
This is a Dell desktop with an Nvidia card.
How much memory your host OS is having?
How much memory does your guest OS is having?
Are you using the vmware-tools for linux?