I am having exactly the same problem.
Windows XP, boot camp partition, running on an intel iMac (dual core, 4GB), OS X 10.7.3
Upgraded from VM 3.x to 4.x
Same problems, current VM version: 4.1.1
If I wait long enough, maybe a half hour or longer. XP eventually becomes somewhat "useable".
I mainly use VM to launch XP to launch explorer to finally launch a remote desktop connection to my window maching at work.
If I can finally get there, I am then usually OK.
But what used to be less than 5 minutes is not a 30-45 minute task.