Ah, so I'm not the only one who has run into this.
I have poked around a bit and have been able to determine that the install scripts are triggering the failure on the existence of /proc/xen. As best as I have been able to tell, recently Redhat has added the Xen balloon driver to even the "normal" (non-xen) kernels (in my case, both i686 and x86_64 versions of the 2.6.9-78.ELsmp kernel). I have not yet been able to figure out if there is a way to disable this within the kernel or not, short of rolling your own kernel.
The workaround I performed to get the install to complete was to revert to an earlier kernel version -- 2.6.9-67.0.7.ELsmp seems to be okay for me. You might be able to get away with switching back to the -78 kernel line after the install without ill effects, although I have not tried this. Considering that RC1 was running just fine with the Xen balloon driver sitting around, I'd guess it would work just fine.
I'd be interested in hearing a more "official" opinion on this matter, perhaps from someone actually affiliated with VMware, as this seems to be a legitimate "bug" -- perhaps a better method to determine if the kernel is truely "xen-enabled" is necessary.
That is exactly the same conclusion I came to this morning. I tried an lsmod in order to determine if any xen modules had snuck into my kernel; results = none. I also tried turning of Hyper Threading in the BIOS to no avail. A co-worker suggested turning off virtualization. Alas, my processors don't have such a capability. As it turns out, I still have a 2.6.9-67.0.7.ELsmp kernel available in /boot so I am going to give that a try.
Thanks fo pointing me in the right direction.
I can confirm that RHEL4 Update 4 with a kernel at 2.6.9-78.0.1.ELsmp does not install correctly. The existance of /proc/xen seems to throw off the install script. I rebooted on a 2.6.9-67.ELsmp and all went well. Hope this helps! Also, thanks to all for your help especially ktower.
Yup, thanks for catching that. We've got folks looking into the issue.