First off, Are those SATA or SAS? I would guess you meant SAS drives (since SATA comes in 150GB Sizes and SCSI/SAS is usually refered to as 146GB Sizes
Seconds, make sure that controller has a BBWC (batter backup cache) or its not going to matter alot which configuration you have, your performance is going to be pretty bad.
Next, it all depends on how many VM's you want to run, the type of performance you expect to require, and your comfort with drive failure effects.
If you only have 5x146GB Drives, then I would go with maximum space configuration or maximum performance
RAID 5 is going to give you 530GB or so...30GB used for VMWARE Installation and 500gb for VM's... In a small test or non busy environment thats upwards of 30VM's (more then recommended on a single volume, but overall I doubt you will get that many VM's anyhow. More likely to have fewer with larger disk sizes.
If you want pure performance and recovery, then go with 4xRAID0+1 and a hotspace...thats roughly 268GB of vmfs space onboard...or safely 14 - Windows 2008 Standard VM's with standard disk size of 15GB (Windows 2003 usually has drive of 12-15GB on average) and that leaves space for snapshots,etc.
You could also go with 2x146Gb RAID1, 3xraid 5 - but since ESX doesn't need that much space (134Gb) you will end up with a roughtly 100gb vmfs volume and a 268Gb volume.
I just realized you said esxi so not much space at all for the install...certainly not 30gb...
