For those of you interested in creating a low-cost
(relatively speaking) lab for VSphere 4.0 that will also do VMware FT, here's
what I've used for my three white box VMware FT setup.
White Box 1 -- AMD AM2 Opteron 1354 (quad-core single socket
CPU to save money - CPU cost is $85.00 each delivered from Newegg), Asus M2N-LR
motherboard (about $100 delivered Open Box from Newegg, much less on eBay when
available) and you'll need to get the latest M2N-LR BIOS installed but you may
not be able to flash the board using the Opteron 1354 since the board may
require a lesser processor to flash the BIOS -- I had a low-end AM2 3600 CPU
sitting around and used that for flashing, then put in the AMD Opteron 1354),
two PCI-X dual port Intel gigabit cards (I got mine for $16.50 delivered for
each dual port card on eBay -- this combination of add-in NICs and the two
gigabit NICs on the motherboard that work properly with ESX 4.0 give me six
physical gigabit NICs in the white box), 8GB of non-ECC memory (4x 2GB GSkill
PC2-6400 DDR 800 -- the board will take ECC memory if that's your preference
but I had this GSkill memory already), 20G Maxtor IDE drive for $15.00 each as
the ESX 4.0 boot drive. For CPU fan/.heatsync I'm using a Zerotherm NV120
($50.00 from Newegg or about 60% of that used on ebay when available). I don't
put a permanent CD/DVD drive into any ESX host in my lab and just put the
CD/DVD drive temporarily in when I need it. I also never use a permanent floppy
drive and just plug in a USB floppy drive if I need it for say BIOS flashing.
White Box 2 -- identical to White Box 1 above
White Box 3 -- AMD AM2 Opteron 1356 (also a quad-core single
socket CPU to save money and it's within the 400MHz speed difference limit required
by VMware FT between CPU speed in White Box 1 & 2 above), Asus M2N-L
motherboard, combination of PCI-e dual port Intel gigabit NICs in the two PCI-e
16X slots and straight PCI 1000MT gigabit NICs to get to a total of 6 physical
gigabit NICs in the white box, 8GB of ECC memory (this CPU, motherboard and
memory came as a bundle I'd bought on eBay). Note I couldn't get the two
existing Marvell physical gigabit NICs on the M2N-L motherboard to work with
ESX 4.0 and haven't had time to chase down a NIC driver shoehorn process for
the Marvell NICs so I just disabled the Marvell NICs in the system BIOS for now
and use the other PCI-e and PCI gigabit NICs for my FT purposes. The CPU
Heatsync is also a Zerotherm NV120. Boot drive is also a Maxtor 20GB IDE drive.
Regular VMs (non-COS VMs) sit on an Openfiler 2.3 box
providing shared iSCSI storage for the cluster of white boxes. One gigabit NIC
on each box is dedicated to FT Logging-- the likely max number of FT Primary
and Secondary VMs on any single box is likely three to six of these FT Primary
or Secondary VMs before FT logging might get swamped. So far I'm running a
total of five FT protected light-duty VMs in the cluster and they seem to have
no problems and FT does correctly transfer over to the Secondary if the Primary
FT protected VM fails. I'm also running non-FT VMs in the cluster and there are
also two other ESX 4.0 boxes in the same cluster that are not FT capable (they
have AM2 Kuma processors which are VMotionable between the Opteron 1354/1356 CPUs
but ESX 4.0 needs a keystroke hit on the bootup on those Kuma boxes to keep the
bootup from stopping -- something having to do with the Kuma / M2N-E
motherboard combination used in those non-FT capable white boxes and ESX 4.0 --
if I put a standard AM2 Brisbane CPU in those M2N-E boxes they boot normally
for ESX 4.0 (but won't VMotion straight-away with the Opteron 1354 / 1356 CPUs)
so the problem is with using Kuma processors and ESX 4.0). I don't down ESX
boxes much so this isn't a problem for me at the moment.
Note that the M2N-LR boxes will not go into Standby Mode for
unknown reasons (regardless of which NIC is the Standby-capable VMkernel NIC)
but the M2N-L box will sometimes go into Standby Mode so it's likely a BIOS
issue with these Asus motherboards I suspect. The Asus M2N-E / Kuma CPU
motherbaord/CPU combination have no problem going into Standby mode but with
the Kuma processors in the M2N-E motherboards the boot back up process needs a
keystroke hit so that's a problem.
Also note the huge Zerotherm NV120 heasyncs cover the
closest (the first) motherboard slot in every motherboard I use. Some people
have just trimmed the fins of f the NV120 to make room as necessary.
VCenter runs as a VM on another cluster.
These are not a supported configuration so don't go using
this in your company production setup. Hope this helps folks wanting to build a
relatively economical VMware FT lab setup.
Datto