12 Replies Latest reply on Oct 13, 2009 4:03 PM by Datto

    Datto's Inexpensive VMware FT Capable Lab Hardware

    Datto Expert

      For those of you interested in creating a low-cost

      (relatively speaking) lab for VSphere 4.0 that will also do VMware FT, here's

      what I've used for my three white box VMware FT setup.

       

       

       

       

       

       

       

      White Box 1 -- AMD AM2 Opteron 1354 (quad-core single socket

      CPU to save money - CPU cost is $85.00 each delivered from Newegg), Asus M2N-LR

      motherboard (about $100 delivered Open Box from Newegg, much less on eBay when

      available) and you'll need to get the latest M2N-LR BIOS installed but you may

      not be able to flash the board using the Opteron 1354 since the board may

      require a lesser processor to flash the BIOS -- I had a low-end AM2 3600 CPU

      sitting around and used that for flashing, then put in the AMD Opteron 1354),

      two PCI-X dual port Intel gigabit cards (I got mine for $16.50 delivered for

      each dual port card on eBay -- this combination of add-in NICs and the two

      gigabit NICs on the motherboard that work properly with ESX 4.0 give me six

      physical gigabit NICs in the white box), 8GB of non-ECC memory (4x 2GB GSkill

      PC2-6400 DDR 800 -- the board will take ECC memory if that's your preference

      but I had this GSkill memory already), 20G Maxtor IDE drive for $15.00 each as

      the ESX 4.0 boot drive. For CPU fan/.heatsync I'm using a Zerotherm NV120

      ($50.00 from Newegg or about 60% of that used on ebay when available). I don't

      put a permanent CD/DVD drive into any ESX host in my lab and just put the

      CD/DVD drive temporarily in when I need it. I also never use a permanent floppy

      drive and just plug in a USB floppy drive if I need it for say BIOS flashing.

       

       

       

       

       

       

       

       

       

      White Box 2 -- identical to White Box 1 above

       

       

       

       

       

       

       

       

       

      White Box 3 -- AMD AM2 Opteron 1356 (also a quad-core single

      socket CPU to save money and it's within the 400MHz speed difference limit required

      by VMware FT between CPU speed in White Box 1 & 2 above), Asus M2N-L

      motherboard, combination of PCI-e dual port Intel gigabit NICs in the two PCI-e

      16X slots and straight PCI 1000MT gigabit NICs to get to a total of 6 physical

      gigabit NICs in the white box, 8GB of ECC memory (this CPU, motherboard and

      memory came as a bundle I'd bought on eBay). Note I couldn't get the two

      existing Marvell physical gigabit NICs on the M2N-L motherboard to work with

      ESX 4.0 and haven't had time to chase down a NIC driver shoehorn process for

      the Marvell NICs so I just disabled the Marvell NICs in the system BIOS for now

      and use the other PCI-e and PCI gigabit NICs for my FT purposes. The CPU

      Heatsync is also a Zerotherm NV120. Boot drive is also a Maxtor 20GB IDE drive.

       

       

       

       

       

       

       

       

       

      Regular VMs (non-COS VMs) sit on an Openfiler 2.3 box

      providing shared iSCSI storage for the cluster of white boxes. One gigabit NIC

      on each box is dedicated to FT Logging-- the likely max number of FT Primary

      and Secondary VMs on any single box is likely three to six of these FT Primary

      or Secondary VMs before FT logging might get swamped. So far I'm running a

      total of five FT protected light-duty VMs in the cluster and they seem to have

      no problems and FT does correctly transfer over to the Secondary if the Primary

      FT protected VM fails. I'm also running non-FT VMs in the cluster and there are

      also two other ESX 4.0 boxes in the same cluster that are not FT capable (they

      have AM2 Kuma processors which are VMotionable between the Opteron 1354/1356 CPUs

      but ESX 4.0 needs a keystroke hit on the bootup on those Kuma boxes to keep the

      bootup from stopping -- something having to do with the Kuma / M2N-E

      motherboard combination used in those non-FT capable white boxes and ESX 4.0 --

      if I put a standard AM2 Brisbane CPU in those M2N-E boxes they boot normally

      for ESX 4.0 (but won't VMotion straight-away with the Opteron 1354 / 1356 CPUs)

      so the problem is with using Kuma processors and ESX 4.0). I don't down ESX

      boxes much so this isn't a problem for me at the moment.

       

       

       

       

       

       

       

       

       

      Note that the M2N-LR boxes will not go into Standby Mode for

      unknown reasons (regardless of which NIC is the Standby-capable VMkernel NIC)

      but the M2N-L box will sometimes go into Standby Mode so it's likely a BIOS

      issue with these Asus motherboards I suspect. The Asus M2N-E / Kuma CPU

      motherbaord/CPU combination have no problem going into Standby mode but with

      the Kuma processors in the M2N-E motherboards the boot back up process needs a

      keystroke hit so that's a problem.

       

       

       

       

       

       

       

       

       

      Also note the huge Zerotherm NV120 heasyncs cover the

      closest (the first) motherboard slot in every motherboard I use. Some people

      have just trimmed the fins of f the NV120 to make room as necessary.

       

       

       

       

       

       

       

       

       

      VCenter runs as a VM on another cluster.

       

       

       

       

       

       

       

       

       

      These are not a supported configuration so don't go using

      this in your company production setup. Hope this helps folks wanting to build a

      relatively economical VMware FT lab setup.

       

       

       

       

       

       

       

       

       

      Datto