Skip navigation
VMware
4,112 Views 12 Replies Last post: Oct 13, 2009 4:03 PM by Datto RSS
Datto Expert 369 posts since
Mar 10, 2006
Currently Being Moderated

Jun 26, 2009 12:46 PM

Datto's Inexpensive VMware FT Capable Lab Hardware

For those of you interested in creating a low-cost

(relatively speaking) lab for VSphere 4.0 that will also do VMware FT, here's

what I've used for my three white box VMware FT setup.

 

 

 

 

 

 

 

White Box 1 -- AMD AM2 Opteron 1354 (quad-core single socket

CPU to save money - CPU cost is $85.00 each delivered from Newegg), Asus M2N-LR

motherboard (about $100 delivered Open Box from Newegg, much less on eBay when

available) and you'll need to get the latest M2N-LR BIOS installed but you may

not be able to flash the board using the Opteron 1354 since the board may

require a lesser processor to flash the BIOS -- I had a low-end AM2 3600 CPU

sitting around and used that for flashing, then put in the AMD Opteron 1354),

two PCI-X dual port Intel gigabit cards (I got mine for $16.50 delivered for

each dual port card on eBay -- this combination of add-in NICs and the two

gigabit NICs on the motherboard that work properly with ESX 4.0 give me six

physical gigabit NICs in the white box), 8GB of non-ECC memory (4x 2GB GSkill

PC2-6400 DDR 800 -- the board will take ECC memory if that's your preference

but I had this GSkill memory already), 20G Maxtor IDE drive for $15.00 each as

the ESX 4.0 boot drive. For CPU fan/.heatsync I'm using a Zerotherm NV120

($50.00 from Newegg or about 60% of that used on ebay when available). I don't

put a permanent CD/DVD drive into any ESX host in my lab and just put the

CD/DVD drive temporarily in when I need it. I also never use a permanent floppy

drive and just plug in a USB floppy drive if I need it for say BIOS flashing.

 

 

 

 

 

 

 

 

 

White Box 2 -- identical to White Box 1 above

 

 

 

 

 

 

 

 

 

White Box 3 -- AMD AM2 Opteron 1356 (also a quad-core single

socket CPU to save money and it's within the 400MHz speed difference limit required

by VMware FT between CPU speed in White Box 1 & 2 above), Asus M2N-L

motherboard, combination of PCI-e dual port Intel gigabit NICs in the two PCI-e

16X slots and straight PCI 1000MT gigabit NICs to get to a total of 6 physical

gigabit NICs in the white box, 8GB of ECC memory (this CPU, motherboard and

memory came as a bundle I'd bought on eBay). Note I couldn't get the two

existing Marvell physical gigabit NICs on the M2N-L motherboard to work with

ESX 4.0 and haven't had time to chase down a NIC driver shoehorn process for

the Marvell NICs so I just disabled the Marvell NICs in the system BIOS for now

and use the other PCI-e and PCI gigabit NICs for my FT purposes. The CPU

Heatsync is also a Zerotherm NV120. Boot drive is also a Maxtor 20GB IDE drive.

 

 

 

 

 

 

 

 

 

Regular VMs (non-COS VMs) sit on an Openfiler 2.3 box

providing shared iSCSI storage for the cluster of white boxes. One gigabit NIC

on each box is dedicated to FT Logging-- the likely max number of FT Primary

and Secondary VMs on any single box is likely three to six of these FT Primary

or Secondary VMs before FT logging might get swamped. So far I'm running a

total of five FT protected light-duty VMs in the cluster and they seem to have

no problems and FT does correctly transfer over to the Secondary if the Primary

FT protected VM fails. I'm also running non-FT VMs in the cluster and there are

also two other ESX 4.0 boxes in the same cluster that are not FT capable (they

have AM2 Kuma processors which are VMotionable between the Opteron 1354/1356 CPUs

but ESX 4.0 needs a keystroke hit on the bootup on those Kuma boxes to keep the

bootup from stopping -- something having to do with the Kuma / M2N-E

motherboard combination used in those non-FT capable white boxes and ESX 4.0 --

if I put a standard AM2 Brisbane CPU in those M2N-E boxes they boot normally

for ESX 4.0 (but won't VMotion straight-away with the Opteron 1354 / 1356 CPUs)

so the problem is with using Kuma processors and ESX 4.0). I don't down ESX

boxes much so this isn't a problem for me at the moment.

 

 

 

 

 

 

 

 

 

Note that the M2N-LR boxes will not go into Standby Mode for

unknown reasons (regardless of which NIC is the Standby-capable VMkernel NIC)

but the M2N-L box will sometimes go into Standby Mode so it's likely a BIOS

issue with these Asus motherboards I suspect. The Asus M2N-E / Kuma CPU

motherbaord/CPU combination have no problem going into Standby mode but with

the Kuma processors in the M2N-E motherboards the boot back up process needs a

keystroke hit so that's a problem.

 

 

 

 

 

 

 

 

 

Also note the huge Zerotherm NV120 heasyncs cover the

closest (the first) motherboard slot in every motherboard I use. Some people

have just trimmed the fins of f the NV120 to make room as necessary.

 

 

 

 

 

 

 

 

 

VCenter runs as a VM on another cluster.

 

 

 

 

 

 

 

 

 

These are not a supported configuration so don't go using

this in your company production setup. Hope this helps folks wanting to build a

relatively economical VMware FT lab setup.

 

 

 

 

 

 

 

 

 

Datto

V1RTU4L Lurker 1 posts since
Dec 14, 2006
Currently Being Moderated
3. Jul 17, 2009 3:12 PM in response to: Datto
Re: Datto's Inexpensive VMware FT Capable Lab Hardware

Hi Datto,

 

thanks for that guide.

 

Is there a reason why you use OpenFiler instead of for example FreeNAS ? I do not really know OpenFiler. Do you know it has a better performance than FreeNAS ?

 

thx in advance

Symbion Tech Novice 9 posts since
Aug 29, 2008
Currently Being Moderated
5. Jul 27, 2009 3:21 AM in response to: Datto
Re: Datto's Inexpensive VMware FT Capable Lab Hardware

I set up two openfiler boxes recently with DRBD and heartbeat to create a highly availabile SAN. I've done it many times now and there's an excellent guide out there:

 

 

 

 

 

 

 

http://www.howtoforge.com/installing-and-configuring-openfiler-with-drbd-and-heartbeat

 

 

 

 

 

 

 

 

 

I think it's the same article that i used from the http://www.the-mesh.org/ which appears to be down just now.

 

 

 

 

 

 

 

 

 

I built my whitebox last year with the following spec:

 

 

 

ESX 3.5 U2 running on:

 

 

 

ASUS P5WDG2-WS Pro (2 x PCI-X)

 

 

 

Q6600

 

 

 

8GB OCZ ReaperX DDR2 1066 (running at 800 as overclocking to 1066 was not stable)

 

 

 

Intel MT 1000 Dual Port (in PCI-X slot)

 

 

 

Adaptec 2130SLP PCI-X 128MB RAID Controller

 

 

 

2 x 15k 36GB in RAID 1

 

 

 

2 x 15k 146GB in RAID 1

 

 

 

HP rm435 DVD-ROM

 

 

 

Now i'm looking to put together two boxes for 4.0 and one for openfiler (two would be a waste just for my home lab) the same as yourself.

 

 

 

I was going to go with a P45 based board and get 16GB per box (OCZ PC6400 4x4GB for each one). Q9550's for the processors and i looked up vm-help and there's pic-e dual & quad port nics that work with 4.0. At least six ports per box (preferably 8). The on-board nics are no use on all the 4.0 compatible P45 boards i've looked at. Your setup is excellent and far less expensive but I think it may be worth the extra cost to get the 16GB.

 

 

 

 

 

 

 

 

 

For my openfiler box i'll use my existing whitebox and will replace the SCSI controller and drives with 8 X 1GB SATA II drives (not decided yet on which ones) and a PCI-E RAID controller.

 

 

 

What do you think?

32bit Lurker 2 posts since
Oct 12, 2009
Currently Being Moderated
7. Oct 12, 2009 10:38 AM in response to: Datto
Re: Datto's Inexpensive VMware FT Capable Lab Hardware

 

I've got the M2N-LR running with the latest BIOS 0515 and a Phenom 9650 (the 4x2.3Ghz thing) but ESXi doesn't liky computer (have a look at the attachement).

 

 

Does someone have a solution für my problem?

 

 

Attachments:
Gabrie Master vExpert 1,080 posts since
Jun 6, 2005
Currently Being Moderated
8. Oct 12, 2009 12:51 PM in response to: Datto
Re: Datto's Inexpensive VMware FT Capable Lab Hardware

Just wanted to add my whitebox that supports FT:

http://www.gabesvirtualworld.com/?p=531

 

This is what I bought:

- Asus Barebone V3-P5G45 iG45, SATA2 RAID, HDMI (zilver)

- Intel Core 2 Quad Q9400 2.66GHz FSB1333 Box

- Intel PCI-e Adapter Pro1000PT Dual Port LAN 1000Mbit Bulk

- 2x OCZ 2×2GB DDR2 SDRAM PC6400 CL5.0 Platinum (Total 8GB)

- Western Digital 640GB SATA300 16MB, WD6400AACS

 

(Warning, the onboard nic will not work in ESX, I therefore added the dual port nic as an extra)

 






http://www.GabesVirtualWorld.com

http://www.GabesVirtualWorld.com
Dan_Dada22 Enthusiast 56 posts since
Oct 16, 2006
Currently Being Moderated
9. Oct 13, 2009 1:30 PM in response to: Gabrie
Re: Datto's Inexpensive VMware FT Capable Lab Hardware

 

From where did you buy the "Asus Barebone V3-P5G45 iG45, SATA2 RAID, HDMI (zilver)"?  I am not able to find it online in the States.

 

 

Gabrie Master vExpert 1,080 posts since
Jun 6, 2005
Currently Being Moderated
10. Oct 13, 2009 1:33 PM in response to: Dan_Dada22
Re: Datto's Inexpensive VMware FT Capable Lab Hardware
http://www.GabesVirtualWorld.com

Bookmarked By (0)

Share This Page

Communities