VMware Cloud Community
VM0Sean
Enthusiast
Enthusiast
Jump to solution

ESXi budget server setup?

Hello!

I know that this question has been asked several times but not recently.  I'm looking to use desktop/workstation-class hardware for a budget ESXi server.  We're looking to run VMs that need 4GB of ram each and fairly decent CPU speed.  Ideally being able to run 3 concurrently.

Preconfigured servers are too pricy for what we want considering that we can setup something ourselves for 2/3rds the price.

What I'm really wondering is what processor / mobo combination people recommend.  I know that people swear by Intel but you get a lot more cores from AMD these days which seems to be more important.  I'd also prefer to have embedded net and especially SATA/raid that works with ESXi but we can add-on if necessary.

The official hardware support list pretty much only covers server-class hardware but I know that a lot of people run on desktop class hardware.  I have looked at the Ultimate White Box guide and see a lot of options there - but most seem to be more than 1yr old.  So what do people suggest?

The Asus M4A78 seemed to be a very popular series but you can't buy it new anymore - Is the M4A88 or M4A89 series just as good?

For example the Phenom II X6 seem to be pretty cheap ($200 CDN) and a whole variety of motherboards under $100.  Adding in 16GB of ram and a few hard drives with a raid card and intel network adapter and we should have a powerful system for well under $1000.

Thanks in advance!

Tags (2)
Reply
0 Kudos
1 Solution

Accepted Solutions
DSTAVERT
Immortal
Immortal
Jump to solution

I would suggest that you try http://vm-help.com or http://ultimatewhitebox.com

Good luck

-- David -- VMware Communities Moderator

View solution in original post

Reply
0 Kudos
9 Replies
DSTAVERT
Immortal
Immortal
Jump to solution

In my opinion, while you can build an ESXi host machine from desktop components, you end with an unsuitable platform for a production environment. If you look you can find, end of production, datacenter returns, overstock, refurbished, server machines from Dell, HP, IBM, etc etc.

Building yourself means you are responsible for understanding the relationship between the components. Does this controller work well with those hard drives or motherboard. Just because things function well in a desktop environment or a standalone file server doesn't mean they will work well when it is called upon to support multiple operating systems all competing for resources and disk IO. Real servers are created with a specific set of components. The firmware is updated as a complete package down to individual hard drives. If there are issues the manufacturer takes care of creating new firmware to correct some timing error or . . . .

Do yourself and your organization a favor. Look for a real mainstream server.

-- David -- VMware Communities Moderator
Reply
0 Kudos
VM0Sean
Enthusiast
Enthusiast
Jump to solution

Although I appreciate the opinion buying full server for us at the moment is not an option.  Does anyone have any suggestions as to recent desktop class motherboards and CPU combos that work well?

Thanks in advance!

Reply
0 Kudos
DSTAVERT
Immortal
Immortal
Jump to solution

My point is you can find a server within your budget if you look harder.

-- David -- VMware Communities Moderator
Reply
0 Kudos
VM0Sean
Enthusiast
Enthusiast
Jump to solution

... and I understand that, but for various reasons that I do not want to enumerate here that is not an option for us based on our requirements and limitations.  Any servers that we have been able to find that meet what we need are too expensive based on what we get out of them.  There are many people with perfectly working desktop based configurations so that's what we're looking for right now.

Reply
0 Kudos
DSTAVERT
Immortal
Immortal
Jump to solution

I would suggest that you try http://vm-help.com or http://ultimatewhitebox.com

Good luck

-- David -- VMware Communities Moderator
Reply
0 Kudos
VM0Sean
Enthusiast
Enthusiast
Jump to solution

Thanks for the reply - however as I mentioned in my post I've already looked and there's nothing recent there: everything is at least a year old.  For example there are a lot of people reporting success with Asus M4A785TD however that board is about 18 months old and not available anymore.  There are more recent versions (88T and 89T) but no mention of those yet so I don't know if they are good.  Hence why I'm asking here for someone who has done it recently Smiley Happy

Do you happen to know of any other repositories for not-officially-supported motherboard/cpu combos?  Both those sites are great but just aren't recent unfortunately.

Reply
0 Kudos
DSTAVERT
Immortal
Immortal
Jump to solution

The big problem with Desktop components is they often change even mid production run. What worked really well a month ago doesn't now or the current one doesn't work with someone's recommended controller. The forums are full of people's struggles with matching odd components. Most motherboards work. Most embedded components do not. Find supported controllers and NICs from the HCL

-- David -- VMware Communities Moderator
Phylum
Enthusiast
Enthusiast
Jump to solution

As much as it pains me to say it: He's right.  Try to find approved hardware on the lower end of the spectrum.

  1. Dell Outlet PowerEdge Servers - http://www.dell.com/us/dfb/p/servers
  2. Dell Outlet Data Storage and Backup - http://www.dell.com/us/dfb/p/storage-products
  3. Dell Outlet Precision Workstations (you might be blessed & find something that works) - http://www.dell.com/us/dfb/p/workstations
  4. HP Business outlet - http://www.hp.com/sbso/buspurchase_refurbished.html
  5. HP Renew - http://www.hp.com/united-states/renew/index.html

As you so aptly pointed out most of those posts are old.  This too frustrated/annoyed me.  But then I thought for a moment:

A Dell T110 can be had for 499 and its on the HCL, rivaling the cost (headache included) of a DIY/BTO system.

No wonder people abandonded rolling the dice or buying a 400$ motherboard only to have 3/5 components working.  If your budget is 1k that gives more flexibility.

I'm off my soap box - Now for more about what your ears/eyes are itching to hear/see:

You can build a pretty awesome system for 1K but will it work with ESXi?  That's a question we can't provide a guaranteed answer for.  I have a Q6600 nesting in Gigabyte GA-G33M-DS2R with an onboard ICH9R (RAID 0/1/5) controller.  I also have a Dell XPS 710 with a QX6700 and onboard RAID 0/1 and an AMCC 3ware 9500S-8 laying around.  With all of this wonderful hardware you would think its a no brainer:

  • Stand up two systems
  • Setup RAID 5
  • Install ESXi 4.1
  • Be on my merry way.

After lots of research, and some trial & error hoping that things had changed between then and now (per the whitebox sites), I was only able to get ESXi 3.5i running on the Gigabyte but without RAID.  Its not what I would consider efficient use of the machine but it works.  I ran ESXi 4.0 on a Dell Optiplex 760 (core 2 duo 6GB RAM 160GB SATA) for a year, and running one Win 7 VM and one Win XP VM concurrently brought it to its knees.  (Darn I/O!)  I only mention that because you may run into performance issues if running more than one VM per SATA disk.  All in all, it works, but its a unsupported setup.  I've also got a Tyan Thunder K8SD Pro (S2882-D) laying around but I have reservations about not being able to use some of the built-in features of the mobo.  Is it worth the trouble?

If you're going to go the BTO route,

You may also want to consider updating the whitebox HCL's with your findings if you go down this route.

VM0Sean
Enthusiast
Enthusiast
Jump to solution

Thanks for the input everyone!  We ended up with a pretty amazing setup that I'm stoked about.

We ended up putting together three servers using Asus M4A88TD-V motherboards and an Athlon X6 1100T with 16GB of RAM.  Using an older computer as a file server we put in 5x2TB drives and used FreeNAS to create a 7.3TB (usable) ZFS array shared over iSCSI.

We wound up with:

- 18 cores at 3.6GHz each (did hardcore testing on the slight overclock and it's rock solid).

- 48GB ram.

- Storage has 8GB of RAM cache and redundant 4TB for VMs plus 3.3TB for system / filesystems shared into the VMs.

- 320MB/s avg read, 240MB/sec avg write performance native to the file server.

- 120MB read and write performance per server limited only by the gigabit connection because ESXi doesn't support link aggregation.

The performance is amazing.  Even better is I hear that ESXi 5 will support link aggregation - meaning we'll be able to use the full native performance out of the file server box shared amongst the ESX servers.

The embedded gigabit connection on the motherboard is not supported by ESXi but everything else is.  Adding in $30 PCIe Intel EXPI9301CT network cards solves that problem.  Dedicated card for iSCSI, separate card for network traffic and management.

Best part - total cost was $2000.

<Soap Box disclaimer>: Using an officially supported setup would have run us in excess of $5k easy.  If we wanted the same filesystem read/write performance that price would have been even higher.  There's a reason why large data centers use basic off the shelf workstation components in massive redundant setups - the price simply isn't worth using server grade hardware if you're a little creative and take some precautions.

Now I'm not saying that this is for everyone.  If you have mission critical servers running then of course use officially supported server-grade redundant hardware.  But for everyone else the price point makes it very difficult to justify the additional cost.

Reply
0 Kudos