Cheap ESXi host for home lab

[imrazor]

So I got my evaluation license for ESXi 6.0 a while back and tried installing it on the various systems I had available (no real servers, alas.) Despite having some hardware that was top tier at one time, the only thing that ESXi would install cleanly on was an ancient consumer-grade Dell Inspiron 570. No problems with network or SATA drivers or VT-d/IOMMU, just installed - no muss, no fuss. However, this thing is pretty limited. It has a triple core processor, and maxes out at 16GB of RAM. Now I was looking online and saw some old Dell Precision workstations selling for peanuts. The models I was looking at were the T3500 and T5500, but I'm not sure if ESXi would install on either of those boxes. (I almost got one with a Xeon W3520, but researched that Xeon and found that it does not support VT-d.)

So I'm on a shoestring budget and can't afford a real server, but I might be able to swing one of these old workstations. A second option that occurred to me would be to upgrade the triple core processor in the Dell Inspiron 570 to an Athlon/Phenom quad CPU. However, the Xeon would have 4 actual cores plus 4 logical cores thanks to hyperthreading. And if I really wanted to, I could upgrade to a six-core/12 thread Xeon like the L5640.

Comments welcome...

[imrazor]

Nobody answered this, but I just wanted to post this in case anyone else ever contemplated using a Dell T5500 as a VM server. ESXi 6.0 installed flawlessly. If you do buy a Dell Tx500 workstation, watch out for the W-series Xeon. My research indicates that they do not support VT-d, which is an issue if you're interested in PCI/PCIe passthrough.

On that note, I wanted to mention that I got GPU passthrough working with this workstation with the included Dell Quadro FX 3800. That's an old card, but it allows me to use DirectX 10 games and such in a VM. There is a performance penalty in the neighborhood of 10% - 15% (at least according to 3DMark Vantage), but it still allows me to use 3D apps and games in a VM. I did have to install an even older card (Nvidia 7300GS) for the ESXi console to grab and leave the Quadro available for passthru. I may also try experimenting with Steam streaming from the VM to another machine on the network, or perhaps to another VM. The main limitation I'm running into is RAM, and to a lesser extent, available cores. This machine came with a 4C/8T Xeon and only 12GB of RAM. I'm probably going to need to upgrade to at least 18GB of RAM and a 6-core Xeon, which are fortunately pretty cheap under 3GHz.

I'm a little worried about using a newer video card thanks to this old Dell being strictly BIOS. A GPU with a UEFI-only BIOS may not work properly in this machine. In spite of that, this has been a very interesting and relatively cheap project that will let me learn more about virtualization, VMWare and Windows Server.

[jpsider]

Here is my Blog, where I walk through what a lot of folks are doing int he community with Home Labs (Intel NUC)

https://invoke-automation.blog/

Works flawlessly out of the box.

[imrazor]

The NUCs are interesting devices, but didn't suit my desire to research PCI/PCIe passthru and was outside of my budget. A quad-core NUC according to my very casual research seems to go for $450 - $650 dollars. And if I recall correctly that price does not include storage or RAM. On the other hand, I picked up this Dell workstation for $125 (including shipping) and it came with almost everything I needed. All I had to do was add a couple of SATA hard drives. Admittedly, if I want to do more than tinker I'll need more RAM and maybe a six-core CPU, but they're so old now that they can be had very cheaply. Since it's a home lab, performance is not a top priority, though installing a guest OS and updating it can be time consuming.

[melloa]

vSphere ran fine on my Dell 2950 III. Here I see people selling for $50-100. Noisy and might not be compatible with future releases? Yes, but you can't get cheaper than that for a test/home/lab. In fact I have moved mine around to identify what are my real needs: From the 2950 to a Intel S5000SP with 2xx5450, to a single i3. The i3 is running at 70% CPU utilization, so will be moved to an i5. All with used hardware laying around. For the future: SM X11SS plus e3-1240v5 would do great and only costs $360. One thing I've learned is that old servers are noise, expensive to run (power bill), not always compatible, and close to be incompatible. Good luck!

[imrazor]

I thought about getting an old server, but I didn't feel like dealing with the noise. A workstation usually doesn't support as much RAM or as many hard drivesas a server, but runs more quietly and can pull double duty as a desktop if necessary. Servers also don't usually have PCIe x16 slots for GPUs, which was one thing I wanted to play with. If I had a spare room or closet to throw a machine in I might have considered a server, but I'm short on space.

[melloa]

True, RAM is a problem with you using desktops for your tests, but when you want to break the 32GiB barrier you'd laying right at a server motherboard, that most likely will require ECC. Any decent MB+CPU to start will give you a 5 bills cost, but each 16GiB ECC will cost you $100-200 or more.

As I'm an old timer from the time that each line of code was one card, I'm modifying my servers to be quiet enough to be on my desk and will start my road on VMWare.

Regarding HDs, why would you need more than a couple, maybe three. You can always use a NAS for your storage need to, if you really want lots of disks on your ESXi box, a controller will give you as many as you can fit on your case/slots. The box I'm runig my i3 for test has 10 HDDs and 1 SDD, but only the SDD is used for boot and 2 HDDs for the VMs, just because I run a security system saving to a huge VHD to won't impact my network. Everything else is via NFS to the NAS.

Just some food for tough from a non-IT/home user-tester.

[melloa]

Will be paying a visit to learn. Tested vbox and kvm, now on vSphere. Looking good so far

[imrazor]

RAM is one reason I passed on the Precision T3500, and instead opted for the T5500. The T3500 only goes up to 24GB of non-ECC RAM, but the T5500 can go up to 48GB of ECC DDR3 RAM. If I opt to add a 2nd CPU, I can buy a CPU riser card with 3 extra DIMM slots for $75. That will bring my maximum RAM to 72GB of RAM, which should be more than adequate for my purposes. That will cause issues with my eval VMWare license, though (max 1 CPU socket.)

If you look around a bit on eBay, you can find ECC DDR3 1066MHz RAM going for $1/GB. I know it's used RAM, but there's no way I'm paying full retail for new ECC RAM. Too expensive.

I need hard drive bays because I have several small hard drives (~500GB), and a single one of those probably isn't enough space for all the test VMs I'm going to create. I don't think a NAS would be a good idea because my home network is an 802.11n WiFi network. The wifi router has 4x1Gbps ports on it, but it's consumer grade so I don't know how well it would handle huge file transfers or booting an OS. A neat bonus of the T5500 is an eSATA port for external expansion, plus a Dell SAS 6/iR RAID card. However, SAS is probably too expensive for my puny budget and the RAID card doesn't want to talk to any of my SATA drives.

[melloa]

It seems you got your thoughts straight on what you want/need. One last thought would be, if you want to:

- Consider a FOSS router (i.e. pfSense - runs on cheap hardware and works great. Mine runs on a Pentium)

- Consider re-flashing old Perc cards to IT mode (Perc 5/i support up to 2TiB, enough for your 5ooGiB HDDs, support up to 8 HDDs, and are cheap in eBay. I used them on 2950s with ZFS and worked great, but now on M1015s to break the 2TiB barrier).

Good luck!

[Tad_P_N]

I use both T3500 and T7500 as ESXi hosts.

So far they've run ESXi <5, 6.0, 6.5, and 6.7.

I am now getting notifications during install that the processor may not be supported in future versions, but for now it still works great.

ESXi version: 6.7.0

EESXi build number: 8169922

AAlso, I am using the built in Sata controller for the T3500 and both the SATA and SAS controller on the T7500 (both running sata drives)  No issues but I have not tried RDM yet.

TThe precisions are great systems, have excellent build quality, and are super cheap on ebay or dellrefurbished.com...  I just picked up a perc310 for <30 on ebay to see if that will speed up drive access too for some new SSDs.  Amazing how much enterprise power you can buy used

[edward_walton]

If I remember correct, only one or two Dell Precision are officially supported by VMware for the newest versions of ESXi.

The R7910 & R7920 rack mount Precisions?

But if you find a Precision model that you like and want to know if it'll work with ESXi, then my suggestion would be to skip the VMware hardware compatibility guide and head straight to Dell:

www.support.dell.com

Or the direct Precision link (until it gets updated to another URL😞
Support | Dell US

Locate the Precision model you are interested in.

Then click "Drivers & Downloads".

Then click the drop-down menu for Operating Systems.

If you see the version of ESXi you want to run, then you'll see that Dell has certified that Precision model for VMware.

Otherwise, the biggest issues you'll probably run into will be Datastore related.

Will ESXi recognize your local storage controller?

And if not, then will a PERC RAID/HBA controller work in that computer?

Not all computers have a x8 PCIe slot for a PERC to drop into.

A x4 slot will "work", but only if the end of the slot is open so the x8 card can drop into place.

Unfortunately, the Precision T5500 is not certified to run ESXi.

[edward_walton]

Btw, I am on the lookout for a used Dell T420 or T620 tower server.

Both are certified for ESXi 6.5.

Both are "quiet" vs. the "airplane whoosh" you get with rack mount server PSU's.

Both have great RAM and dual Xeon E5 Processor options as well as USB 3.0 for newer PCIe card support.

And even support a few GPU options such as AMD FirePro.

And can be rack-mounted if you somehow locate the rack-conversion kits Dell used to sell.

And are available with 3.5" HDD chassis configurations that work with SATA or SAS.

This will be my personal/media server, so inexpensive 3.5" SATA HDD's are all I need.