Home lab - Will this work?

[dafunkphenom]

I am not familiar with ESXi or it's capabilities as I have never experimented with a bare metal hypervisor. I would like to know if what I have in mind will work before investing my time and money into this project

Goals - To build a ZFS bases NAS / syslog server under one box for my home lab

Hardware

HP Microserver N40L - Dual core 1.3GHz AMD Athlon 2 - NC107i PCI-e Gigabit NIC. I will be adding an extra gigabit NIC as well.

I will be putting the max 8GB DDR3 ECC RAM

For storage I will need to do a minor mod so I can get 5 x HD's. The machine comes with 4 drive bays so I will be putting in 4 x 2TB SATA drives to be used as RAID-Z storage. I want to add a drive caddy to the 5.25 inch optical drive to install an extra SSD drive used for the operating systems of both VM's. Here is a link for somebody that did the mod - http://paulroberts69.wordpress.com/2012/01/22/installing-a-hdd-in-the-optical-drive-bay-on-an-hp-mic...

Configuration

I ultimately want to have 2 VM's running on this machine.

VM1 - FreeNas 8

VM2 - FreeBSD or Linux used as primarily a syslog / web server

Questions and concerns. Will this be possible?

I want to boot ESXi off the motherboards internal USB port.

Will I be able to install the OS from both VM's if I partition the added SSD drive that I have installed in the optical drive bay?

Will the FreeNas VM be able to setup the RAID-Z amongst the 4 physical SATA drives?

Each VM will be assigned it's own physical NIC as they will run on 2 separate network segments behind my pfSense firewall. I don't imaging this will be much of an issue if I used bridged networking.

If this is not possible I will probably install FreeBSD on it's own and try to get my configuration above to work. I am not familiar with FreeBSD and would like the nice GUI interface of the FreeNAS to manage my storage and Samba shares, hence this is why I would like to virtualize.

Any advice or tips is greatly appreciated.

[Jackobli]

dafunkphenom wrote:

Goals - To build a ZFS bases NAS / syslog server under one box for my home lab

Configuration

I ultimately want to have 2 VM's running on this machine.

VM1 - FreeNas 8

VM2 - FreeBSD or Linux used as primarily a syslog / web server

Questions and concerns. Will this be possible?

Shure this will be possible.

I want to boot ESXi off the motherboards internal USB port.

No problem. Just put your USB stick into that port and install vSphere on it.

Will I be able to install the OS from both VM's if I partition the added SSD drive that I have installed in the optical drive bay?

You might want to create one datastore on that SSD and use this datastore to install your VM on it. It seems a little waste of money to use a SSD for that. Do you expect a decent load on that VM or is it just because you want to avoid to install another rotating spindle in that DVD bay?

Will the FreeNas VM be able to setup the RAID-Z amongst the 4 physical SATA drives?

You would typically use these drives as raw device mapping (rdm) and pass them directly to your guest OS.

Each VM will be assigned it's own physical NIC as they will run on 2 separate network segments behind my pfSense firewall. I don't imaging this will be much of an issue if I used bridged networking.

You might create three vSwitches and use a pNIC as uplink for each of them. You also might use VLAN and avoid the third pNIC. It really depends on the bandwith and the lab situation.

If this is not possible I will probably install FreeBSD on it's own and try to get my configuration above to work. I am not familiar with FreeBSD and would like the nice GUI interface of the FreeNAS to manage my storage and Samba shares, hence this is why I would like to virtualize.

FreeNAS is kind of cool. My tests some times ago where ok. But the performance (especially for samba) was not really dazzling. That may be better now.

In this setup, you are also able to carve iSCSI devices or NFS shares out of FreeNAS and re-use them in vSphere as datastore for other VM.

[dafunkphenom]

Will I be able to install the OS from  both VM's if I partition the added SSD drive that I have installed in  the optical drive bay?

You might want to create one  datastore on that SSD and use this datastore to install your VM on it.  It seems a little waste of money to use a SSD for that. Do you expect a  decent load on that VM or is it just because you want to avoid to  install another rotating spindle in that DVD bay?

It is partly because I don't want another rotating spindle. I don't want to generate much more heat, power and noise than what already will be used when I already have the max amount of memory and all drive bays loaded with 3.5" SATA.

The other reason is that I will be running 2 VM's not just FreeNas. The other will be a complete OS either FreeBSD or Linux, primarily used as a syslog server to capture activity from my pfSense firewall. I may setup a small web server as well. I was originally going to just install Ubuntu on the SSD and configure mdadm raid and use VirtualBox for a VM for my syslog server but the more I started reading and coming across folks that have whiteboxes with ESXi the more I became interested in using a bare-metal hypervisor. I only have experience working with VMWARE Workstation and VirtualBox but would like a bit more experience working with quasi exterprise grade stuff since I am an I.T guy working in the industry.

Thanks for your suggestions. I must admit I am new to ISCSI and SAN. I mainly want to run a NAS as a Samba network share / container for my media that will be read by my WD-TV Live. My wife will also connect over WiFi to backup her files I've read about all the cool things that can be done with ISCSI such as remote boot etc, By the end of my project I should have everything running at Gigabit speeds, (I have to buy a new WiFi/Ethernet router, my only bottleneck). I've read that this is the minimum speed required before it's worth while using ISCSI targets.

[Jackobli]

Ok, seems reasonable. As the HP microserver does not contain a battery backed RAID controller the SSD will allow you to get decent speed.

vSphere itself does no (read or write) caching (unlike Windows, Linux, BSD *nx).

[dafunkphenom]

What size of USB flash drive should I use for the vSphere install?

[Jackobli]

One or two GByte should be enough.

You will set swap and logs to the SSD after installation, so the above size is ok

[dafunkphenom]

Are there any issues with running 2 VM's/Datastore on the SSD as well as being used for the SWAP and logs from vSphere?

[Jackobli]

If you got enough physical RAM there should be no swapping.

The logs will be written not immediatly but every some ten minutes or so.

So you should do quite fine, if there is to much load on the SSD you will perhaps need to buy a supported RAID-Controller.

I got no experience with SSD until now, sorry.

[dafunkphenom]

I think I will have to sacrifice the RAM. It looks like buying 2 x 4GB sticks of ECC RAM would cost me about $450. The box already comes with a 2GB module so if I buy another 2GB module straight from HP I can get it for $70. I doubt I need 8GB for my config anyway.

This is not working out too bad.

$600 taxes inc for the Microserver.

$400 for 2 x 3.5" 2TB SATA + 1 x 120GB SSD (I already have 2 x 2TB drives kicking around and I have a PCI-e Gigabit NIC from an old box)

$70 for 2GB ECC memory module

So probably all said and done between $1000 - $1100 taxes in. I'm from good old Ontario Canada.

That's within budget. Thanks again for your help.

[Jackobli · Accepted solution]

dafunkphenom wrote:

I think I will have to sacrifice the RAM. It looks like buying 2 x 4GB sticks of ECC RAM would cost me about $450. The box already comes with a 2GB module so if I buy another 2GB module straight from HP I can get it for $70. I doubt I need 8GB for my config anyway.

Try to look for Kingston or other compatible memory instead of HP original.

I was looking here (Europe, Switzerland) locally and found them for about 40$. Kingston is listing this memory as KTH-PL313E/4G

$600 taxes inc for the Microserver.

$400 for 2 x 3.5" 2TB SATA + 1 x 120GB SSD (I already have 2 x 2TB drives kicking around and I have a PCI-e Gigabit NIC from an old box)

$70 for 2GB ECC memory module

So probably all said and done between $1000 - $1100 taxes in. I'm from good old Ontario Canada.

Ah, got some far relatives in and around London, ON.

Greetings from nightly Bern, Switzerland

[dafunkphenom]

Awesome, they are about $50 here. So you own one of these Microservers too?

I have a buddy from Otelfingen, Switzerland.

It pays to hit the message boards. This one from VMWARE has always been reliable.

[Jackobli]

dafunkphenom wrote:

Awesome, they are about $50 here. So you own one of these Microservers too?

No, not really, but I read a lot about them. There is a german Blog I'm following. That guy is a storage expert and wrote some articles about these HP servers. I got a self built whitebox (Asus Mainboard, 2 x Opteron with 4 cores), HP RAID Controller with SAS Disk and BBU.

I have a buddy from Otelfingen, Switzerland.

Funny, ok, Switzerland is small, but I've grown some 15 Kilometer away from Otelfingen.