New Server Project

[TopHatProductio]

Hello! It's been a while since I last posted here with my own topic. I now have a dedicated ESXi server in the works. The server project is meant to replace (and exceed) my previous workstation - a Dell Precision T7500. Here are the specs for the hardware:

HPE ProLiant DL580 G7

    OS   :: VMware ESXi 6.5u3 Enterprise Plus
    CPU  :: 4x Intel Xeon E7-8870's (10c/20t each; 40c/80t total)
    RAM  :: 256GB (64x4GB) PC3-10600R DDR3-1333 ECC
    PCIe :: 1x HP 512843-001/591196-001 System I/O board + 
                1x HP 588137-B21; 591205-001/591204-001 PCIe Riser board
    GPU  :: 1x nVIDIA GeForce GTX Titan Xp +
                1x AMD FirePro S9300 x2 (2x "AMD Radeon Fury X's")
    SFX  :: 1x Creative Sound Blaster Audigy Rx
    NIC  :: 1x HPE NC524SFP (489892-B21) +
                2x Silicom PE310G4SPI9L-XR-CX3's
    STR  :: 1x HP Smart Array P410i Controller (integrated) +
                1x HGST HUSMM8040ASS200 MLC 400GB SSD (ESXi, vCenter Appliance, ISOs) + 
                4x HP 507127-B21 300GB HDDs (ESXi guest datastores) +
                1x Western Digital WD Blue 3D NAND 500GB SSD + 
                1x Intel 320 Series SSDSA2CW600G3 600GB SSD +
                1x Seagate Video ST500VT003 500GB HDD
    STR  :: 1x LSI SAS 9201-16e HBA SAS card +
                1x Mini-SAS SFF-8088 cable + 
                        1x Dell EMC KTN-STL3 (15x 3.5in HDD enclosure) + 
                                4x HITACHI Ultrastar HUH728080AL4205 8TB HDDs +
                                4x IBM Storewise XIV v7000 98Y3241 4TB HDDs
    I/O  :: 1x Inateck KU8212 (USB 3.2) +
                1x Logitech K845 (Cherry MX Blue) +
                1x Dell MS819 Wired Mouse
            1x Sonnet Allegro USB3-PRO-4P10-E (USB 3.X) +
                1x LG WH16NS40 BD-RE ODD
    PRP  :: 1x Samsung ViewFinity S70A UHD 32" (S32A700)
            1x Sony Optiarc BluRay drive
    PSU  :: 4x HP 1200W PSUs (441830-001/438203-001)

The details for the ProLiant DL380 Gen9 will appear here once data migration is complete. VMware Horizon (VDI) will have to wait for a future phase (if implemented at all). The current state of self-hosted VDI is Windows-centric, with second-hand support for Linux and no proper support for macOS.

The planned software/VM configurations have been moved back to the LTT post, and will be changing often for the foreseeable future.

Product links and details can be found here.

ESXi itself is usually run from a USB thumb drive, but I have a drive dedicated to it. No harm done. A small amount of thin provisioning/overbooking (RAM only) won’t hurt. macOS and Linux would have gotten a Radeon/FirePro (ie., Rx Vega 64), for best compatibility and stability, but market forces originally prevented this. Windows 10 gets the Audigy Rx and a Titan Xp. The macOS and Linux VMs get whatever audio the Titan Z FirePro S9300 x2 can provide. The whole purpose of Nextcloud is to phase out the use of Google Drive/Photos, iCloud, Box.com, and other externally-hosted cloud services (Mega can stay though).

There are three other mirrors for this project, in case you're interested in following individual conversations from the other sites (in addition to this thread).

• Level1Techs
• Linus Tech Tips
• TechPowerUp

P.S. Out of all the sites that I've ever used, this forum has one of the best WYSIWYG editors I've used in a while

Kudos to the devs!

[TopHatProductio]

Just purchased 7x HGST HUH728080AL4200's - 4 more to go. They're ~80-100 USD a pop. Also waiting to see how and when I'll get the other Dell EMC KTN-STL3, which is 150-300 USD. My next target(s) will be the Xeon E7-8890v4's, which are going for ~250 USD/unit. Once I pull that off, it'll be time to shop for a ProLiant DL580 Gen9 that comes with a Smart Array P830i Controller. Drive trays/caddies will be a challenge, since they tend to be stupidly expensive on the HPE side of things. RAM can always be purchased later, once prices for DDR4 ECC drop (as DDR5 takes over). On a side note, I've also managed to get OnlyOffice EE running with an external MariaDB instance. Working on doing the same for NGINX Proxy Manager, so I can move all database instances to a separate Docker host when the time comes! If anyone thinks I should leave all Docker containers on the same host, let me know. One less VM I suppose...

EDIT: A new topic appears:

• https://community.searchlab.eu/t/pertaining-to-how-yacy-crawls-websites/1090

[TopHatProductio]

I've purchased the rest of the HGST HUH728080AL4200's. Did some more research, and it looks like all standard models of the DL580 Gen9 came with the Smart Array P830i Controller. If such is the case, then I only need to concern myself with getting more cache modules - which should be cheap. I already have a bunch of potentially compatible drive caddies from my run-in/mishap with a certain ProLiant tower server that I won't mention here. I've already made an offer for 8 memory cartridges (HP 802277-001). Only need to focus on CPUs from here on. Then, to grab the server itself...

[TopHatProductio]

I've grabbed 4 memory cartridges. The Xeons are the only things left to grab before I go for the server itself >:D

[TopHatProductio]

I managed to get an SPI board for cheap. I guess I'll keep it as a spare, in case anything happens to the one that comes with the DL580 Gen9. The Xeons are next. I'm okay with delaying the purchase of the server itself if I have all of the other parts I need. I can go for the cheapest one available once DDR5 becomes more common, which will make it easier to get one with more ECC RAM at a lower price. HDD photos incoming maybe, if I can get up the energy to take the photos.

[TopHatProductio]

Got a Netapp DS4243, and am now just looking to get the Xeons next. I'll hold off on getting the server until 2023 if necessary, to let prices drop.

[TopHatProductio]

Grabbed 4 of the 656364-B21's, and am now left with only Xeons to grab (again). Also am doing some a bit of asking around before I bring the server in-house:

• https://www.reddit.com/r/HomeDataCenter/comments/wegsxj/network_monitoring_suggestions/
• https://www.reddit.com/r/homelab/comments/weh2xi/documentation_suggestions/

As mentioned before, I can wait until next year to get the server itself. As long as I have all of the other parts, I can wait for prices to drop on the Gen9 chassis.

[TopHatProductio]

I grabbed the Xeons and the RAM today. Now just waiting to see if I can get a decent offer on the server/chassis itself. That should help to keep the price on it low.

[TopHatProductio]

Just applied this on my Windows Server VM:

• https://helpdeskgeek.com/windows-10/how-to-disable-windows-10-telemetry/

Now when I switch to my domain account in 2022/2023, I'll have one less thing to worry about...

[TopHatProductio]

Decisions, decisions...

• https://linustechtips.com/status/323896/

[TopHatProductio]

I've got everything but the server and its rail kit at this point. Bumped RAM up to 384GB. I'll wait until 2023 if necessary, to get a good price on the remaining pieces. More pre-planning underway - stay tuned...

[D_G_Tal]

Onto the next issue:

• https://communities.vmware.com/message/2980646#2980646

Also have to look into this at some point (even though the server will be mostly hiding behind a VPN):

• https://kb.vmware.com/s/article/55636

[TopHatProductio]

Not sure if I understood what you were talking about. Are you referring to an issue with the forum? Test formatting? Just curious.

[BigMike23]

Hardware pricing could start coming down once the chip manufacturing catches up also PC makers had bad earnings this quarter.

[TopHatProductio]

Currently looking into switching from Remotix (bought out by Acronis) to RustDesk. RustDesk isn't as fast or performant as the former, but it is free and open source. Also won't have to worry about changes in licensing. Only issue is, it's not working on my current tablet. The app glitches out on it, so I have to go back to managing everything on just PC. Also considering using MikroTik's Dude instead of LibreNMS.

You might be right on this one - Getting the DL580 Gen9 looks like it will most likely have to wait until late 2022/early 2023 at the earliest. Most sellers want at least 700-800 USD for a barebones config, which is kinda tough to pull off at the moment.

Also wishing I could have gotten the Radeon Pro v340, but those are going for 1.5k USD currently. My current objective is to work on prepping the containers and VMs for when it's finally time to pull the trigger on getting the Gen9 in-house. Moving Nextcloud's db to Docker is going to be nerve-wracking for me, since if anything goes wrong, I may have to rebuild that instance from scratch. Not a fun idea. Open Street Maps + OSMR (Docker stack) and FreePBX (VM) tasks are next...

[TopHatProductio]

Some troubleshooting is underway:

https://www.reddit.com/r/docker/comments/wzn65q/troubleshooting_osrm/

[TopHatProductio]

The memory requirements for the Linux VM appear to have more-than-doubled. I may have to give it upwards of 80GB of RAM to get OSRM past the extraction stage...

[TopHatProductio]

Pertaining to the next phase...

The Linux VM, without YaCy, will need at least 80-96GB of RAM. I think the dedicated DB host will have to go, seeing that 85% of that RAM will be OSRM alone. YaCy will need some for web crawls and indexing as well. Those two are very heavy. Also very close to just yolo'ing the Bliss OS VM and going for it either way. At least I may end up with some resources to spare as a result.

[TopHatProductio]

Just finished troubleshooting an unexpected Artix issue. Still need to config FreePBX…

[TopHatProductio]

Onto the next task(s):

• https://help.nextcloud.com/t/configuring-custom-tile-server-in-maps/145394
• https://www.reddit.com/r/HomeDataCenter/comments/xb7g3n/looking_for_a_quiet_rack_solution/

[TopHatProductio]

August has been a long month, and it's been a long time coming.

Back in early-August, I ended up delaying the purchase of the DL580 Gen9. This threw a wrench into my previous plans since I had the Gen9 planned with over 300GB of RAM. I was depending on the larger pool of RAM being available for things like OSRM, YaCy, Android VM, etc. The first thing I did was update Windows Server 2016 to 2019. I finally got dark theme, but my previous license had to be replaced. Still a little peeved about that unexpected cost. Kicked out Technitium DNS since it was redundant to AD DNS at this point, and added OCCWeb to Nextcloud for easier updates in the future (running commands after initial update).

At about half-way through August, I decided to put OSRM on the table. After a bunch of reading, interpreting, getting help, and guessing, I finished figuring out that thing late last week. Didn't even find this page until I was near the end. I'd say kill the page being used on Docker Hub for OSRM and replace it with that instead. Would have saved me a ton of time last month! By the way, I ran into this little issue in the middle of troubleshooting OSRM. No biggie, finally finished getting OSRM to work at the end of August.

I was supposed to get started with FreePBX back in August, so the I could work primarily on YaCy this month and into October. That's not going to happen. At least I managed to shove RustDesk in there somehow. I also had to re-build xBrowserSync, since I accidentally broke that container while trying to change its IP address. Also had time to setup GPO for local admin account(s). So it wasn't a complete bust in falling behind.

Still need to look into using custom tile servers with Nextcloud Maps (unknown) and PhoneTrack (yes), finish setting up FreePBX, switch Google Voice to VoIP.ms (port my number over), start working on YaCy Grid, Convert Windows Server and Windows 10 VMs to GPT/UEFI, Update Nextcloud, and buy a bigger SSD for my current laptop. Wait - almost forgot that I need more EaseUS backup licenses for my setup. This all needs to happen before mid-2023, preferably.

I'm also considering doing a clean install of Artix OpenRC, due to a small issue I've been having since 2021. On the bright side, I kinda want a new desk and rack!

Something tells me that I'm more than a month behind XD

btdubs, didn't vSphere 8 just get announced? I'm still waiting to move to vSphere 6.7 on my next server 😐