Mac Pro 5,1 ESXi - PCIe Card & SSD

[JCostelloMS]

Hello,

We are attempting to upgrade a few of our Mac Pro 2012 servers with newer hardware.

Currently our Mac Pro 2012 servers run ESXi 5.5 and we were interested in knowing if we could add a PCI express card that would support one or two SSDs in the Mac Pro and vmWare would recognize and be able to boot from the PCIe card SSD drive(s).  I have looked all over the web for an answer to this, but have not turned up any solid information that this would work.

We are specifically looking at these three cards:

Sonnet Technologies Tempo

https://www.amazon.com/Sonnet-Technologies-Tempo-Drives-TSATA6-SSD-E2/dp/B0096P62G6

Apricorn Velocity Duo

Amazon.com: Apricorn Velocity Duo SSD Upgrade Kit and Disc Array for PC and Mac (VEL-DUO): Computers...

OWC Accelsior S

Amazon.com: OWC Accelsior S: PCIe to 2.5" 6Gb/s SATA SSD Host Adapter: Computers & Accessories

All three are apparently recognized by the Mac Pro hardware out of the box, but what I cannot determine is if they would be recognizable by vmWare and bootable as well.

Additionally, if any of these cards can be used, would there be a huge performance drop if the ESXi OS was installed on an SSD connected to the motherboard SATA on Mac Pro and data (vmdks) were stored on the PCIe SSDs? Would the VMDKs perform better or worse if the boot drive was attached to the SATA II in the Mac Pro?

Thanks,

[Bottacco]

Hi, I don't know about the PCIe mounted SSDs because I have found no info either. But about the performance hit of running the ESXi OS on one of the SATA-2 ports of the motherboard you can you can do it safely. I have tried running it from a USB-3 pendrive connected to one of the included USB-2 ports compared to running it from an SSD on a SATA-2 of the motherboard and the difference is negligible, so don't worry because there won't be much of a performance drop except in edge cases with many VMs running full power simultaneously.

We just use old Mac Pros 5,1 to set up labs for old versions of OS X/macOS and it works beautifully, but we would also benefit from PCIe SATA-3 SSDs. If get a hold on one of those we may try it, but there is no mention of driver support anywhere. We also looked into installing some type of PCIe SATA-3 card with eSATA ports on it. There is some unofficial support and we may look into it because it may be a cheap solution for an external RAID5/6 datastore.

[imrazor]

If you search the Amazon reviews, you'll see that the OWC has one review that indicates that someone successfully used the Accelsior with ESXi, though he doesn't mention which version of ESXi. However, he used the card in Dell server rather than a Mac Pro.

I've been looking at the Accelsior myself for a Dell tower, though it seems rather expensive for what it is...

EDIT: Random thought - if you pass through the SATA-III controller on the card to a VM, it may not matter whether or not ESXi has a driver for the controller...

[wila]

Hi,

Why not get a PCIe NVMe card? Those are cheaper than those "strap a sata SSD on a PCIe card" solutions and work too, at least with ESXi 6.5.

--

Wil

[Bottacco]

imrazor, pass through option is not viable if what you need is a ESXi datastore where all the VMs are stored in. If it were for having data accesible in a specific VM it would be fine.

[Bottacco]

wila, for our needs it could work, although prices are still expensive if you need 1TB or more. But worth looking into it. I will try one next month and let's see what comes out of it.

Cheers and thanks for the suggestion.

[imrazor]

wila​ I'm not an expert on disk controllers, but doesn't NVMe require support hardcoded in BIOS/EFI? I think the old Mac Pros we're discussing here lack that support.

EDIT: After a bit more research, it seems that they will work as storage devices, but you will not be able to boot from them. See here: macOS High Sierra: Mac Pro Support and NVMe SSDs - Latest Apple Mac Pro News & Tips While the article is about High Sierra rather than ESXi, I think it likely that ESXi won't boot from an NVMe drive in a Mac Pro. But that may not matter if you just boot off a traditional SATA drive and put your VMs on the NVMe drive.

[wila]

Hi,

doesn't NVMe require support hardcoded in BIOS/EFI

I don't think so. For booting your mac Pro from the NVMe then yes it might be, but not for simply using it. It's just another PCIe device and as long as that is recognized it works.

FWIW I got myself some cheap (EUR 20,-) PCIe controllers for my lab and they worked in pretty much anything I tested.

That includes a 10 year old desktop running Windows 8, a linux box running a 4.x kernel as well as on vSphere 6.5 on a Dell R320

Here's the controller I used.

Interface-converter 28554C212

and some NVMe PCIe SSDs, but those can be anything you like.

For production usage I suggest a better controller, but this worked for me.

To get back to the answer for TS. It doesn't really matter what you use for booting vSphere. A usb stick is fine. It is only read from during boot, so putting a faster disk for booting only helps during the boot process. FWIW I boot all of my vSphere hosts from internal USB sticks. If you want your VMs to have a faster disk access then you can either use fast (SSD) host cache or put your VMs on faster storage.

--

Wil

[imrazor]

wila So that must be why my Dell T5500 has an internal Type A USB port - not a header, an actual internal USB port inside the chassis. Thing is though, I worry about the long-term reliability of a random USB stick. Ever have any problems with data corruption on your boot media?

[wila]

Hi,

No, not with my ESXi hosts. I think the reason for that is that it really just is read on boot, it doesn't write logs or anything.

Been running from usb sticks for 5 years now.

Now on a few freeNAS boxes that I have and boot from mirrored USB I have had to replace the usb sticks several times already.

Also note that even if the USB stick dies on your vSphere host, you can basically just re-install in 15 minutes, leaving your VMs alone.

The only issue is the vSphere host configuration that you have to do again.

An anecdote is that I once had an ESX boot device die on me (that was a RAID disk, not a usb stick) and I could still use the VMs, shut them down etc.. it was only when I ran a "df -h" command from the command line that the console hung. Not seen that on ESXi yet, so cannot comment on it.

However my suspicion is that you can pull the usb stick and still run your VMs.

Of course I would expect issues over time. Would be nice to run as a test in a nested environment, but not having the time for that now.

--

Wil