DirectPath I/O support for Haswell Lynx Point SATA AHCI controller

[Digian]

Hi,

Im building a new ESXi 5.5 system on Asus H87i-itx board with Intel i5 Haswell 4670T chip. Mainboard and CPU both support VT-d / DirectPath I/O so far as I can tell, but I have a problem.

In ESXi I can see most devices available for DirectPath I/O passthrough such as Video, Audio, NIC and USB controller, everything is available for DirectPath I/O passthrough except for Lynx Point SATA AHCI controller.

Can anybody help diagnose why the SATA controller is missing, and steps for more detailed debugging ?

Previous Cougar Point AHCI SATA controller is supported in ESXi so why not Lynx Point.

Also, Lynx Point SATA AHCI controller is listed on VMware HCL page here:

VMware Compatibility Guide: I/O Device Search   Intel "Lynx Point" Series Chipset 6-Port SATA AHCI  8086:8c02

Strange thing is my ESXi can see the Lynx Point SATA AHCI controller in Storage Adaptors correctly, showing 6 x VMHBA listed, so why does it not appear in passthrough, and not even greyed out ?

Is there any extra diagnostic I can do please tell me ?

Would like to avoid doing physical raw mapping (rdm) if possible, DirectPath I/O Passthrough would be much nicer method. 🙂

RDM mapping of local SATA storage for ESXi

[Dashers]

Has anybody tried this with ESXi 5.5 Update 2 yet?  This was released last week and now lists Lynx Point as a supported chipset.  The problem with the HCL is it isn't clear what level of support.

If anybody has a Lynx Point chipset and done the updated, please could you feedback for the rest of us?

[vinnie_ki]

The ahci reset failure for the guest os may be a particular bug for esxi 5.5 (update 1). I had the same thing with an Asmedia ASM 106x ahci controller which was directly passed trough on esxi 5.5 update 1, while on esxi 5.1 update 1 the controller worked very well. In esxi 5.5 update 2 it also works fine again. So you may be on to something. Can you or someone else reproduce those steps with esxi 5.5 update 2? In fact it may be possible that it will work. I am not an expert on esxi and it looks a bit complicated so I am not going to try it myself, but I am curious if it works.

As for the better Lynx Point support in esxi 5.5 update 2, I can tell you that only support for the raid modus of the Lynx Point sata controller is added. But it is still not possible to pass it trough in ahci mode or in raid mode.

[ScottMerrilees]

Interesting, but I've made my decision for the moment, and won't be revisiting it again for a couple of years.   Linux, KVM, and docker mean I don't need to worry about DirectPath, or VMware breaking it in the next update.

[sharix]

I'm happy to confirm that ESXi 5.5U2 doesn't prevent the Lynx Point AHCI Controller (as found on haswell based mainboars with the Intel C226 Chipset) to be used as PCI passthrough device anymore!

Just add it's chip ID to /etc/vmware/passthru.map, like ScottMerrilees mentioned in post #17, so a patch would look like

--- /etc/vmware/passthru.map

+++ /etc/vmware/passthru.map

@@ -22,4 +22,6 @@

1000  0058  d3d0     default

# NVIDIA

10de  ffff  bridge   false

+# INTEL Lynx Point AHCI

+8086  8c02  d3d0     false

Tested under FreeBSD 10.1 guest OS (amd64), with single vector MSI, working as expected (less than 100MHz for 2500irqs/s@ahci0):

Hopefully DirectPath I/O is on par with 5.1 again 🙂

[vinnie_ki]

Thanks to ScottMerrilees and Sharix, I now have my Lynx Point AHCI (Z87 chipset) running in passtrough mode on Esxi 5.5 upd 2

I can confirm that the provided solution works for me too. I tested it succesfully with ZFSGuru as a VM.

For the novices:

• Activate the ssh service in Vsphere Client using the instructions in this link: VMware KB:     Using ESXi Shell in ESXi 5.x
• Use Putty or any other terminal program to connect to you Esxi SSH server.
• Use vi, for example: vi /etc/vmware/passthru.map
• Add these 2 lines at the bottom of the file:

          # INTEL Lynx Point AHCI

          8086  8c02  d3d0     false

• If you don't now how to use vi, than google for a manual.
• Reboot the Esxi server.
• Add the Lynx Pont AHCI to the passtrough devices and reboot the Esxi server again.
• All done!

[muellererwin]

Thanks very much for your effort.

I followed your instructions and I could passthrough the Lynx Point AHCI.

But when I connect the PCI device to my virtual maschine I get the following error:

The PCI-Passthrough-Device-ID (0x-73fe) is invalid          (translated from German)

Any ideas????

[vinnie_ki]

@ Muellererwin:  Can you give as much information as possible in order for us to be able to help you? For example: esxi version, hardware, chipset, screenshots, etc.

[muellererwin]

Here we go:

Hardware:

Intel DH87RL, i7-4770, 16GB

256 SSD on ASMedia 1061 PCIe x1 SATA 2 ports (no passthrough, boot drive)

5 x ST2000DL0032D9VT166 (directly from my synology 1511+, partition types 0xfd Linux RAID) with the purpose to run them in Soft RAID under XPEnology with onboard Lynx Point SATA (passthrough)

Software

ESXi 5.5 U"3" Kernel 2143827

VM: XPEnology 5.0-4528

Modifications (as described above):

/etc/vmware/passthru.map

          # INTEL Lynx Point AHCI

          8086  8c02  d3d0     false

Happy Christmas Days to everyone.

P.S.:

I also tried to RDM passthrough the five disk drives instead of the Lynx Point controller and ended also with an error because of the partition type 0xfd (I guess)

see: RDM pyhsical passthrough of disk problem with partition type 0xfd (raid autodetect arrays)

!!!! SOLVED !!!!!

[MrWhippy]

Hi all, thanks to you I have this working also, though I have hit another snag and wondered if you could possibly suggest a solution...

I am running ESXi5.5u2 on an Asrock Z87E-ITX board which only has one PCI-E slot currently housing a graphics card. As such the Lynx Point controller is the only SATA controller available for passthrough, which I have successfully configured as I mentioned. I am booting ESXi from USB and seeing "The ESXi host does not have persistent storage" (which is as expected). So I am wondering for those of you who pass through their Lynx Point controller successfully, how are you also making use of a datastore in ESXi?

I am trying to build an ESXi all-in-one and would prefer not to have to rely on an external NFS datastore.

Also, just adding one small but important extra step to vinnie_ki's instructions:

For the novices:

• Activate the ssh service in Vsphere Client using the instructions in this link: VMware KB:     Using ESXi Shell in ESXi 5.x
• Use Putty or any other terminal program to connect to you Esxi SSH server.
• Use vi, for example: vi /etc/vmware/passthru.map
• Add these 2 lines at the bottom of the file:

          # INTEL Lynx Point AHCI          8086  8c02  d3d0     false

• If you don't now how to use vi, than google for a manual.
• run auto-backup.sh
• Reboot the Esxi server.
• Add the Lynx Pont AHCI to the passtrough devices and reboot the Esxi server again.
• All done!

Cheers!

Message was edited by: MrWhippy Fixed a typo.

[vinnie_ki]

Off topic:

@MrWhippy

I also have the Asrock Z87E-ITX board. So you will probably have two mini pci-e slots of which one is occupied by the Wifi card. You can use both mini pci-e slots for a sata card with two sata connectors. Use one with an Asmedia 106x chipset. That one is not supported out of the box by Esxi, but you can easily use Esxi-customizer to inject the VIB file into the installation image. There are many manuals for on the net.

I use this one of Delock in de mini pci-e slot on the back:

http://www.delock.de/produkte/G_95225/merkmale.html?setLanguage=en

And this one of DX in de mini pci-e slot on the front (because it's only a half size slot, you will have to do a little bit of DIY to hold it in place):

http://www.dx.com/p/mini-pci-express-sata-3-0-expansion-controller-card-black-green-337644

On topic:

I noticed when I don't have a harddisk connected to the Lynx Point AHCI, it doesn't show up in Esxi and you cannot pass it trough. When I connect a harddisk it shows up again. So make sure you connect a harddisk when you try to make it work on Esxi!

@muellererwin

I searched around a bit, but I am afraid I couldn't find a solution for you.

[MrWhippy]

@vinnie_ki

HUGE off topic thanks, that's exactly what I was after! I would DM you instead of posting here but it seems I first need 10 "points" before doing so. Unfortunately I am away from my box for 2 weeks so won't be able to make any hardware changes until then, but I am looking forward to it. Do you happen to have a build log/ steps of your box somewhere online I can check out in the meantime?

In the interest of keeping this post on topic (mostly), I found it was also possible to pass through the Lynx Point controller in RAID mode under ESXi5.5u2:

lspci -v

0000:00:1f.2 RAID bus controller Mass storage controller: Intel Corporation 82801HR/HO/HH (ICH8R/DO/DH) 6 port SATA AHCI Controller [vmhba0]

  Class 0104: 8086:2822

# INTEL Lynx Point RAID

8086  2822  d3d0     false

Not sure if that will be relevant or useful to someone, but you never know... I'd be interested to hear from anyone doing so.

[pierrepardon201]

Still works on ESXi 6.0.0 (Build 3029758) Update 1

[vinnie_ki]

This fix is still working under Esxi 6.0 U2. It would be nice if it works out-of-the-box. But unfortunally VMware hasn't implemented it.

The only problem I encounterd, was to get the ssh server to function correctly. Ik found the solution in this post.

If you don't want to use ssh for this fix, you can also use the consol directly on the Esxi server bij pressing ALT+F1.