VMware Cloud Community
gsbjoe
Contributor
Contributor

VM guest will not boot with pass through device enabled after upgrading from 5.1 to 5.5

I am testing out a server configuration with a new HP ML350p.  Installed in this server, I have an HP H222 SAS controller with an LTO-4 tape drive attached to the SAS controller.

I installed the HP custom ESXi 5.1 hypervisor and configured passthrough and was able to install the HP H222 driver in my Windows 2008 R2 guest and then was able to configure the tape drive.

A week later, HP posted the HP custom ESXi 5.5 iso.  I downloaded and install v5.5.  When I try to power on the VM that has this passthrough device, I get the following message:

Failed to start the virtual machine.

The systemId does not match the current system or the deviceId, and the vendorId does not match the device currently at 13:0.0

I then tried upgrading the hardware version of this particular VM.  Still, no go.

I was able to reinstall v5.1 and restore my configuration.

For grins, I just installed a fresh copy of v5.5 (not upgraded) and configured pass through.  Same thing.  No boot on this particular VM.  If I remove the PCI device, I can boot the VM, but then I lack my hardware.

10 Replies
Frank_Dekker
Contributor
Contributor

Hi gsbjoe,

Did you solve your problem ?

Have the same situation, HP ML350P G8 with ESXi 5.5 (hp Version)

HP 222 controller in passthru mode.

Added the pci device to VM, power up, Failed to register the device pciPassthru0 for 7:0.0 due to unavailable hardware or software support.

We used the H222 with esxi 5.0 / 5.1 without any problems ......

Kind regards,

Frank

Reply
0 Kudos
gsbjoe
Contributor
Contributor

Sadly, no.  I even opened a ticket, which was not very helpful.  This is about the only reference they pointed me to:  VMware Compatibility Guide: I/O Device Search

Back in October, this guide said the firmware had to be at 15.10.01.00 for compatibilty with ESXi v5.5, which my card was at 15.10.01.00.  I see the compatibility guide now shows 15.10.02.00.  What version is your firmware?

My box is now in production in a remote office, so I can't simply upgrade the firmware and swap hypervisors easily.  I'd recommend upgrading the firmware, but I don't know where you are supposed to obtain it.  It isn't available on HP's web site!

http://h20566.www2.hp.com/portal/site/hpsc/template.PAGE/public/psi/swdDetails/?sp4ts.oid=5177962&sp...

H222.jpg

Reply
0 Kudos
Frank_Dekker
Contributor
Contributor

Server is in production now (to early), tried last night with ILO (mounting the HP SPP iso) and also with the new Intelligent Provision a firmware update for the HP222

(Firmware is still 13.10.53.00 !!)

Maybe a downgrade to esxi 5.1 is an option.

Reply
0 Kudos
gsbjoe
Contributor
Contributor

Before I opened a ticket with VMware, I pulled the card out of the ML350, installed it in another server with Windows installed on it, and upgraded the firmware within the OS, since the update didn't install from the latest HP Service Pack for Proliant.  Even so, the VM guest would not boot with ESXi v5.5.

I ended up installing a thumbdrive in the internal USB port and installed the HP branded VMware ESXi v5.1 to the thumbdrive.  Once the hypervisor booted, I reconnected my datastores, added my guests back to the inventory and booted up my guests without issue - including the guest with pass through enabled.  I'm still running my backups on that guest without issue.  Wish I could have all my hypervisors at the same version.  Also made me realize that these VMware upgrades should not be taken lightly.  If I was in a time crunch to get the upgrade performed and my services back up and running, I wouldn't have a margin to deal with passthrough not working.  I know, I know, in an ideal world, we would have a test environment that mimics our production environment exactly.  Well, I only have 1 HP H222 card, and it's in production.  Let's face it, we usually have older hardware in our test environment (I feel lucky to have some HP DL360 G5's as my test bed, and pass through doesn't even work with that model!).

gsbjoe
Contributor
Contributor

In a test, I installed an HP Ultra320 SCSI card in an HP DL380p Gen8 and installed VMware v5.5.  I configured passthru on the device, restarted the host, added it to a guest and the guest wouldn't boot (different error).  I shut it down, installed v5.1, configured passthru and was able to boot my guest with the hardware, and I have access to my SCSI tape drive attached to that Ultra320.  My conclusion is, passthru just isn't reliable in v5.5.  If you need to configure a passthru device, stay on v5.1.  Other than the unlocked memory limitation of v5.5, I don't see any other reason to upgrade. 

Reply
0 Kudos
NTShad0w
Enthusiast
Enthusiast

Mates,

im using nics, riad cards and graphics cards with VMDP form 2 years on supermicro "mainboard" server, hcl certified, and last days i bought HP DL380p g8 with e5-2630 server and observe such problem that when i wanna to vmdp onboard nic or any raid controller i try (lsi and adaptec) i have same error:

"

An error was received from the ESX host while powering on VM VMDP-test03.

Failed to start the virtual machine.

Failed to register the device pciPassthru0 for 7:0.0 due to unavailable hardware or software support

"

Its working ok (nics and raid cards) on supermicro mainboard X8DAH with 2x L5638 server on vmware ESXi v5.5 b1331820 but on the same vmware version (v5.5) not working on HP DL380p G8.

Im not test yet ESXi version 5.0 or 5.1 on that new server (HP DL380p G8) but in my first view it may be connected to a lot of settings in bios, especially power saving and throttling, so after test that it work or not with esxi v5.1 i will try to tune some bios default settings, get U informed/updated soon.

p.s. - great and very very long topic about VMDP other things than NICS is here:

VMDirectPath and ATI Radeon

regards

NTShad0w

--------------------

after some time with searching for my older ESXi v5.1 USB stick.. i boot that host (HP DL380p G8)... and looks like VMDP still not working with error (thios time, on ESXi v5.1 b1157734):

"

Device 7:0.0 is not a passthrough device

"

where 7.0.0 is a device id, i check 1 onboard NIC (Broadcom) and a LSI RAID Card also give me same error, so in my opinion for a 1st look its problem with a HP ML/DL Gen8 hosts (eventually new hosts with ESXi 5.x in general), so with big probability its something with default bios settings (I hope), if no, there someone need to make a support request if have and if are from bigger company (to not be dropped on thr tree...:P).

So next is to search bios settings...a lot of settings Smiley Sad

------------------------

hmm, I thonk i found a source of our problem mates... which is HP (G6, G7 and G8) and VMware (from version ESXi v5.1 U2)...:

http://h20565.www2.hp.com/portal/site/hpsc/template.PAGE/public/kb/docDisplay/?javax.portlet.begCach...

in my case, I updated the BIOS to newest one (2013-11-14) but in truly nothing change, same errors on VMware ESXi v5.1 (b1158834, its not U2 which is build 1483097) and on ESXi v5.5 b1331820

My HP DL380p Gen8 have 1x 2630 cpu, try vms with 2,4,8,12 and 16gb vram with vnx modifications and without, ehh notworking, need some more mates try other ideas and knowledge...

regards

NTShad0w

Reply
0 Kudos
jonathgar
Contributor
Contributor

Hi, I've just found an article from HP Support Center related to this topic.

It announced the end of support for "PCI passthrough" on HBAs, etc.

So, we wont be able to continue using the "PCI passthrough" on a host running esxi 5.1 U2 or higher.

I paste here the extract.

Regards

------------------------------------------------------------------------------------------

SUPPORT COMMUNICATION - CUSTOMER ADVISORY

Document ID: c03835455

Version: 1

Advisory: VMware ESXi 5.1 - The Virtual Machine Fails to Start and the Error Message "Device (PCI Bus/Dev/Fun) Is Not a Passthrough Device" Will Be Displayed

NOTICE: The information in this document, including products and software versions, is current as of the Release Date. This document is subject to change without notice.

Release Date: 2013-07-11

Last Updated: 2013-07-11


DESCRIPTION

VMware made a change in the VMware ESXi 5.1 Patch 02 to avoid potential data corruption that impacts the device passthrough capability used by VMDirectPath and vDGA (Virtual Dedicated Graphics Acceleration) features. For ProLiant servers without the appropriate System ROM revision, the result is that all devices in the system will be restricted from passthrough to the VM when the ESXi host is running ESXi 5.1 Patch 02 (or later).

VMware did not take into account the valid capability of PCI devices to reserve memory via RMRR (Reserve Memory Region Request). As a result, when a VM is configured for device passthrough, the memory reservation of that passthrough device is not communicated to the Guest OS. With the Guest OS unaware of the memory reservation, the opportunity for data corruption exists due to the fact that both the Guest OS and passed through PCI device believe that they each own the same region of memory.

Therefore, HP does not support the VMDirectPath feature which is utilized for passthrough of network and storage devices . There are currently no plans to enable passthrough of these devices.

When this issue occurs, the following message is displayed:

HP is planning to support the vDGA features which are utilized for passthrough of display adapters (GPUs). These GPUs do not utilize RMRR and can be configured for passthrough if the appropriate revision of the ProLiant System ROM is utilized.

SCOPE

Any ProLiant server that supports VMware ESXi 5.1. For a list of supported HP servers, refer to the support matrix available at:http://h18004.www1.hp.com/products/servers/vmware/supportmatrix/hpvmware.html

For an updated list of HP servers that support vDGA, refer to the VMware Compatibility Matrix:http://www.vmware.com/resources/compatibility/search.php Non-HP site .

RESOLUTION

HP does not support the VMDirectPath feature which is utilized for passthrough of network and storage devices . There are currently no plans to enable passthrough of these devices.

HP is planning to support the vDGA features, which are utilized for passthrough of display adapters (GPUs). These GPUs do not utilize RMRR and can be configured for passthrough if the appropriate revision of the ProLiant System ROM is utilized.

  • For ProLiant G6 and G7-series servers, there is no solution available to support the vDGA features.
  • For ProLiant Gen8-series servers, update the System ROM to a version dated June 2013 (or later) to enable support for GPU passthrough.
  • For the ProLiant WS460c Gen8 server blade , update the System ROM to a version dated 5/1/2013 (or later).
The latest revision of the System ROM is available as follows:
  1. Click on the following link:http://www8.hp.com/us/en/support-drivers.html
  2. Click on Drivers & Software.
  3. Enter a product name (e.g., ProLiant DL380 G7) and click on Search.
  4. Select the specific ProLiant model.
  5. Select the appropriate operating system.
  6. Click on the category BIOS-System ROM.
  7. Locate, download, and install the version of the System ROM, based on the server platform, as described above.
---------------------------------------------------------------------------------------------------------------------------------------------
Reply
0 Kudos
Frank_Dekker
Contributor
Contributor

Downgraded yesterday the server with esxi 5.1 u1 , my problem is solved now !

didn't change any bios settings / firmware of the hp222 still the same old version.

Thanks gsbjoe

Reply
0 Kudos
NTShad0w
Enthusiast
Enthusiast

yep I also, check version ESXi v4.1 U3 (b800380) and today I checked version ESXi v5.0 U3 (b1311175) and VMDP working ok on HP DL380p Gen8 (1x E5-2630 cpu).

and last working version in my opinion is ESXi v5.1 U1 (b1065491).

one of next version that dont work is ESXi 5.1 b1158834 (its something between U1 and U2 as I know)

regards

NTShad0w

Reply
0 Kudos
ClaytonRoss
Contributor
Contributor

Hello ALL,

I have seen here and in other places that this is an HP problem ,

But does this only effect the HP custom ISO of ESXI  or will a clean vanilla install have then same problem too?

What is the point of the HP iso other than drivers?

Reply
0 Kudos