Hi. I have an ESXi 6.7 host and I need to add BCM57810 card to it, but with the card installed I get pink screen with
MEM_ALLOC bora/vmkernel/main/memmap.c:3950
cr0=0x8001003f cr2=0x0 cr3=0x100168000 cr4=0x38
I updated firmware of the card to the latest I could find: 14.07.06
I installed 1.0.69.1 version of qfle3 driver.
Every time I try to boot with the card inserted it fails. Without the card the host is working without problems. What can I do?
This PSOD looks like an issue with the qfle3 driver. Can you run the below commands and share the output?
esxcfg-nics -l
esxcli software vib list | egrep -i 'qfle|bnx2'
Cheers,
Supreet
Can you share the screenshot of the complete PSOD screen?
Cheers,
Supreet
Sure.
What is the server make and model? Are you using a VMware standard image or a hardware vendor custom image for ESXi?
Cheers,
Supreet
It's not a server. I am using GA-H81M-S1, Intel Pentium G3450, 4GB RAM. Without BCM57810 it was working fine for months now. BCM57810 is working fine in Windows or Linux on the same PC. VMware standard image.
With untested, unsupported and non-certified hardware (or a assembly of hardware), we are bound to have such issues. That's the reason, there is a hardware compatibility list in place.
Cheers,
Supreet
Yeah, and I do not have issues with this board itself. I't a home lab that have been running fine. It's ESXi that won't start with BCM57810 installed. So, I would appreciate if someone who was experiencing similar issues with that card could help. Or if somebody knows where to look in logs to get more info about what makes it crash.
It is failing during memory allocation and since it is failing even before the dump system gets initialized, I don't think any of the logs would be of any help in this regard.
Cheers,
Supreet
Thanks for that info. It seems I'll have to switch to something else to do my test. It's a shame as ESXi in my opinion is most user friendly. Well, not so much since vSphere Client is no longer available for current versions but still. Thanks again for your time.
What is the boot mode set in the BIOS? Is it UEFI or BIOS? Is CSM turned on or off? It might make a difference since this is a problem occurring at boot up time.
The CR4 register value indicates PAE is expected (address size wider than 32-bits, i.e. access memory location beyond 4GB boundary), so it might work with UEFI mode.
Thanks, that got me into playing with RAMs. The thing is, when BCM57810 is connected, the MB does not see one slot of RAM making it only 2GB for ESXi. I tried using 2x 4GB (still with the card on only one is recognized) and ESXi loaded a little further but got another error. Anyway, thanks for your help.
This PSOD looks like an issue with the qfle3 driver. Can you run the below commands and share the output?
esxcfg-nics -l
esxcli software vib list | egrep -i 'qfle|bnx2'
Cheers,
Supreet
You have gotten further but not by much :smileylaugh: ; less than 16 seconds based on the VM kernel uptime of the PSOD screenshots.
The thing is, when BCM57810 is connected, the MB does not see one slot of RAM making it only 2GB for ESXi.
Is this evident already at the UEFI/BIOS level (i.e. going into UEFI/BIOS setup, it already only sees one stick of RAM? Was this problem also present with the Windows and Linux setups?
I am assuming that you inserted the card on the PCIe x16 slot. Try inserting the card on the PCIe x1 slots assuming that there are no physical impediments such as capacitors and other motherboard components that the "teeth" of the PCIe card connectors will hit. Also try reducing the amount of video memory setup in the UEFI for the integrated graphics of the CPU. These steps are to try to remedy the situation of seeing only one slot of RAM when the card is inserted.
Well, those 16 seconds allowed me to notice that ESXi saw 4GB RAM when 8GB was inserted. Yes, it is the same at UEFI BIOS level. I didn't check in Linux or Windows but as it is like this in BIOS, I don't think it was different. Yes, the card is inserted in 16x slot. The card is 8x so I can't try it in 1x slot. I did reduce video memory to minimum and there was no difference.
esxcfg-nics -l (obviously without the card connected):
[root@localhost:~] esxcfg-nics -l
Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 0000:03:00.0 ne1000 Down 0Mbps Half 68:05:ca:17:b4:6d 6000 Intel Corporation Gigabit CT Desktop Adapter
vmnic1 0000:04:00.0 r8168 Up 1000Mbps Full 50:3e:aa:02:3b:ec 6000 Realtek Semiconductor Co., Ltd. TG-3468 Gigabit PCI Express Network Adapter
vmnic2 0000:02:00.0 r8168 Down 0Mbps Half 40:8d:5c:3a:9f:b7 6000 Realtek Semiconductor Co., Ltd. Onboard Ethernet
esxcli software vib list
[root@localhost:~] esxcli software vib list | egrep -i 'qfle|bnx2'
qfle3 1.0.69.1-1OEM.670.0.0.8169922 QLC VMwareCertified 2018-10-07
net-bnx2 2.2.4f.v60.10-2vmw.670.0.0.8169922 VMW VMwareCertified 2018-06-27
net-bnx2x 1.78.80.v60.12-2vmw.670.0.0.8169922 VMW VMwareCertified 2018-06-27
qfle3f 1.0.25.0.2-14vmw.670.0.0.8169922 VMW VMwareCertified 2018-06-27
qfle3i 1.0.2.3.9-3vmw.670.0.0.8169922 VMW VMwareCertified 2018-06-27
scsi-bnx2fc 1.78.78.v60.8-1vmw.670.0.0.8169922 VMW VMwareCertified 2018-06-27
scsi-bnx2i 2.78.76.v60.8-1vmw.670.0.0.8169922 VMW VMwareCertified 2018-06-27
Looking at this Dell webpage
From the picture and the "Overview" mentioning PCIe x1 support, you should be able to try the PCIe x1, unless the different OEM versions have different specs. The only issue would be if there are motherboard components such as capacitors that get in the way of card connector teeth.
The other alternative is to force a passthru by manually editing the esx.conf (assuming it is still the same with ESXi 6.7) by specifying "passthru" for the PCIe slot where the card is to be inserted.
https://kb.vmware.com/s/article/1010789
If it is in passthru mode, the ESXi driver should not load and thus avoid the PSOD once the card is inserted. May not be exactly what you have originally in mind, but if this successfully boots the ESXi with the 57810 card inserted, at the very least you could try the passthrough through a VM.
For the above PSOD, let us uninstall the qfle3 related drivers so that the card is forced to use the bnx2x driver. Run the below commands and reboot the host. Once host is rebooted, check if the qfle3 drivers are uninstalled. Post validation, insert the card and see if you still hit the same PSOD.
esxcli software vib remove --vibname=qfle3
esxcli software vib remove --vibname=qfle3f
esxcli software vib remove --vibname=qfle3i
Cheers,
Supreet
I tried upgrading qfle3f and qfle3i drivers as I upgraded only qfle3 and now it booted fine, no PSOD. It's still a mystery why only half of RAM is recognized on this system, but doesn't matter much as this card is only needed for some tests. Thank you for your help and pointing me in the right direction.
Hi, I have the same problem, with card inserted the systems sees only 1 ram slot. Did you understand how solve the problem using the nic card with full ram? Thank you