- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Just a quick heads up:
- A new Intel X(L)710 firmware 6.01 has been released on Aug 22 - with no release notes to be found: Download Non-Volatile Memory (NVM) Update Utility for Intel® Ethernet Converged Network Adapter XL71...
- There's also a new Linux based driver 2.1.26 that corresponds to said firmware: https://downloadmirror.intel.com/24411/eng/releasenotes.txt - some of the issues fixed sound interesting, but as I do not know the root cause of our problems yet, I cannot really tell whether they would be fixed by that driver.
- No mention of the firmware/driver on the VMware HCL or driver downloads so far (which isn't really surprising, given how recent they are).
- My VMware SR regarding the X(L)710/i40e(n) problems has now supposedly resulted in an internal ticket to VMware dev/engineering, unfortunately I don't know any details as of yet.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I just finished updating my 6 host vSAN cluster to the i40en 1.4.3 drivers. This has cut latency by over 50% and seems to have helped the general behavior of the cluster. I was also able to enable flow control although it is unclear as to if the NIC is actually obeying this configuration.
I was previously on 2.0.6 i40e drivers and this driver still seemed to have some issues.
I am running a Dell branded x710 so I don't have access to 6.x firmware at the moment.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi cjckalb,
Did you manage to resolve your issue?
I have experienced something similar on i40e 2.0.6 however this happens rather rarely and therefore I have not managed to see whether disabling LRO/TSO helps.
Do you by any chance have managed to simplify the scenario that gets the PSOD to reproduce constantly? Having that would make it easier to get VMware/Intel to fix it.
==
Thanks,
Adam
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Adam,
we did not use i40e for too long, so we did not have the chance to isolate what exactly caused the PSODs. I would suggest trying the latest i40en 1.4.3 + the latest firmware 6.01 instead, as that combination would not cause PSODs for us. Even with the latest driver + firmware, it would however reset the interface intermittently (like once or twice a month). Unfortunately, we could not isolate the exact traffic pattern and/or other conditions causing this aswell. The last thing VMware support suggested was turning off TSO. We have since completely switched to QLogic (Broadcom) NICs a couple of days ago, since neither Intel nor VMware seemed too interested in determining and fixing the root cause, and we could not waste any more time on this issue.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am sorry to hear that you have had this issue with the Intel Ethernet CNA X710.
This issue has been addressed with the latest i40en driver and 6.01 firmware.
Here are the links to the download pages.
Brian Johnson
Solutions Architect -- Intel Corp
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Just tried the new i40en drivers (1.5.6) but had to switch back to the i40e_2.0.7 immediatly again, as VLAN tagging was no longer working properly with the i40en drivers.
Our X710-DA4 cards are of course already on the 6.01 NVM firmware and TSO/TSO6/LRO is disabled on our hosts. (due to problems and PDODs with both the i40e and i40en driver back in the days...and we never dared to enabled it again)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Just got this PSOD today with 6.01 NVM firmware and i40e_2.0.7 driver. (UseHwTSO, UseHwTSO6 and TcpipDefLROEnabled disabled)
Did now also disable Vmxnet3HwLRO and Vmxnet3SwLRO, but don't think it's worth much. Really seems like today is the time to dump these NICs....(like everything that comes from Intel lately)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Did anyone ever get his Intel X710 NICs to work without issues using esxi 6.0 or 6.5 ?
So far we have tried multiple driver / firmware combinations, multiple vmware versions etc... for nearly two years now... nothing ever was fully stable...
During times with high traffic load (usually during the nightly backups) the NIC sometimes just stops passing traffic to the vm and never recovers until the VM is reset.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'm currently experiencing similiar problems with HPE branded cards (562SFP+).
Log entries show the following:
2018-03-02T11:15:56.116Z cpu12:66212)i40en: i40en_HandleMddEvent:6969: Malicious Driver Detection event 0x02 on TX queue 0 PF number 0x00 VF number 0x00
2018-03-02T11:15:56.116Z cpu12:66212)i40en: i40en_HandleMddEvent:6995: TX driver issue detected, PF reset issued
Driver/firmware versions tested so far:
driver 1.4.3 and firmware 5.60
driver 1.5.6 and firmware 5.60
driver 1.5.6 and firmware 6.01
Driver 1.5.6 is HPE version 2018.02.12
Firmware 6.01 is HPE version 10.2.5
There is a new driver version 1.5.8 listed on the HCL but the recommended version from HPE is 1.5.6.
Support request with VMware is open but no solution so far...
HPE firmware: https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX_87c83853cb5a4bc5949e9b0dd5#tab5
HPE driver : Drivers & Software - HPE Support Center.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Seeing the same exact thing on a brand new install of Dell R740 servers with the dual port x710. This card is a piece of sh*t and should just be thrown in the trash. Even with the 1.5.8 driver + 6.01 firmware combo, and disabling the optimizations, still dies under load if vlan tagging is being used. That of course required deviating from Dell's supported firmware updates as their latest only puts 6.00 on the card. Intel has known about these issues for years and hasn't solved it.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
We have the same issue and it's really frustrating as this has caused some of our productions hosts to go offline.. I hope it gets fixed soon.
Intel X710 / i40en driver on vmware - any fix? |Intel Communities
Regards
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We recently received 8 brand new HPE DL380 Gen10 servers with the HPE branded cards (562SFP+)
We are experiencing the same issues as everyone else. In the ESXi console we associated two of these nics for the management network. After assigning an IP address, mask, gateway, and DNS servers we restart a host. During the restart we have a continuous ping going to the management network IP address. Part way through booting up the ping returns. After about 30 seconds the ping drops and never comes back. If we remove one of the nics, the ping returns. In other words, with two of these nics as part of the management network, they will not pass traffic. We are on driver version 1.5.8 and firmware version 10.2.5
Any solutions yet? I have an open ticket with VMWare and HPE for nearly two weeks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
So far, we don't have such problems on Dell r740xd running ESXi 6.5 U1 with Intel x710 network adapters :
- firmware version : 18.3.6
- driver type : i40en
- driver version : 1.5.6
- no TSO desactived or whatever
- connected to Cisco Nexus 5000 network switchs
On those x710 adapters, we're using lacp lags, vlan tagging and jumbo frames : no problems yet (ESXi are up and running since ~ 20 days without network outages or psod).
We're not using SR-IOV, maybe VXLAN in a near future.
I'll keep you posted if we face any problems !
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Maybe the suggestions here will help?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Problem solved, I figured out the issue this morning. The two Juniper switch ports used for the management network have jumbo frames enabled. Once ESXi was installed on the new HPE server, the first step is to go into the ESXi console and setup the management network. Nics are assigned, IP addresses added etc. When this is done ESXi in the background is creating a vSwitch and Kernel port. The vSwitch was made just fine and it had its MTU set for 9000 which is correct for Jumbo Frames. The Kernel port was also made but its MTU was set to 1500, not 9000. This cannot be seen from the ESXi console, you have to go into the web interface to see this. Once I changed the kernel port to 9000 for MTU I could add the second nic and it worked perfect. When ESXi made the vSwitch and Kernel port it should have set both to an MTU of 9000 and it did not.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
My issue was actually not solved and it was not a Jumbo Frames mismatch. It turns out the Intel X710 based nics do not handle LLDP properly, this post gives the details
Where did my host go.... - Virtual Ramblings
A few more posts that do not give this nic glowing reviews
https://lonesysadmin.net/2018/02/28/intel-x710-nics-are-crap/
vSphere 6.5 with Intel X710 network adapter
We just put in one Broadcom nic for testing and it seems to be working fine. May end up swapping all of our Intel nics for Broadcoms
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Does this issue resolved?. I saw new driver version i40en version 1.5.8 released on Feb 2018. Does this resolves the issue?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'd say this is resolved with the following driver update: Download VMware vSAN (
VMware ESXi 6.7 i40en 1.7.1 NIC Driver for Intel Ethernet Controllers X710,XL710,XXV710 and X722 family)
I built a server using a SuperMicro X11SPi-TF which have two 10GBE Intel X722 + X557 on board.
Before updating the driver I had two occurances where the network would not respond (IPMI using a dedicated NIC would respond), and using esxcli to disable/enable the NICs would resume normal operation leading to a driver error so my HW was OK.
After updating the driver with the one in the link above, everying is as it should with no network interruptions for several weeks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This is for ESXi 6.7 only. There is still no solution for ESXi 6.5.
Our last support case with vmware was solved by replacing the cards with different ones.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Fresh news for the i40en driver for ESXi 6.5 (version 1.7.5) :
https://my.vmware.com/group/vmware/details?downloadGroup=DT-ESXI65-INTEL-I40EN-175&productId=614
Haven't be able to test it already.
We're currently using the 1.5.8 driver, and we're suffering some network outage from time to time (the Malicious Driver Detection bug).