VMware Cloud Community
sabarirjpm
Contributor
Contributor

High pNic error rate detected. Check the host's vSAN performance view for details

Received this alert in vCenter and the kb aritcle https://kb.vmware.com/s/article/83627 says that it is related to the error conditions between vmnic and the physical port. Can someone help me to identify which port has issues for me to debug further. I see this alert in two of the seven esxi hosts.

0 Kudos
26 Replies
ralfklein
Enthusiast
Enthusiast

Hi. Can it be that the ESXi hosts are freshly installed? Some ESXi hosts also showed me this message shortly after the installation. After the traffic was higher, the message disappeared

0 Kudos
sabarirjpm
Contributor
Contributor

Hi ralfklein

These are updated couple of months before. The article says that the error can appear even if it is found before the upgrade. Not sure how to check the current counters or reset the counters to see if it happens again.

0 Kudos
arshad21dvmw
Contributor
Contributor

Hi ,

What's the hardware of the server. You might have to look into the server manager ( IDRAC for DELL or iLO for HPE) of the servers for more CRC errors. 

Identifiying which interface is having a drop packet on an esxi, ideal option is to run esxtop 

  1. Run esxtop via ssh on the problem esx host 
  2. Type n
  3. Search for any packets being dropped under %DRPTX and %DRPRX.
0 Kudos
sabarirjpm
Contributor
Contributor

Hi Arshad

 

Thanks for the reply. esxtop shows the %DRPTX and %DRPRX as 0.00. Manually acknowknowledged the alarm and reset it to green and haven't seen it reoccur

0 Kudos
TheBobkin
Champion
Champion

@sabarirjpm , In the vSphere client, if you select the host in question and go to Monitor > vSAN > Performance > Physical Adapters > Select vmnic used (and/or other one) it will show you any pNIC errors for the time period in question - unfortunately the graphs can only show 24hrs worth of data at a time so it would be worthwhile validating the timestamp of when the alarm/alert was triggered and check the 24hrs prior to that.

0 Kudos
SoheilSP
Contributor
Contributor

Hi

I have the same issue. Our Vcenter ESXi and vSAN are the latest versions .We changed NIC and SFP+ and Fiber ، upgrade NIC driver. but still, we have the problem.

Any body can help ? 😞

0 Kudos
Kamikadz3
Contributor
Contributor

Hi,

error is reported on ESX 7.0.2 and higher. To troubleshoot it you need to check ESX NIC counters.

Run following script then review output:

#/usr/lib/vmware/vm-support/bin/nicinfo.sh

 

____________
Call a spade a spade
0 Kudos
Digory34
Enthusiast
Enthusiast

What is your Network card ? you can check the driver release to see if this alarm can be safely ignored. Should be the case for some of intel.

0 Kudos
Kamikadz3
Contributor
Contributor

I'd say check NIC counters first. Then compare results with kb#83627

(https://kb.vmware.com/s/article/83627)
Also, confirm if this is Intel card (x710 or x810) or other vendor

____________
Call a spade a spade
0 Kudos
SoheilSP
Contributor
Contributor

Hi

I ran the script and the output file is attached. I have this issue on VMnic5. I don't really understand, can you help Me? 🙂

0 Kudos
Kamikadz3
Contributor
Contributor

      Total receive errors: 236952

Two days ago HP published this:
https://support.hpe.com/hpesc/public/docDisplay?docId=a00126305en_us&docLocale=en_US%20

Looks like most of times FW upgrade is going to fix problem on Qlogic part.
Also, AFAIK next 7.0.3 update, should fix most of pNIC error warnings

____________
Call a spade a spade
0 Kudos
SoheilSP
Contributor
Contributor

Hi Kamikadz3

Thank you so much for helping me 

0 Kudos
basteku73
Enthusiast
Enthusiast

We had this on fujitsu servers.

I had to update the fw of my network cards and install the appropriate driver that was supported for this fw in the compatibility list.

I had this on vSphere 7.0.2,

Regards, 

Sebastian

0 Kudos
Kamikadz3
Contributor
Contributor

>>>I had this on vSphere 7.0.2<<<

This is because VMware introduced that alarm in vSphere 7.0.2. It is possible that lot of environments had that issue for years, and  upgrade to 7.0.2 uncover that problem.

🙂
 

 

____________
Call a spade a spade
0 Kudos
jamiegravatte
VMware Employee
VMware Employee

Hi @sabarirjpm , did this answer your question? If so, please check the "verified solution" button in order to better help your peers find this information!

Thanks!

Jamie - Digital Support Team

0 Kudos
buffalix
Contributor
Contributor

We have this problem for quite a while and after reseting it to green it will come back again.

 

0 Kudos
buffalix
Contributor
Contributor

pause rate at 2-3%

0 Kudos
Kamikadz3
Contributor
Contributor

What type of NIC you use? 

____________
Call a spade a spade
0 Kudos
buffalix
Contributor
Contributor

Intel(R) Ethernet Controller 10 Gigabit X540-AT2

Driver Info: 

         Bus Info: 0000:82:00:1

         Driver: ixgben

         Firmware Version: 4.04.0 0x8000063a, 19.5.12

         Version: 1.13.1.0

 

Thanks

0 Kudos