NIC detected, no flow/traffic 6.5.0 Update 1 (Build 7388607)

[mohsh86]

I have an AMD Ryzen 1800x on Asus Prime B350 Motherboard.

I initially had issues with the onboard Realtek Gb LAN, i tried the community vib, searched forums, eventually found a workaround of shutting down the esxi, cut off the power from the MB, then powering it back again.

Since this isn't practical, i bought an Intel E1G42ET  (supposedly supported) and chucked in on the 2nd PCIe x16 slot (the rest of the slot empty of course). esxi detect the two nics (driver igb), but:

1. There is no flow from the NIC/s

2. NIC will show Up - 1000 Full when connected, but when disconnected, will still show Up with speed of 65535 and Full duplex?!

any idea where i should start looking ?

Cheers

[Edit-1]

To confirm the issue isn't hardware, i booted a live Ubuntu and tried the 3 nics, all works like a charm!

[Edit-2]

Driver currently used by ESXi is "igb"

   Module: igb

   Module File: /usr/lib/vmware/vmkmod/igb

   License: GPL

   Version: Version 5.0.5.1, Build: 4564106, Interface: 9.2 Built on: Oct 26 2016

   Build Type: release

   Provided Namespaces:

   Required Namespaces: com.vmware.driverAPI@9.2.3.0, com.vmware.vmkapi@v2_3_0_0

   Containing VIB: net-igb

   VIB Acceptance Level: certified

however, listing VIBs containing "igb" would yeild:

igbn                           0.1.0.0-15vmw.650.1.36.7388607        VMW      VMwareCertified     2018-02-05

net-igb                        5.0.5.1.1-5vmw.650.0.0.4564106        VMW      VMwareCertified     2018-02-05

is the system using the right driver ?

[Finikiez]

What do you mean saying 'there is no flow'?

How do you check?

[mohsh86]

I check via Wireshark, hooking a laptop directly to the NIC and sniffing through Wireshark, no packets.

[Finikiez]

can you show your virtual switch configuration?

As well can you show management network settings (which vmnic is used) from DCUI?

[mohsh86]

vSwitch1

   Name: vSwitch1

   Class: etherswitch

   Num Ports: 3072

   Used Ports: 4

   Configured Ports: 1024

   MTU: 1500

   CDP Status: listen

   Beacon Enabled: false

   Beacon Interval: 1

   Beacon Threshold: 3

   Beacon Required By:

   Uplinks: vmnic1

   Portgroups: Intel_vmnic1

management network and VM network are on vSwitch 0 that is hooked up to the built-in LAN (Realtek - vmnic0) which is working fine, i am trying to get the Intel 82576 dual NIC to work.

adding vmnic1/2 as uplinks to the management network (setting as stand-by, pulling main cable ) does not get them to initiate traffic

[Finikiez]

Ok.

So you have virtual machine portgroup on vSwitch1 and some VMs connected to it. Right? Can VMs in the same port group ping each other\or connect with any other protocol (RDP\SSH\etc..)?

Do you have VLAN id assigned to this port group?

Can you show some outputs as well:

esxcli network nic list

than

esxcli networ nic get -n vmic# (where# is the number of affected network adapter)

as well can you attach vmkernel.log from the host?

[mohsh86]

I was able to ping/rdp between guests on the same Port Group / vSwitch, i've attached all files in /var/log as well as dmesg output

[root@pastech-vms:~] esxcli network nic list

Name    PCI Device    Driver  Admin Status  Link Status  Speed  Duplex  MAC Address         MTU  Description

------  ------------  ------  ------------  -----------  -----  ------  -----------------  ----  -----------------------------------------------------------------------------------------

vmnic0  0000:04:00.0  r8168   Up            Up            1000  Full    88:d7:f6:7a:68:fa  1500  Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller

vmnic1  0000:07:00.0  igb     Up            Up            1000  Full    90:e2:ba:f9:15:4c  1500  Intel Corporation 82576 Gigabit Network Connection

vmnic2  0000:07:00.1  igb     Up            Down             0  Half    90:e2:ba:f9:15:4d  1500  Intel Corporation 82576 Gigabit Network Connection

[root@pastech-vms:~] esxcli network nic get -n vmnic1

   Advertised Auto Negotiation: true

   Advertised Link Modes: 10BaseT/Half, 10BaseT/Full, 100BaseT/Half, 100BaseT/Full, 1000BaseT/Full

   Auto Negotiation: true

   Cable Type: Twisted Pair

   Current Message Level: 7

   Driver Info:

         Bus Info: 0000:07:00.0

         Driver: igb

         Firmware Version: 1.2.1

         Version: 5.0.5.1

   Link Detected: true

   Link Status: Up

   Name: vmnic1

   PHYAddress: 1

   Pause Autonegotiate: true

   Pause RX: true

   Pause TX: true

   Supported Ports: TP

   Supports Auto Negotiation: true

   Supports Pause: true

   Supports Wakeon: true

   Transceiver: internal

   Virtual Address: 00:50:56:5f:eb:8b

   Wakeon: MagicPacket(tm)

[mohsh86]

Another thing worth mentioning:

1. i ran wireshark on the Win10 guest, when ever i change a setting on the corresponding vSwitch the following packet is received.

2.  I've just checked the switch, looks like port 9 (where vmnic1 is connected) is sending bytes (bytes received on switch statistics), however, connecting the NIC directly to my laptop with wireshark on yields nothing.

[Finikiez]

Can you try to update the driver https://my.vmware.com/group/vmware/details?downloadGroup=DT-ESXI55-INTEL-IGB-525&productId=353  ?

As well can you describe a bit more your test when you tried to check network connectivity? I'd like to see ifconfig/ipconfig from both source and destination and information about VLAN ID assigned to VM port group?

[Finikiez]

You can capture network traffic inside port group only when you configure Promiscuous mode.

Can you also say what is a NIC Teaming policy you use on vSwitch1\port group?

[mohsh86]

I've enabled Promiscuous mode on vSwitch1, nothing is coming from the outside.

Teaming policy is inherited from vSwitch1

vSwitch1

[mohsh86]

this is the output of statistics for NIC:

NIC statistics:

     rx_packets: 1350

     tx_packets: 0

     rx_bytes: 299706

     tx_bytes: 0

     rx_broadcast: 645

     tx_broadcast: 0

     rx_multicast: 702

     tx_multicast: 0

     multicast: 702

     collisions: 0

     rx_crc_errors: 0

     rx_no_buffer_count: 0

     rx_missed_errors: 7663

     tx_aborted_errors: 0

     tx_carrier_errors: 0

     tx_window_errors: 0

     tx_abort_late_coll: 0

     tx_deferred_ok: 0

     tx_single_coll_ok: 0

     tx_multi_coll_ok: 0

     tx_timeout_count: 6

     rx_long_length_errors: 0

     rx_short_length_errors: 0

     rx_align_errors: 0

     tx_tcp_seg_good: 0

     tx_tcp_seg_failed: 0

     rx_flow_control_xon: 0

     rx_flow_control_xoff: 0

     tx_flow_control_xon: 0

     tx_flow_control_xoff: 7668

     rx_long_byte_count: 299706

     tx_dma_out_of_sync: 0

     tx_smbus: 0

     rx_smbus: 0

     dropped_smbus: 0

     os2bmc_rx_by_bmc: 0

     os2bmc_tx_by_bmc: 0

     os2bmc_tx_by_host: 0

     os2bmc_rx_by_host: 0

     rx_errors: 0

     tx_errors: 0

     tx_dropped: 0

     rx_length_errors: 0

     rx_over_errors: 0

     rx_frame_errors: 0

     rx_fifo_errors: 7663

     tx_fifo_errors: 0

     tx_heartbeat_errors: 0

     tx_queue_0_packets: 0

     tx_queue_0_bytes: 0

     tx_queue_0_restart: 0

     rx_queue_0_packets: 0

     rx_queue_0_bytes: 0

     rx_queue_0_drops: 0

     rx_queue_0_csum_err: 0

     rx_queue_0_alloc_failed: 0

     rx_queue_0_ipv4_packets: 0

     rx_queue_0_ipv4e_packets: 0

     rx_queue_0_ipv6_packets: 0

     rx_queue_0_ipv6e_packets: 0

     rx_queue_0_tcp_packets: 0

     rx_queue_0_udp_packets: 0

     rx_queue_0_sctp_packets: 0

     rx_queue_0_nfs_packets: 0