VMware Cloud Community
daoyama
Contributor
Contributor

Atom C3758/X553 GbE driver for ESXi 6.5/6.7

A few days ago, I bought Atom C3758 based Mother Board.

Unfortunately, ESXi 6.5/6.7 have no inbox driver for X553 GbE.

So I build x553 only driver from ixgbe-5.3.6 linux driver.

I'm using it with following parts.

Asrock Rack C3758D4I-4L

Kingston KVR24R17D4K4/128

Intel ET Dual Port Server Adapter

(initial connection by igb inbox driver)

Transcend JetFlash 720S

SilverStone SST-CS280 / SST-ST45SF

I have tested with IPv4, MTU9000, no VLAN only. (mainly iSCSI connections)

Other setting may cause problems.

However, if you are interested in the driver, try it.

To use this driver:

# esxcli software acceptance set --level=CommunitySupported

# esxcli software vib install -v /tmp/net65-x553-5.3.6.x86_64.vib

Tags (2)
29 Replies
wuebra
Contributor
Contributor

Hello,

thanks for your driver! The installation works without problems.

Unfortunately I have problems with it.

Example:

VM: Win Server with VM Tools - vmnext3 or e1000

Copy files PC -> VM 118 Mbit!

Copy files VM -> PC 300 kbit!

If I load ISO files from the ESXI host, then it's full speed. Apparently it's the VM Tool driver? This is so at 5 different VM. (Windows, Linux)

Reply
0 Kudos
adavt
Contributor
Contributor

I have tried on supermicro A2SDi-8C (с3758/x553) too. Same problems.

Reply
0 Kudos
chefmagier
Contributor
Contributor

Hi,

I'm quite new to ESXI / -Cli, but somewhat advanced on Linux/Ubuntu.

I installed your driver package but I still get no new NIC detection.

Supermicro A2SDi-H-TF with 8086:15c8 ("X553/X557-AT")

Where gets the driver installed? Is it called "ixgbe" or "ixgben"? Or even some other name?

In /usr/lib/vmware/vmkmod/ I still see both older drivers vom the 6.7-installer and no newly installed driver whatsoever..

Help? Smiley Wink

edit: Installed it with "-f" with no change. "vmkload_mod -l | grep x553" shows nothing, as does "esxcli software vib list"

Reply
0 Kudos
daoyama
Contributor
Contributor

I forget to write about I/F setting in VMs.

I'm running FreeNAS with VMXNET3 and RDM on the ESXi host.

This is my current setting on iSCSI I/F in FreeNAS.

"mtu 9000 -tso"

As you can see, disable TSO is default my VM.

I don't know about Windows server, but if the vmxnet3 driver have

the setting, try to disable TSO or LRO.

If you have iperf3 on both side, try to check with/without TSO/LRO.

Reply
0 Kudos
daoyama
Contributor
Contributor

>I installed your driver package but I still get no new NIC detection.

Do you reboot the host?

The driver (not only this) is loaded on startup with bundled pciids.

If loaded pciids, you can see by  the command "vmkload_mod -l | grep x553".

Reply
0 Kudos
chefmagier
Contributor
Contributor

Hi,

Yes, of course I did a restart.

Sadly with an esxi-BSoD. So the driver seems not compatible with 6.7 or 6.5u2.

For which version did you compile?

Perhaps you could post a source how to compile for 6.7?

Reply
0 Kudos
Protogen
Enthusiast
Enthusiast

I have just finished building an ixgbe driver for Intel X553/7 NICs that can be used with ESXi 6.5 or later.

I created it by doing a code merge of Intel v5.1.3 + VMware v4.5.3.

VMware 6.5 and A2SDi-16C-HLN4F (cluster 2-node) | ServeTheHome and ServeThe.Biz Forums

Reply
0 Kudos
adavt
Contributor
Contributor

I couldn't download from the website "servethehome" even after registration ("You do not have permission to view this page or perform this action."). Could you place for downloading the driver here?

Reply
0 Kudos
chefmagier
Contributor
Contributor

Hi,

I can use the drivers from servethehome after doing a bios reset and :smileyalert: disabling ixgbe and ixgben. now I have the onboard NIC as physical devices in the GUI.

i'll do some tests in the next days.

thanks

Reply
0 Kudos
Protogen
Enthusiast
Enthusiast

I can use the drivers from servethehome after doing a bios reset and disabling ixgbe and ixgben. now I have the onboard NIC as physical devices in the GUI.

i'll do some tests in the next days

You should not need to disable the ixgbe or ixgben drivers.  My driver is named ixgbe_x553_7 and it will operate side-by-side with both ixgbe and ixgben.

You must make sure you uninstall daoyama's driver (net65-x553-5.3.6.x86_64.vib) because it maps to the same device IDs as my driver.  I'm not sure what would happen if you have both installed at the same time.

Reply
0 Kudos
DmitryDovidenko
Contributor
Contributor

It's awesome! Thank you!

Can you post this driver here?

Reply
0 Kudos
chefmagier
Contributor
Contributor

Yep

Protogen
Enthusiast
Enthusiast

Sorry I didn't get a chance to post my driver here.  The past two days I've been rebuilding my ESXi server on my Supermicro A2SDi-16C-HLN4F motherboard now that my X553 NICs are working.  Smiley Happy

I see however that chefmagier​ came to the rescue - thanks for posting in my absence.  Smiley Wink

Cheers

Craig

Reply
0 Kudos
Ubinewbie
Enthusiast
Enthusiast

Just wanted to say a big thank you for the drivers you compiled, saved my life! Or my home lab for that matter...

I successfully used it to build a custom ESXi iso with this tool: https://www.v-front.de/p/esxi-customizer-ps.html#download

Reply
0 Kudos
Protogen
Enthusiast
Enthusiast

Just wanted to say a big thank you for the drivers you compiled, saved my life! Or my home lab for that matter...

I successfully used it to build a custom ESXi iso with this tool: https://www.v-front.de/p/esxi-customizer-ps.html#download

Glad to hear it!  Smiley Happy

I stumbled upon the very same tool only a couple of days ago.  I used it to rebuild my ESXi 6.7 server.  It was a breeze to use (I had more trouble upgrading powershell and installing powercli).  Yeah, really great tool.

Reply
0 Kudos
wuebra
Contributor
Contributor

nice.....my custom iso works now great with your driver !

thank you so much

Reply
0 Kudos
Protogen
Enthusiast
Enthusiast

During last week I took my Supermicro A2SDi-16C-HLN4F motherboard (which I'd been using solely for testing while I was doing the driver code merge) and rebuilt it as my production server.  This finally allowed me to do some proper performance testing (SSDs at both ends, dedicated test network).

Everything went well and, same as before, I was getting around 80 MB/sec throughput.  However, when I took a closer look I noticed some sub-optimal timing values around buffering and flow control.

So, I decided to review the code again and I ended up making a few tweaks.  When I tested the updated driver I was able to reach a sustained throughput of 90 MB/sec - a performance increase of 12%.

The updated driver is attached.  Hopefully it performs just as well for you.  Smiley Happy

Ubinewbie
Enthusiast
Enthusiast

Interesting, im running the same motherboard. In file transfer tests between VM's on this host and a NAS, I get around ~110MB/sec, which is the most ive seen on a 1GB network.

This is using the original drivers you supplied, so odd that your getting slower speeds!

Reply
0 Kudos
Protogen
Enthusiast
Enthusiast

In file transfer tests between VM's on this host and a NAS, I get around ~110MB/sec, which is the most ive seen on a 1GB network.

That's impressive.  Very nice!

This is using the original drivers you supplied, so odd that your getting slower speeds!

My test results are directly related to my particular test setup (i.e. VM config, VM OS platform and version, NAS tuning, network switch, etc).  It's unlikely that another setup would perform exactly the same.

What's more important is the percentage increase in performance between the two drivers.  If anyone is getting a slower than expected speed, then perhaps the updated driver might be helpful.

Given your excellent results it appears there's some other factor slowing down my system.  Now that I'm aware of it I'll definitely look into it - so thank you!  Smiley Happy

Reply
0 Kudos