VMware Modern Apps Community
PaulB2
Enthusiast
Enthusiast

Maximum number of nics?

I am trying to configure 4 nics in photon v4, 3 work but when I add the 4th all the networking stops.

What is the maximum number of network cards, eth0, eth1, eth2, etc. supported?

Can anyone supply a config with more than 3 nics?

I have been using a Windows VM for testing my vsphere cluster networks and am able to get 10 nics in a Windows server OS.

9 Replies
DCasota
Expert
Expert

Did you try to add additional nics to the vm on the ui or programmatically? Here's a PowerCLI sample (see ref)

get-vm $VM | new-networkadapter -NetworkName "VM Network" -Type "VMXNET3" -StartConnected

Afaik tThe maximum network interfaces is hypervisor-related to the vHW and version.

https://kb.vmware.com/s/article/1003497

https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/plan/plan-hyper-v-scalability...

https://cloud.google.com/vpc/docs/create-use-multiple-interfaces

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html

 

In addition, there might be a specific maximum network interface quota. As example, on Azure GenV2 virtual machine size Standard_B2ms it has a limitation of 3 nics.

New-AzVM : The number of network interfaces for virtual machine ph4 exceeds the maximum allowed for the virtual machine size Standard_B2ms. The number of network interfaces is 4 and the maximum allowed
is 3.
ErrorCode: NetworkInterfaceCountExceeded

 

Quickly checked: On Workstation 9 with a Photon OS 2.0 including 5 vmnics started successfully with 4 nics in operational carrier state = off (networkctl).
Edit /etc/systemd/network/10-eth0.network in section [Match] use Name=eth* instead of Name=eth0,restart then systemctl restart systemd-networkd.

 

Reply
0 Kudos
PaulB2
Enthusiast
Enthusiast

Thanks for the reply. I spent quite a bit of time trying to get this to work without success. I did find this interesting link which discusses Linux and muti-home settings. https://netbeez.net/blog/avoiding-arp-flux-in-multi-interface-linux-hosts/

Adding the 4th adapter killed all the connections every time. I ended up going with w2019 core which works fine with 10 adapters.

Reply
0 Kudos
DCasota
Expert
Expert

Would you mind to share your findings? You say you switched to a Microsoft w2k19 core edition because of the fully supported multi-home feature up to 10 nics.

VCSA 7 includes multi-home feature support up with 4 nics. The commercial version of Photon is part of VCSA.

In which other scenarios is the full support of more than 4 nics highly desirable? Are the findings on Arm or on x86_64 cpu architecture?

Reply
0 Kudos
PaulB2
Enthusiast
Enthusiast

This was on x86_64. My use case is unique. I have ESXi clusters with multiple vlans and when they are configured or when adding a new host I need to validate that all  vlans are functioning. I have one cluster with 19 different vlans. So I build 2 VMs and add all the vlans for testing. Those 2 VMs will remain present for testing future changes.

Reply
0 Kudos
DCasota
Expert
Expert

Hi @PaulB2, the text-based setup wizard of Photon actually does not support configuring multiple nics + Virtual Guest Tagging (VST).

For testing purposes a kickstart config with systemd network vlan interface setup might be a good starting point. Interesting use case.

Reply
0 Kudos
anilspp
Enthusiast
Enthusiast

This thread gave lot of helpful info.

Reply
0 Kudos
ualonso
Contributor
Contributor

I understand these are photon images for Tanzu... if that is the case I'm also interested in adding vmxnet3 NICs but i don't find any info on how to do that

In my case I'm using TKGS and using the VMClass mechanism when creating the cluster only allows to define NICs of type directpath.. but not VMXNET3

Any guidance would be appreciated

Thanks! in avance

Reply
0 Kudos
ualonso
Contributor
Contributor

I have found https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.5/vmware-tanzu-kubernetes-grid-15/GUID-bu... which will allow to create images with several vNICs but there are two questions remaining:

- once downloaded the TKG-Image-Builder from code.vmware.com the package doesn't indicate how to change the hardware spec that will be put in the OVF...

 

- how to bind the vNIC to the network (L2 segment) at cluster creation time?

 

Thanks!

Reply
0 Kudos
ualonso
Contributor
Contributor

Answering to the first question myself:

Looking to the code of image-build-ova.py it seems this has to be specified with the following parameter: OVF_TEMPLATE and there is a sample XML in the git repository of the image builder

Any clue on the second question?

Reply
0 Kudos