VMware Cloud Community
TheCableGuy96
Contributor
Contributor

Multiple NICs on Multiple vLANs

Hi,
 
I'm new to ESXi and have managed to get hold of an old Dell R720xd and put ESXi 8 on it.
 
It has 4 NICs and I'm really struggling to see how I can assign each one a different IP on different vLANs so I can assign different nodes t different vLANs.
 
Would anyone be able to offer me any help please?
 
Thank you in advance!
Tags (3)
Reply
0 Kudos
27 Replies
depping
Leadership
Leadership

You don't need to assign IP addresses to the NICs itself. You assign an IP to the different types of interfaces you need, like for instance: management, vMotion, iSCSI etc.

You can create what VMware calls "portgroups" and set a VLAN ID on that level, if you then add VMs to that portgroup the traffic will be tagged with that VLAN. This is what is called "virtual switch tagging". Some documentation can be found here:

https://kb.vmware.com/s/article/1004074

https://kb.vmware.com/s/article/1003806 

TheCableGuy96
Contributor
Contributor

Hi depping,

 

Thank you for your reply. 

So from those articles I believe I need to setup "Virtual Switch Tagging". I have managed switches and know how to set tagged ports so I can manage that.

I have created the port groups now, but I don't see how to assign each one an IP address? When I try to edit the settings for it there are no IP options, please could you clarify where to do this?

Also with the port groups I am wanting to setup "Route based on IP hash" NIC teaming (from what I have read this is the best one for load balancing). I am worried though that when I turn this on I may lose access to the server if not setup correctly. I have managed switches but when I switch this on do I simply turn on LACP on the switch for each port or is that something different? If it said "LACP" in the options I would be more confident but it doesn't so I'm not 100% it's the same thing?

Thank you again for your help.

Reply
0 Kudos
depping
Leadership
Leadership

Why do you want to give the NIC an IP address? That is the whole thing, you don't need to give a NIC an IP address, you give the "vmkernel interface" an IP address, which then takes care of "management" traffic, or "vMotion traffic". VMs will get their IP assigned in the operating system etc.

Reply
0 Kudos
TheCableGuy96
Contributor
Contributor

Okay sorry it's just all new to me.

So I've now created a new VMKernel NIC and set it to the new port group for the additional vLAN. I have also set the IP and subnet and assigned it to that to a new TCP/IP stack I created for this vLAN (had to do this in SSH).

However, when I go to set the gateway and DNS settings for the new TCP/IP stack it's greyed out. The gateway below this is not reyed out but fails to set. So I googled around and found someone else who had the same issue. I followed what they did:

esxcli network ip interface ipv4 set --interface-name=vmk1 --type=static --ipv4=192.168.10.40 --netmask=255.255.255.0 --gateway=192.168.10.254

This appears to have worked from what I see:

[root@esxi:~] esxcli network ip interface ipv4 get
Name IPv4 Address IPv4 Netmask IPv4 Broadcast Address Type Gateway DHCP DNS
---- ------------- ------------- -------------- ------------ -------------- --------
vmk0 192.168.1.40 255.255.255.0 192.168.1.255 STATIC 192.168.1.254 false
vmk1 192.168.10.40 255.255.255.0 192.168.10.255 STATIC 192.168.10.254 false

But it's not showing as being set in ESXi Host Client and I still don't see how to set the DNS for it?

192.168.10.40 is also not responding to ping even though I set all 4 NIC ports as part of that vLAN (I've tried tagged and untagged).

Thanks for your support.

 

Reply
0 Kudos
TheCableGuy96
Contributor
Contributor

I keep trying to reply but the system keeps removing it and saying my reply has been marked as spam... not sure how to respond now?

Tags (1)
Reply
0 Kudos
TheCableGuy96
Contributor
Contributor

So I've now created a new VMKernel NIC and set it to the new port group for the additional vLAN. I have also set the IP and gateway and assigned it to that to a new TCP/IP stack I created for this vLAN (had to do this in SSH).

However, when I go to set the gateway and DNS settings for the new TCP/IP stack it's greyed out. The gateway below this is not reyed out but fails to set. So I googled around and found someone else who had the same issue. I followed what they did:

esxcli network ip interface ipv4 set --interface-name=vmk1 --type=static --ipv4=192.168.10.40 --netmask=255.255.255.0 --gateway=192.168.10.254

This appears to have worked from what I see:

[root@esxi:~] esxcli network ip interface ipv4 get
Name IPv4 Address IPv4 Netmask IPv4 Broadcast Address Type Gateway DHCP DNS
---- ------------- ------------- -------------- ------------ -------------- --------
vmk0 192.168.1.40 255.255.255.0 192.168.1.255 STATIC 192.168.1.254 false
vmk1 192.168.10.40 255.255.255.0 192.168.10.255 STATIC 192.168.10.254 false

But it's not showing as being set in ESXi Host Client and I still don't see how to set the DNS for it?

192.168.10.40 is also not responding to ping even though I set the ports as part of that vLAN (I've tried tagged and untagged).

Thanks for your support.

Reply
0 Kudos
TheCableGuy96
Contributor
Contributor

So I've now created a new VMKernel NIC and set it to the new port group for the additional vLAN. I have also set the IP and gateway and assigned it to that to a new TCP/IP stack I created for this vLAN (had to do this in SSH).

However, when I go to set the gateway and DNS settings for the new TCP/IP stack it's greyed out. The gateway below this is not reyed out but fails to set. So I googled around and found someone else who had the same issue. I followed what they did:

esxcli network ip interface ipv4 set --interface-name=vmk1 --type=static --ipv4=192.168.10.40 --netmask=255.255.255.0 --gateway=192.168.10.254

This appears to have worked from what I see:

esxcli network ip interface ipv4 get
Name IPv4 Address IPv4 Netmask IPv4 Broadcast Address Type Gateway DHCP DNS
vmk0 192.168.1.40 255.255.255.0 192.168.1.255 STATIC 192.168.1.254 false
vmk1 192.168.10.40 255.255.255.0 192.168.10.255 STATIC 192.168.10.254 false

But it's not showing as being set in ESXi Host Client and I still don't see how to set the DNS for it?

192.168.10.40 is also not responding to ping even though I set the ports as part of that vLAN (I've tried tagged and untagged).

Thanks for your support.

Reply
0 Kudos
TheCableGuy96
Contributor
Contributor

@depping I cannot reply with the information you need to continue my support because the automated system keeps removing it. I've tried several times editing it but it still deletes it as spam when it's not. 

Reply
0 Kudos
depping
Leadership
Leadership

I just unmarked them as spam. I am wondering what you are trying to achieve here? 

Reply
0 Kudos
TheCableGuy96
Contributor
Contributor

I'm trying to achieve 2 things....

1) I want my virtual switch to have multiple VMKernel NICs, each one residing on a different vLAN assigned to a different gateway and specify the DNS servers (just like you would setup any NIC but each on is on a different vLAN).

2) Once that is done I want to setup LACP with the 4 NICs.

That is all but I'm struggling.

Reply
0 Kudos
depping
Leadership
Leadership

I don't understand why you need multiple vmkernel interfaces to configure LACP/portchannels? It feels like you are applying "OS" logic to ESXi, which doesn't work like that.

Reply
0 Kudos
TheCableGuy96
Contributor
Contributor

The two are separate.

 

I need multiple vmkernel interfaces so I can assign different VMs to different vLANs.

 

The LACP is just so there's plenty of load balancing through the 4 NICs.

Reply
0 Kudos
depping
Leadership
Leadership

you don't need VMkernel interfaces to assign VMs to different VLANs, you need different Portgroups, and assign VLAN IDs to those portgroups and connect the VMs NIC to that portgroup and then configure IP / Gateway / DNS within the VM itself.

For Load Balancing you could simply go with "route based on virtual port id", which provides load balancing based on the virtual nics connected, much easier to configured and typically just as efficient.

Reply
0 Kudos
TheCableGuy96
Contributor
Contributor

Ahh okay that makes sense now.

So on the switch the ports for all NICs will be tagged or untagged?

 

And will DHCP still work on different vLANS? For example, if I set a VM to vLAN10 will it pickup the DHCP server on that subnet automatically?

Reply
0 Kudos
depping
Leadership
Leadership

On the switch port for the NICs you typically only allow the VLANs which would be used by the system. (Management, vMotion, VM Portgroups etc) As described here: https://kb.vmware.com/s/article/1003806

Yes, DHCP still works.

Reply
0 Kudos
TheCableGuy96
Contributor
Contributor

perfect, thank you for all your help!!!

Reply
0 Kudos
jc296
Enthusiast
Enthusiast

BTW - you will need Enterprise licensing iand a vDistributed Switch f you want to utilise LACP or "Route Based upon Physical nic load".

depping is spot on: keep it simple. I would make all the physical switch ports as Trunk and the create 1 or more std vSwitches and add your physical adapters in esxi networking accordingly.  Add as many vSW Port-groups as you need for your individual VM VLANS/vnics.

I would set aside 1 physical adapter for your esxi mngt vmkernel port (with IP address) as a back door.

Reply
0 Kudos
TheCableGuy96
Contributor
Contributor

I think I've done it correctly....

I've got a virtual switch which has all 4 NICs added as "uplinks".

The NIC Teaming is set to "Route based upon originating Port ID"

It's showing as vmnic0 is Active and all the others are on standby.

I've added all the vLANs as port groups and on the switch added each NIC port to trunked on each vLAN. 

Does that sound right?

Reply
0 Kudos
a_p_
Leadership
Leadership

Except for the Active/Standby Configuration it looks ok.

Although Active/Standby will work, it will limit network traffic to the active vmnic.
I'd suggest that you set all vmnics to Active. ESXi will then assign the VMs to the 4 vmnics in a round-robin manner as the VMs are powered on.

André

Reply
0 Kudos