VMware Cloud Community
davidrough
Contributor
Contributor

Want to set up multiple VLANS between Switch and ESXi 6.7U3

Struggling to get VLANs to work.

I have a couple of different LAB environments I want to set up on a singlw ESXi server.

I have set up the following:

Screenshot 2020-01-26 at 01.13.41.png

Assigned uplinks 1,2,3 for use for LABS

Here are the physical NICs:

Screenshot 2020-01-26 at 01.14.42.png

I then setup a Port Group as follows:

Screenshot 2020-01-26 at 01.16.38.png

Screenshot 2020-01-26 at 01.17.30.png

On the Ubiquiti Controller I have set the following Network up for VLAN 4:

Screenshot 2020-01-26 at 01.19.54.png

When I assign port group LAB - ALPINE to the VM there is no network connection at all and no suggestion VLAN ID is being used. I can set a static IP of 10.0.0.1/24 but the VM is not physically bound to this network.

Screenshot 2020-01-26 at 01.21.20.png

vmnic1, vmnic2, vmnic3 are plugged in to a 24 Port switch with the 3 ports aggregated but not assigned to any particular network.

Any help on what to do here. I've used pfsense in the past to achieve something similar however would rather not use pfsense this time unless I need to.

Tags (2)
Reply
0 Kudos
17 Replies
sjesse
Leadership
Leadership

If this is a lab, simplify it, using one port on the switch. You mention aggregating a bunch of them, just try one check network connectivity and move on to more complicated setups.

Ardaneh
Enthusiast
Enthusiast

Hi,

I don't know about "Ubiquiti Controller", but when you create a PortGroup with VLAN tagging (for your scenario VLAN 4), you don't need to put VLAN tag on a guest os. so you should remove it from your Alpine Network Settings or you need to change PortGroup settings and put VLAN 0.

(Just a hint: If you are using a nested hosts LAB, you should change "MAC Address Changes" and "Forged Transmits" for your PortGroup to Accept.)

I hope this could help you.

Reply
0 Kudos
NathanosBlightc
Commander
Commander

If I understood well, you have a Ubiquiti controller (I think on the physical side of your network) and you need to make it connected to the virtual infrastructure. (or maybe vice versa) So if you want to make it possible on VLAN 4 for example, you need to do like following procedure:

1. Set the VLAN ID on corresponding Port Group that is defined inside the ESXi (like you did)

2. Configure the physical switch ports in the Trunk mode that uplinks of ESXi (vmnic ports) are connected to them. So they can trunking the tagged framed between virtual-physical worlds

3. Configure the VLAN ID in your Application like this Ubiquiti web admin console.

4. Check the connectivity between them again.

Please mark my comment as the Correct Answer if this solution resolved your problem
Reply
0 Kudos
davidrough
Contributor
Contributor

Thanks Amin Masoudifard

What you say seems to be make sense and tacked step 2 and set up the trunking however no success.

I cross posted on to VMware forums here as well with exact further steps taken:

Unable to get VLANs working | Ubiquiti Community

Any suggestions appreciated.

Reply
0 Kudos
davidrough
Contributor
Contributor

Ardaneh​ Thanks for the response. Just tried following some of your advice and unset the VLAN ID on the port group and changed "MAC Address Changes" and "Forged Transmits" for the PortGroup to Accept.

Made no difference.

I don't see how I can have different VLANs residing on the same ESXi server unless I tag each nic according to the VLAN it needs to be in by therefore having the PortGroup tagged.

Reply
0 Kudos
sjesse
Leadership
Leadership

Don't tag the nic in the vm, the portgroup vlan tag strips it, you need to only tag the ports on the physical switch.

Reply
0 Kudos
scott28tt
VMware Employee
VMware Employee

If you’re setting a VLAN on your port group you don‘t set anything on your vNICs or in your guest OS - this is no different from the physical world in a setup where you would set the VLAN on the switch port.


-------------------------------------------------------------------------------------------------------------------------------------------------------------

Although I am a VMware employee I contribute to VMware Communities voluntarily (ie. not in any official capacity)
VMware Training & Certification blog
Reply
0 Kudos
sjesse
Leadership
Leadership

(Just a hint: If you are using a nested hosts LAB, you should change "MAC Address Changes" and "Forged Transmits" for your PortGroup to Accept.)

This only is needed if you have a esxi vm in esxi, in normal situations it doesn't matter, what is meant by nested

Reply
0 Kudos
scott28tt
VMware Employee
VMware Employee

Moderator: Moved to vSphere vNetwork


-------------------------------------------------------------------------------------------------------------------------------------------------------------

Although I am a VMware employee I contribute to VMware Communities voluntarily (ie. not in any official capacity)
VMware Training & Certification blog
Reply
0 Kudos
davidrough
Contributor
Contributor

Thanks for replying Smiley Happy

So just to clarify I am not setting any VLAN from within the VM's OS and trying to set the VLAN on the Port Group. Here's how things are currently looking:

Screenshot 2020-01-26 at 19.49.17.png

Reply
0 Kudos
NathanosBlightc
Commander
Commander

Did you configure all the pNICs in the aggregated state?! If it's true, so you need to setup the IP Hash load balancing.

Can you check it step by step simply please? Setup one uplink (vmnic) for the vSwitch, remove the VLAN tag and test the connectivity. If everything works correctly, then configure the VLAN ID in both of virtual & physical networks and at last add the other uplinks if you need ...

Please mark my comment as the Correct Answer if this solution resolved your problem
Reply
0 Kudos
Ardaneh
Enthusiast
Enthusiast

davidrough​ As you know, there are 3 different network design in vSphere:

VST (Virtual Switch Tagging)

EST (External Switch Tagging)

VGT (Virtual Guest Tagging)

----------------------------------------

If you want to make your scenario according to VST you should have at least one physical port in trunk mode as your uplink, some PortGroups with VLAN tagging (for example VLAN 4) and some guest os without any tagging inside them.

For EST you must have one physical port in access mode with a VLAN tag as your uplink, one PortGroup with no tag (VLAN 0), guest os with no tagging.

For VGT you should have at least one physical port in trunk mode, one PortGroup with VLAN 4095 (it is trunk mode of a PortGroup), some VMs that have VLAN tagging inside the guest os

So you should decide to have one of these and as @sjesse told you, make it simple.

Hope this could help you

Reply
0 Kudos
davidrough
Contributor
Contributor

Amin Masoudifard tried to take things back to first principals here.

Firstly, when set up without any VLANS and a single pNIC as shown:

Screenshot 2020-01-27 at 13.38.16.png

then there is network connectivity from VM.

On the switch side the pNIC "vmnic3" is connected to Switch Port 4 with an open ALL profile.

My expectation is that I can then tag the Port with VLAN as shown:

Screenshot 2020-01-27 at 13.47.12.png

And change switch port 4 to trunk for 1002 vlan. However no network connectivity then exists for VM.

Reply
0 Kudos
sjesse
Leadership
Leadership

"Switch Port 4 with an open ALL profile."

Can you try just adding just the vlans you want.. Removing the tag works because there is a native vlan set, there may be somethign with that. ESXi wise your doing it right by tagging, its something your doing on the switch side. You may want to post in the ubiquiti forums, and maybe even try a machine to test if the vlans are working. One thing I've done is used wireshark to capture the vlans to make sure they are there.

Reply
0 Kudos
davidrough
Contributor
Contributor

sjesse​  - that was a worthwhile test - exactly the same behaviour shown by my laptop when plugging in to one of the switch ports. As you say definitely looks to be a UniFi configuration issue or quirk. I have actually cross posted already there but faitly muted response - hopefully only due it being the start of the new working week.

Thanks for confirming all looks ok from an ESXi side.

Reply
0 Kudos
sjesse
Leadership
Leadership

On the esxi side there really isn't anything you can screw up if your just doing simple vlan tags on the port groups with the defaults set. Its when you try doing lacp or any type of bonding is where it gets more complicated. In your images I see the tag being listed in the port group. You could try taking an adapter off what you have and add it to a distrubted switch if you have the license, there is a health check you can use that will throw alarms if the switch port isn't configured correctly.

vSphere Distributed Switch Health Check

Reply
0 Kudos
NathanosBlightc
Commander
Commander

And change switch port 4 to trunk for 1002 vlan. However no network connectivity then exists for VM.

How did you test it? are you sure about the configuration of your physical switch and its related ports? attached images seems to be ok but I think you should monitor / debug the states of frames on pSwitch, if it's possible.

Please mark my comment as the Correct Answer if this solution resolved your problem
Reply
0 Kudos