1 2 Previous Next 24 Replies Latest reply on Apr 23, 2020 6:55 AM by TwincaAdmin

    ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings

    TwincaAdmin Novice

      Hi I'm kind of new in this ESXi.... on this level.

       

      ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings. Why is this the case?

      I Have 2 Switches on an ESXi 6.7.U3 With 4 physical NIC's.

      There is assigned 2 NIC's to each Switch.

      Will Load balancing work if there are not active both NIC's?

      And why does it not work with both NIC's Active in the "Failover order"?

       

      By the way... This Esxi Host is connected to an Ubiquiti EdgeSwitch ES-48-Lite

      interface 0/29

      description 'Server-2 NIC-1 VLAN 1 -10-15'

      vlan participation exclude 20,30,100

      vlan participation include 10,15

      vlan tagging 10,15

      exit

       

      interface 0/30

      description 'Server-2 NIC-2 VLAN 1 -10-15'

      vlan participation exclude 20,30,100

      vlan participation include 10,15

      vlan tagging 10,15

      exit

       

      interface 0/31

      description 'Server-2 NIC-3 VLAN 1 -10-15'

      vlan participation exclude 20,30,100

      vlan participation include 10,15

      vlan tagging 10,15

      exit

       

      interface 0/32

      description 'Server-2 NIC-4 VLAN 1-10-15'

      vlan participation exclude 20,30,100

      vlan participation include 10,15

      vlan tagging 10,15

      exit

        • 1. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
          a.p. Guru
          User ModeratorsvExpertCommunity Warriors

          Please explain "ESXi Standard switch blocks the network".

          It's not uncommon to have all vmnics active on the vSwitch as well as on the port groups. In fact that's actually the default configuration for most environments.

          Can you confirm that the physical switch ports used by the vmnics are all configured as tagged (802.1Q) ports.


          André

          • 2. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
            NicolasAlauzet Hot Shot

            To have "Load Balancing" you must have both pNICs as Active.

             

            If you change the settings and there is an issue with traffic. Please check that the configuration for the pNICs is the same (if using trunk, same vlans should be allowed)

             

            Can you provide the SW configuration for all four (4) pNICs?

            • 3. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
              TwincaAdmin Novice

              The machines on the network loses connectivity to the internet, both the ones on the ESXi and the physical computer... I'm no sure but I could be  broadcast storm... But i'm not sure... The network stops responding. As soon as I deactivate the one of the two nic's in the "Failover order" (The Last one) machines begins to appear as online again in e.g. Teamviewer.

               

              Please see the new attached pictures in the first post...

               

              All 4 NIC's on the server is Truk ports with VLAN 1 Untagged, VLAN 10 and 15 Tagged.

              • 4. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
                a.p. Guru
                Community WarriorsvExpertUser Moderators

                Does this happen on both vSwitches, or only on vSwitch1 with the port group for the default VLAN?

                You don't have these ports in any channel configuration, do you?

                 

                André

                • 5. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
                  TwincaAdmin Novice

                  Please see the trade... I think The answer for you question is answered, in the answer to a.p.

                  If not... Please let me know.

                  • 6. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
                    TwincaAdmin Novice

                    interface 0/29

                    description 'Server-2 NIC-1 VLAN 1 -10-15'

                    vlan participation exclude 20,30,100

                    vlan participation include 10,15

                    vlan tagging 10,15

                    exit

                     

                    interface 0/30

                    description 'Server-2 NIC-2 VLAN 1 -10-15'

                    vlan participation exclude 20,30,100

                    vlan participation include 10,15

                    vlan tagging 10,15

                    exit

                     

                    interface 0/31

                    description 'Server-2 NIC-3 VLAN 1 -10-15'

                    vlan participation exclude 20,30,100

                    vlan participation include 10,15

                    vlan tagging 10,15

                    exit

                     

                    interface 0/32

                    description 'Server-2 NIC-4 VLAN 1-10-15'

                    vlan participation exclude 20,30,100

                    vlan participation include 10,15

                    vlan tagging 10,15

                    exit

                    • 7. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
                      NicolasAlauzet Hot Shot

                      All seems to be ok. But if not working you can do this test to try to isolate de problem.

                       

                      Create a new portgroup. Put NIC 3 as ACTIVE, NIC1 unnused adapter.

                      Put a VM in that new portgroup, test connectivity.

                       

                      Repeate the same, but changing NIC3 from ACTIVE to unnused and biceversa.

                       

                      The objective is to isolate traffic for each adapter and see exactly whats not working

                      • 8. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
                        TwincaAdmin Novice

                        Hi Andre

                         

                        IT seems like it is only on the switch 1... Because I can access bother the esxi and the center server... When this happens.... So I think you are rigth it is somehow related to the untagged VLAN1.

                         

                        But I've tried to put VLAN1 in the switch as tagged... But I did not manage to make it work... Should I be able to tag VLAN1 on the ubiquitous switch and then put the old production network to VLAN1.... Should that work? And would it take a reboot of the esxi, to make itwork, if yes?

                         

                        (I did not reboot last I tried)

                         

                        Jens ole

                        • 9. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
                          TwincaAdmin Novice

                          Hi Nicolas...


                          That's an option... I'll try it if I don't manage to make it work, but to be honest, I don't think it is the adapter it self.. more likely it is related to the untagged VLAN1... I was wondering when I took the configuration from the switch CLI .... I did not see the VLAN1 on the ports for the server, even then it is not tagged... But shouldn't it appears in the configuration on the port anyway, as VLAN1 untagged?

                          • 10. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
                            NicolasAlauzet Hot Shot

                            In other switches you need to set as native vlan in the trunk to have the vlan 1 included. And propagate it as untagged.

                             

                            In Cisco sw for example is: switchport trunk native vlan

                            I went through edgemax documentation and wasn't able to find if there is a way to do that.  Maybe you can try adding vlan 1 to your trunk, and first test without changes in esxi (vlan 0) and if doesn't work, move it to vlan 1 (tagged) maybe the sw will move traffic anyways.

                            • 11. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
                              TwincaAdmin Novice

                              It is working with only one active nic... And if you see the previous attached pictures from the UI in the ubiquiti switch... You will see the the VLAN1 is on the trunk as untagged. ... So I really don't get why it does not work with 2 active NIC's.... I have suspicion that is behaving bit like if you put at patch cable as a loop between 2 ports in a normal physical switch. (Which you of cause never should do)

                              • 12. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
                                TwincaAdmin Novice

                                I've now tagged the Default VLAN1. .... And that works With one Active adapter.

                                I'll try out this config... And If it stabile for some time, I'll try to add one more active NIC, and if it crashes again.

                                Until now the problem always starts every time I add both NIC's as active in the "Failover order"

                                 

                                 

                                • 13. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
                                  a.p. Guru
                                  vExpertCommunity WarriorsUser Moderators

                                  But I've tried to put VLAN1 in the switch as tagged...

                                  There should actually be no reason to do this. Using the default/native VLAN is also not unusual, and just requires that the virtual port group doesn't have a VLAN-ID configured.

                                  The issue is indeed strange, because ESXi itself doesn't create loops. The default setting "Route based on originating port ID" works in a round robin manner, where one of the vmnics is assigned to a VM's virtual network adapter at power on. Network changes btw. do not require a host reboot.

                                  Do you have VMs with multiple virtual network adapters, which are assigned to different port groups, e.g. a virtual router?

                                   

                                  Would it be possible to provide (attach) the complete physical switch configuration (s.th. like show running-config) to see whether it contains something that could be related?

                                   

                                  André

                                  • 14. Re: ESXi Standard switch blocks the network if all Nic's is marked as active in "Failover order" In switch settings
                                    TwincaAdmin Novice

                                    I've to or more machines on the physical network there is member of several VLAN...

                                    But All VM's i only member of one VLAN .

                                    Config:

                                     

                                    login as: **********

                                    TwAdmin@192.168.10.2's password:

                                      _____    _

                                    | ____|__| | __ _  ___          (c) 2010-2018

                                    |  _| / _  |/ _  |/ _ \         Ubiquiti Networks, Inc.

                                    | |__| (_| | (_| |  __/

                                    |_____\__._|\__. |\___|         https://www.ubnt.com

                                                 |___/

                                     

                                     

                                    Welcome to EdgeSwitch

                                     

                                     

                                    By logging in, accessing or using the Ubiquiti (UBNT) product, you

                                    acknowledge that you have read and understood the Ubiquiti Licence

                                    Agreement (available in the GUI and at https://www.ubnt.com/eula/)

                                    and agree to be bound by its terms.

                                     

                                     

                                     

                                     

                                    (ES-48-Lite) >enable

                                    Password:************

                                     

                                     

                                    (ES-48-Lite) #show running-config

                                     

                                     

                                    !Current Configuration:

                                    !

                                    !System Description "EdgeSwitch 48-Port Lite, 1.9.0-lite, Linux 3.6.5-03329b4a, 1.1.0.5102011"

                                    !System Software Version "1.9.0-lite"

                                    !System Up Time          "1 days 0 hrs 31 mins 43 secs"

                                    !Additional Packages     QOS,IPv6 Management,Routing

                                    !Current SNTP Synchronized Time: Mar 29 10:22:11 2020 UTC

                                    !

                                    hostname "ES-48-Lite"

                                    network protocol none

                                    network parms 192.168.10.2 255.255.255.0 192.168.10.1

                                    vlan database

                                    vlan 10,15,20,30,100

                                    vlan name 10 "VLAN10_Management"

                                    vlan name 15 "VLAN15_Local_LAN"

                                    vlan name 20 "VLAN20_Guest"

                                    vlan name 30 "VLAN30_IoT"

                                    vlan name 100 "VLAN100_Test"

                                    exit

                                     

                                    network mgmt_vlan 10

                                    ip http session soft-timeout 60

                                    ip http secure-session soft-timeout 60

                                    sshcon timeout 60

                                    configure

                                    ip name server 8.8.8.8 8.8.4.4

                                    username "**************" password ***************** Removed ****************************************************************************************************************************** Removed *********** level 15 encrypted

                                    username "*************" password ***************** Removed ****************************************************************************************************************************** Removed ***********  level 15 encrypted

                                    line console

                                    exit

                                     

                                    line telnet

                                    exit

                                     

                                    line ssh

                                    exit

                                     

                                    snmp-server sysname "ES-48-Lite"

                                    snmp-server location "Twinca"

                                    snmp-server contact "JOM"

                                    !

                                     

                                    interface 0/1

                                    description 'Trunk To ER-6P'

                                    switchport mode trunk

                                    vlan participation auto 1

                                    exit

                                     

                                    interface 0/2

                                    vlan participation exclude 1,10,15,20,30,100

                                    exit

                                     

                                    interface 0/3

                                    description 'MeetingRoomAP (Trunk)'

                                    vlan participation exclude 10,15,30

                                    vlan participation include 20,100

                                    vlan tagging 20,100

                                    exit

                                     

                                    interface 0/4

                                    description 'KontorAP (Trunk)'

                                    vlan participation exclude 10,15,30

                                    vlan participation include 20,100

                                    vlan tagging 20,100

                                    exit

                                     

                                    interface 0/5

                                    vlan participation exclude 1,10,15,20,30,100

                                    exit

                                     

                                    interface 0/6

                                    vlan participation exclude 1,10,15,20,30,100

                                    exit

                                     

                                    interface 0/7

                                    vlan participation exclude 1,10,15,20,30,100

                                    exit

                                     

                                    interface 0/8

                                    vlan participation exclude 1,10,15,20,30,100

                                    exit

                                     

                                    interface 0/9

                                    vlan participation exclude 1,10,15,20,30,100

                                    exit

                                     

                                    interface 0/10

                                    vlan participation exclude 1,10,15,20,30,100

                                    exit

                                     

                                    interface 0/11

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/12

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/13

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/14

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/15

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/16

                                    description 'Konstruktion-02'

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/17

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/18

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/19

                                    description 'Twinca-PC01'

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/20

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                     

                                    interface 0/21

                                    vlan participation exclude 10,15,20,30,100

                                    vlan tagging 30

                                    exit

                                     

                                    interface 0/22

                                    vlan participation exclude 10,15,20,30,100

                                    vlan tagging 30

                                    exit

                                     

                                    interface 0/23

                                    description 'Kontor-PC2'

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/24

                                    description 'Lager-PC01'

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/25

                                    description 'APC UPS-1 SmartConnect VLAN30'

                                    vlan pvid 30

                                    vlan participation exclude 1,10,15,20,100

                                    vlan participation include 30

                                    exit

                                     

                                    interface 0/26

                                    description 'Server-1 NIC1 VLAN 1 -10-15'

                                    vlan participation exclude 20,30,100

                                    vlan participation include 10,15

                                    vlan tagging 10,15

                                    exit

                                     

                                    interface 0/27

                                    description 'APC UPS-2 SmartConnect VLAN30'

                                    vlan pvid 30

                                    vlan participation exclude 1,10,15,20,100

                                    vlan participation include 30

                                    exit

                                     

                                     

                                    interface 0/28

                                    description 'Server-1 NIC2 VLAN 1 -10-15'

                                    vlan participation exclude 20,30,100

                                    vlan participation include 10,15

                                    vlan tagging 10,15

                                    exit

                                     

                                    interface 0/29

                                    description 'Server-2 NIC-1 VLAN 1 -10-15'

                                    vlan participation exclude 20,30,100

                                    vlan participation include 10,15

                                    vlan tagging 1,10,15

                                    exit

                                     

                                    interface 0/30

                                    description 'Server-2 NIC-2 VLAN 1 -10-15'

                                    vlan participation exclude 20,30,100

                                    vlan participation include 10,15

                                    vlan tagging 1,10,15

                                    exit

                                     

                                    interface 0/31

                                    description 'Server-2 NIC-3 VLAN 1 -10-15'

                                    vlan participation exclude 20,30,100

                                    vlan participation include 10,15

                                    vlan tagging 1,10,15

                                    exit

                                     

                                    interface 0/32

                                    description 'Server-2 NIC-4 VLAN 1-10-15'

                                    vlan participation exclude 20,30,100

                                    vlan participation include 10,15

                                    vlan tagging 1,10,15

                                    exit

                                     

                                    interface 0/33

                                    description 'NAS01 - LAN1'

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/34

                                    description 'NAS01 - LAN2'

                                    vlan pvid 15

                                    vlan participation exclude 1,10,20,30,100

                                    vlan participation include 15

                                    vlan tagging 15

                                    exit

                                     

                                    interface 0/35

                                    description 'NAS02 - LAN1'

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/36

                                    description 'NAS02 - LAN2'

                                    vlan pvid 15

                                    vlan participation exclude 1,10,20,30,100

                                    vlan participation include 15

                                    vlan tagging 15

                                    exit

                                     

                                    interface 0/37

                                    description 'Zebra ZM400'

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/38

                                    description 'Fronti Alarm VLAN10'

                                    vlan pvid 10

                                    vlan participation exclude 1,15,20,30,100

                                    vlan participation include 10

                                    vlan tagging 15

                                    exit

                                     

                                    interface 0/39

                                    description 'KontorKopiPrint'

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/40

                                    description 'IBM UPS AP9630 VLAN10'

                                    vlan pvid 10

                                    vlan participation exclude 1,15,20,30,100

                                    vlan participation include 10

                                    exit

                                     

                                    interface 0/41

                                    description 'Server- 2 RMM4 Lite BMC VLAN10'

                                    vlan pvid 10

                                    vlan participation exclude 15,20,30,100

                                    vlan participation include 10

                                    vlan tagging 1,10,15

                                    exit

                                     

                                     

                                    interface 0/42

                                    description 'Server-1 RMM3 VLAN10'

                                    vlan pvid 10

                                    vlan participation exclude 1,15,20,30,100

                                    vlan participation include 10

                                    vlan tagging 1,15

                                    exit

                                     

                                    interface 0/43

                                    description 'APC UPS-1 AP9630 VLAN10'

                                    vlan pvid 10

                                    vlan participation exclude 1,15,20,30,100

                                    vlan participation include 10

                                    vlan tagging 15

                                    exit

                                     

                                    interface 0/44

                                    description 'APC UPS-2 AP9630 VLAN10'

                                    vlan pvid 10

                                    vlan participation exclude 1,15,20,30,100

                                    vlan participation include 10

                                    vlan tagging 15

                                    exit

                                     

                                    interface 0/45

                                    vlan participation exclude 1,10,15,20,30,100

                                    exit

                                     

                                    interface 0/46

                                    description 'UniFi CloudKey'

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/47

                                    vlan participation exclude 1,10,15,20,30,100

                                    exit

                                     

                                    interface 0/48

                                    description 'Trunk to ES-24-Lite'

                                    switchport mode trunk

                                    vlan participation auto 1

                                    exit

                                     

                                    interface 0/49

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/50

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/51

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface 0/52

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface lag 1

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface lag 2

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface lag 3

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface lag 4

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                    interface lag 5

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    interface lag 6

                                    vlan participation exclude 10,15,20,30,100

                                    exit

                                     

                                    service ubnt-discovery-active-scan

                                    exit

                                     

                                    (ES-48-Lite) #

                                     

                                    1 2 Previous Next