Solved: Re: Physical switch prerequisite

vmware3222 · ‎06-29-2016

Hi all,

For a simple lab design

that i need a switch that supports Layer 3 ??

because I heave only 2 simple switch one for management network and one for service network but they are physically separate

Thank you

admin · ‎06-30-2016

Is this related to your double VNIC post? If so can you simplify and retest with a single VNIC. If that fails I would do the following.

1. Use the Traceflow tool between the VMs.

2. Verify the VTEPs can reach each other and the MTU size ping ++netstack=vxlan -d -s 1572 DST_VTEP_IP - if this does not work you need to address the MTU and/or IP connectivity issue.

If both of those pass (traceflow will report firewall rules that may be blocking), I would recommend opening a support case.

View solution in original post

admin · ‎06-29-2016

You can run NSX on L2 or L3 topologies, we will want the switch interfaces to be configured with a 1600 byte MTU. Using a L3 switch design is nice because it shows the abstraction of the VM network topology form the physical network topology, but not required.

vmware3222 · ‎06-29-2016

thanks for your response

then i can use L2 switch but why my logical switch isn't working . VM1 can't ping VM2

knowing that i configured a VXLAN and one LS between 2 VMS 1 and 2 in different esxi physical hosts

admin · ‎06-30-2016

Is this related to your double VNIC post? If so can you simplify and retest with a single VNIC. If that fails I would do the following.

1. Use the Traceflow tool between the VMs.

2. Verify the VTEPs can reach each other and the MTU size ping ++netstack=vxlan -d -s 1572 DST_VTEP_IP - if this does not work you need to address the MTU and/or IP connectivity issue.

If both of those pass (traceflow will report firewall rules that may be blocking), I would recommend opening a support case.

vmware3222 · ‎07-02-2016

Thank you very much

cnrz · ‎07-02-2016

If VM1 can ping VM2 when both are on same ESX host, and not when different hosts, this may be related to an ARP Resolution or VTEP table problem. arp -a command on VM1 does not show the MAC Address of VM2 on its ARP cache.

Can both ESXi hosts are on the same Cluster or within the reach of the Transport Zone of the Logical switch and ESXi hosts can ping each other with MTU 1600 as previous post ?
What is the replication mode of the Logical Switch? If it is unicast, (which is the default) then there is no configuration needed on the Physical Switches in between. Controllers should handle the MAC to VTEP table formation on ESX1 and ESX2. When ESX1 learns the MAC Address of VM1, it tells Controller, and the Controller replicates this information to ESX2 where VM2 resides.For Hybrid and Unicast mode Controllers are involved, but for Multicast Controllers are not involved. The replication mode is defined during Transport Zone configuration (as in Link6),

Unicast and Hybrid these links may be helpful on general check of Controller and ESX host status:

About the overall status of the NSX Controllers and ESX hosts that function in the ARP Mechanism:
- The Controller Cluster Status should be "normal" (Network & Security > Installation > Management)
- Both ESX1 and ESX2 are prepared for VXLAN and VTEPs have been deployed showing "Enabled", and "ready" for Installation Status(Network & Security > Installation > Host Preparation) and Configuration Status(Network & Security > Installation > Logical Network Preparation). VXLAN should be "configured"
- The view of the Controllers and ESXi hosts should be in sync
If Controllers and ESX hosts are ok in general, then the Communication between the Controllers and ESX hosts may be cheked. For each VXLAN VNI, a master controller that owns this VNI is dedicated. Finding the master controller for this VNI and checking the communication with CLI commands on this controller may help, because this controller will provide the Synchronization of the ARP and MAC tables on the ESX hosts.
- The master controller for the VNI may be found by entering the following command on ESX1 as Link5:
  - [root@esx-01a:~] net-vdl2 -l

VXLAN Global States:

Control plane Out-Of-Sync: No --> Control Plane should not be Out-of-Sync

UDP port: 8472

VXLAN VDS: vds-site-a

VDS ID: c2 fb 2e 50 fb 09 5f 02-99 94 60 9f 68 ed 95 33

MTU: 1600

Segment ID: 192.168.130.0

Gateway IP: 192.168.130.1

Gateway MAC: 00:50:56:01:20:a6

Vmknic count: 1

VXLAN vmknic: vmk3

VDS port ID: 161

Switch port ID: 33554441

Endpoint ID: 0

VLAN ID: 0

IP: 192.168.130.52

Netmask: 255.255.255.0

Segment ID: 192.168.130.0

IP acquire timeout: 0

Multicast group count: 0

Network count: 4

VXLAN network: 5002

Multicast IP: N/A (headend replication)

Control plane: Enabled (multicast proxy,ARP proxy)

Controller: 192.168.110.32 (up)

MAC entry count: 1

ARP entry count: 0

Port count: 1

VXLAN network: 5001

Multicast IP: N/A (headend replication)

Control plane: Enabled (multicast proxy,ARP proxy)

Controller: 192.168.110.33 (up) --> This is the master controller for VNI5001

TCP Connection path between master controller and ESX1 and ESX2 may be checked as L2 networking troubleshooting part of Link1: (Both ESX1 and ESX2 should be connected to the master controller)
- # show control-cluster logical-switches connection-table 5001 Host-IP Port ID 192.168.110.51 17528 2 192.168.110.52 46026 3 192.168.210.56 42257 4 192.168.210.51 30969 5 192.168.210.57 12127 6 192.168.210.52 30280 7

The VTEPs Column should be at least 2 for ESX1 and ESX2 as Link3:

1

2

3

nsx-controller # show control-cluster logical-switches vni 5001

VNI Controller BUM-Replication ARP-Proxy Connections VTEPs

5001 192.168.110.202 Enabled Enabled 6 4

Both ESX1 and ESX2 VTEPs should have joined the VNI

1

2

3

4

5

6

nsx-controller # show control-cluster logical-switches vtep-table 5001 VNI      IP              Segment         MAC               Connection-ID
 5001     192.168.250.53  192.168.250.0   00:50:56:67:d9:91 1845
 5001     192.168.250.52  192.168.250.0   00:50:56:64:f4:25 1843
 5001     192.168.250.51  192.168.250.0   00:50:56:66:e2:ef 7
 5001     192.168.150.51  192.168.150.0   00:50:56:60:bc:e9 3

The ARP Table on the Controller should include both VM1 and VM2:
- nsx-controller # show control-cluster logical-switches arp-table 5001

VNI IP MAC Connection-ID

5001 172.16.10.12 00:50:56:ae:f8:6b 6

5001 172.16.10.10 00:50:56:ae:ab:9f 4

5001 172.16.10.11 00:50:56:ae:3e:3d 2

Both ESX hosts MAC to VTEP tables should include the MAC Address of VM1 and VM2.

For Multicast Mode how the forwarding VTEP tables are formed this links may be useful:

VXLAN Series – How VTEP Learns and Creates Forwarding Table – Part 5 - VMware vSphere Blog

vmware3222 · ‎07-02-2016

thank you cery much for your response

I have another problem

DVS has not a physical adapter because he is use by a standard switch and i can't migrate it to the VDS

All

Physical switch prerequisite