Skip navigation

Kevin Barrass's Personal Blog

November 21, 2012 Previous day Next day

My 1st lab I'm blogging shows how you can setup a simple virtual lab to create VXLAN's on then use wireshark to view the VXLAN traffic and peek inside the VXLAN traffic by decoding the packets.

 

On my lab I have a single VM for vCenter 5.1, 2 ESXi 5.1 hosts and a shared storage virtual appliance.

The vCenter and each host as well as storage appliance has a vNic in the VMNet1(host only) Workstation network. Each host also has a vNic in the VMNet4(host only) Workstation network that is DHCP enabled for VXLAN transport. My lasptop has the IP address 192.168.142.1 and has vSphere 5.1 client and Wireshark installed. I have created a single DC with one cluster that I have deployed the vShield Manager "VSM" into. I have also created two VM's for testing. I've used OpenBSD as they have a small footprint and can run various services for testing network traffic.

 

Please note figure 1.0 in Part two of this blog post on how the lab was built on VMware Workstation.

 

Once the VSM has booted up and had its IP, Subnet, default gateway and DNS configure from the VSM CLI I registered it with my vCenter Server.

 

register VSM.png

 

Once the VSM plugin has registered with vCenter Client you navigate to the DC and click the "Network Virtualization" tab. Then click on preperation and edit for the segment ID. Enter some test values in this case a multicast group range of 224.1.1.50-224.1.1.60 and segment ID pool of 5000-5100.

 

create segment ID.png

 

Once the segment ID is added click on connectivity>edit and select the cluster that you want to deploy VXLAN over. This assumes I have already created the vNetwork Distributed switch version 5.1 accross all hosts in the cluster.

 

select cluster.png

 

At this point VXLAN will be deployed onto each host in the cluster, a new VMkernel interface for VXLAN will be created on each hosts in my case vmk1. The vmk1 interfaces will be assigned an IP address using DHCP from the VMware workstation VMNet4 IP address pool.

 

At this point the Cluster is prepared for VXLAN and should have a status of Normal.

 

prepared.png

 

We now create the Scope for the VXLAN's in this case a single cluster. Select add then give the scope a name, descipton and select the cluster(s) for this scope.

 

add scope.png

 

Great, VXLAN is now deployed and ready for us to create our 1st VXLAN. Each VXLAN wil be mapped to a multicast group, if we assigned just two multicast addresses to our range and created 3 VXLANs you would have 2 virtual wires mapped to a single multicast group. This is not a problem as far as I know but you would need to be mindful of mutliple VXLANs broadcast domains being on the same multicast tree and would make pruning each broadcast domain back harder due to a single multicast group for multiple VXLAN's so might need carefull thought and planning.

 

So now to create our 1st VXLAN, simply click on Networks then the green cross for add. Give the VXLAN a name, description and scope which in this case has to be scope-01. Then click ok. The VXLAN is then created and an associated Portgroup is created. The VXLAN is given a VNI "VXLAN Network Identifier" of 5000 and mapped to the multicast group 224.1.1.50.

 

create virtual wire.png

Now we will add each of our two OpenBSD VM's into the VXLAN,

 

assign nic to vxlan.png

 

Now run a continous ping from VM01 "172.16.1.2" to VM02 "172.16.1.3". Then open the packet capture tool Wireshark and choose the interface for VMNet4 which is the virtual network on Workstation we are transporting the VXLAN traffic over.

 

Now start the packet capture whilst the ping session is still running from VM01 to VM02. It is important to make sure that VM01 and VM02 are NOT running on the same host or you will be sat like I was wondering why I didnt see any traffic. The reason for this is we want to see the VXLAN traffic going between hosts over the VXLAN transport network.

When the packet capture starts you should see the VXLAN UDP traffic with a source IP of one ESXi host and a destination of the other ESXi host, with a destination UDP port of 8472. Looking at the packet capture we can only see that it is a VXLAN packet, and that it has a payload. The important thing here is that the traffic is unicast what we havent done is captured the traffic that transported the ARP request using multicast, we will do this later in this blog.

 

packet capture none decoded.png

What we can now do is use the VXLAN decoder in Wireshark to look agt the VXLAN header and the contents of the packet. To do this select a packet and right click then select "decode as" and choose a transport protocol of VXLAN.

 

packet capture set decode to vxlan.png

 

Now we should see the contents of the VXLAN payload i.e. the original packet which in this case is the ICMP echo-request and echo-reply. You can also view the VXLAN header showing the VNI of 5000 which is the VXLAN Network Identifier which was assigned to our VXLAN.

 

packet capture decoded.png

 

As we missed the original ARP request sent by VM01 for VM02's MAC address from VM01 we will run a continous ping to an IP address not in use 172.16.1.4. this will keep VM01 sending an ARP request out as a broadcast. VXLAN will then encapsulate this packet into a multicast packet onto the multicast group 224.1.1.50 and this will be sent to all hosts that would have joined this group through IGMP. In our case we are not using PIM or IGMP snooping. As you can see in the none-decoded traffic flow below the source remains the hosts IP address but the destination is now the multicast group for this and possibly other VXLAN's.

 

packet capture ARP none decoded.png

 

In part 2 of this blog post we will decode the VXLAN multicast traffic, and show what information you can view on the ESXi hosts via an SSH session.

 

Kevin Barrass

Now that we have captured an ARP request that has been encapsulated in multicast by VXLAN we will now use the same option in Wireshark to decode the VXLAN packet. This shows the same VXLAN header and the encapsulated ARP request. The host that received this encapsulated ARP request would add the MAC address of the requesting VM in this case VM01 to its VXLAN mapping.

 

packet capture ARP decoded.png

 

If we re-run the constant ping from VM01 to VM02 we can log into each ESXi host and view its VXLAN mapping of VM MAC addresses. In this case we enable SSH on each host, log into the host and run the command:

 

esxcli network vswitch dvs vmware vxlan network mapping list --vds-name=dvSwitch --vxlan-id=5000

 

Where dvSwitch is our vNetwork Distributed switch name and 5000 is the VNI of our VXLAN. This will then show the mappings for our two VM's VM01 and VM02. The inner MAC is the MAC address of the VM and both the outer MAC and outer IP are that of the recipient ESXi hosts vmk1 VMKernel interface. If we were doing VXLAN over a routed network the out IP would still be that of the recipient ESXi host but the outer MAC would be that of the next hop PIM router. I will cover this in a later blog post on VXLAN over a pure layer 3 network using PIM Sparse Mode.

 

VXLAN mappings for VNI 5000

 

show vxlan mapping.png

 

As you can see the destination for VM01 on host ESXi1 matches the MAC and IP address of that hosts VMKernel interface vmk1.

vxlan vmk1 interface.png

Another thing you can do is to view from the ESXi cli the VNI to multicast group mapping using the command:

 

esxcli network vswitch dvs vmware vxlan network list --vds-name=dvSwitch --vxlan-id=5000

 

There are more combinations of "esxcli network vswitch dvs vmware vxlan" command than Ive played with or could cover in this blog post.

show vxlans and mcast groups via ssh.png

Thats the end of this blog post, as this is my 1st lab blog post I really appreciate any constructive feedback as I hope to add more blogs posts as I lab things up at home depending on free home time

 

VXLAN Lab setup as per Figure 1.0 below:

Computer with 16GB RAM, quad core i5 CPU and Solid state disk.

VMware Workstation verson 8

One VM for vCenter, one VM for shared storage and two ESXi VMs with virtualised VT-x enabled. vShield Manager as a VM on the virtual ESXi hosts.

 

Figure1.0

vxlan blog 1.gif

 

I hope to create a blog post for each of the below in the coming months:

 

          VXLAN over Layer 3 using PIM-SM

          vCNS Edge SSL VPN.

          vCNS Edge firewall and CLI/debugging

          and hopefully more.....

 

Thanks for reading.

Kevin Barrass

kjbarrass Novice

Welcome to my Blog

Posted by kjbarrass Nov 21, 2012

Only just set up this blog but hope over the coming weeks and months to upload how to build VXLAN/vCNS and vCloud labs on VMware Workstation. I will also put up examples of how to do packet captures to get a better understanding of how VXLAN works.

 

Kev Barrass