Virtual switch over multiple esxi

[oxment]

Hello guys,

I need to make a virtual network over multiple hypervisor. I know it is possible to add each NIC on one Vnet but it is just for one hypervisor. Also I find a open vswitch application which works over multiple hypervisor, but it doesn't have support on Vmware. there is just one http://nicira.com which works over multiple esx, but it is not free.

do you have any suggestion or any solution for this problem?

[a_p_]

Depending on your licenses you could use VMware's vDS (vNetwork Distributed Switch).

André

[rickardnobel]

oxment wrote:

I need to make a virtual network over multiple hypervisor.

Could you clarify some more what you want to accomplish?

If there is just that a LAN (or VLAN) should be connected to Virtual Machines on several ESXi hosts than that would not be a problem.

[oxment]

yes it is more like a Lan.

For example I have 3 VMs which are located at diffrent Hypervisors and each VM has one Network Interface with the follwing IP addresses:

192.168.0.1

192.168.0.2

192.168.0.3

I would like to make these computers available to each other. But at the same time, I may need to use second layer network features. So from user point of view, it should be like a physical switch which connects these 3 machines togethere.

P.S. I don't have access to vDS.

[rickardnobel]

oxment wrote:

For example I have 3 VMs which are located at diffrent Hypervisors and each VM has one Network Interface with the follwing IP addresses:

192.168.0.1

192.168.0.2

192.168.0.3

I would like to make these computers available to each other. But at the same time, I may need to use second layer network features. So from user point of view, it should be like a physical switch which connects these 3 machines togethere.

This will not be a problem at all, it is all available from the standard vSwitch.

What do you mean with "second layer network features", could you give some example?

[oxment]

For example, may I use arp scaning ?!

And you suggest me to use Vlan? If I use VLan, may I communicate with other hypervisors?!

[rickardnobel]

oxment wrote:

And you suggest me to use Vlan? If I use VLan, may I communicate with other hypervisors?!

If you just want to connect a single Layer 2 network to your three ESXi hosts and their guests you could do without VLANs, but if you have a physical switch that supports it would be very recommended.

oxment wrote:

For example, may I use arp scaning ?!

Anything that you could send on a ordinary network could be sent to your VMs. So if you have some ARP tool today it will be available to use also on the virtual network.

Just for interest, what do you mean specificly with ARP scaning?

[oxment]

Just for interest, what do you mean specificly with ARP scaning?

The ability of scaning computers on the same network segment by using arp ping. (sometimes firewall doesn't drop ICMP packets so you can't find all computers by using IP ping)

If you just want to connect a single Layer 2 network to your three ESXi hosts and their guests you could do without VLANs, but if you have a physical switch that supports it would be very recommended.

I belibe my switchs are supporting VLans but for installing a Vlan among multiple ESX, I need to hav VDs?

And the other thing is that, I do need to configure the network amon these three VMs dynamically and I may have more than one template( 3VM with the mentioned configuration) so I should be able to run all these templates at the same time without network conflict.

[rickardnobel]

oxment wrote:

The ability of scaning computers on the same network segment by using arp ping. (sometimes firewall doesn't drop ICMP packets so you can't find all computers by using IP ping)

As long as the tool just sends normal ARP Requests then there should be no problem using that with ESXi.

I belibe my switchs are supporting VLans but for installing a Vlan among multiple ESX, I need to hav VDs?

No, it is available out of the box on standard vSwitches. Just check your physical switch and how to configure up VLAN tagging against the host ports. The ESXi part is very easy.

And the other thing is that, I do need to configure the network amon these three VMs dynamically and I may have more than one template (3 VM with the mentioned configuration) so I should be able to run all these templates at the same time without network conflict.

What do you mean with a template here? A VMware template VM or something else? A "template" from vSphere point of view could not run at all, it is just a template to create other unique VMs from.

[oxment]

No, it is available out of the box on standard vSwitches. Just check your physical switch and how to configure up VLAN tagging against the host ports. The ESXi part is very easy.

I will check my physical switch but can you guide me through ESXI setup.

What do you mean with a template here? A VMware template VM or something else? A "template" from vSphere point of view could not run at all, it is just a template to create other unique VMs from.

Template is from my point of view

Template: a group of VMs + network configuration among them( for example I may have a template which have three VMs with the mentioned configuration and then someone come two system and give multiple request for this template) system must be able to run these VMs and configure network requirments of the template.

Actually I am working on a open source project of Apache which allow you to reserve an load VMs on diffrent hypervisors but it doesn't have support for network among these virtual machines, so I am planing to do this part. But I am little bit confused about ESX part.

Thank you very much for your help and guidance.

[rickardnobel]

oxment wrote:

No, it is available out of the box on standard vSwitches. Just check your physical switch and how to configure up VLAN tagging against the host ports. The ESXi part is very easy.

I will check my physical switch but can you guide me through ESXI setup.

Do you have a ESXi installed and ready? Once you have configured the physical switch with VLAN tagging we can look at the ESXi part which is not much work.

[oxment]

yes my switch supports VLAN but how can I tag my NICs on VM?

I've read the following document but I still have problem :

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&externalId=1004252

I don't know where should I put my vlan tag.

[rickardnobel]

oxment wrote:

yes my switch supports VLAN but how can I tag my NICs on VM?

I've read the following document but I still have problem :

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&externalId=1004252

I don't know where should I put my vlan tag.

Unless you have some very special need you should not do the VLAN tagging inside the VM, but on the virtual switch portgroup.

Just edit the settings on the VM portgroup and enter the correct VLAN id. This is the equalient for on a physical switch make the VM ports in "access mode", that is: untagged, and the uplink (vmnic) as "trunk", i.e. tagged.

[oxment]

Since I need to be able to communicate to other hypervisors so I need to use open Vswitch for managing packets. For using open vswitch on ESXI server I need to make an interface for open Vswitch and add this NIC at the same port group of the other VMs. Then it is possible to tag the packet on the open Vswitch, but problem of this solution is the limitation of number of NICs on each VM. So I can't add a lot of NICs to my OpenVswitch.

Then I decided to make two NICs for Open vSwitch (one for external communication and one for internal) then put all the NICs of all VMs and internal open Vwitch NIC at the same port group. Because of this, I need to add VLan tags sometimes before port group. Is it possible to define VLan in VMX file?

[rickardnobel]

oxment wrote:

Since I need to be able to communicate to other hypervisors so I need to use open Vswitch for managing packets.

Could you clarify again what you mean with communicate to other hypervisors? If there is network connectivity with VLAN tagging and similar then that is available out of the box with standard vSwitches on ESXi and there is no need to bring in any 3rd party code like the Open vSwitch.

[oxment]

For example I have a network of 10 connected switchs and each of them are connected to 10 hypervisor( which can be VMware, Xen and ...). So as far as I understood, you are telling me to add the vlan on each port group and then ESX and Physical switch will forward this package to all other hypervisors but I think it will affect on my network traffic?

[rickardnobel]

oxment wrote:

For example I have a network of 10 connected switchs and each of them are connected to 10 hypervisor( which can be VMware, Xen and ...).

You could have a physical network with 10 switches (with different VLANs) and they could be connected to any number of Hypervisors. If your goal is to make VMs on this hypervisors able to communicate with each other then it should not be any problem. A small note is that I have not worked with Xen, but assume that they have VLAN support.

So as far as I understood, you are telling me to add the vlan on each port group and then ESX and Physical switch will forward this package to all other hypervisors

Depending what you mean with "forward to all other hypervisors". A VM on one host that sends a frames destinated to another VM on another host - then yes, this frame will be delivered over the physical network up to the correct VM. It will not go to all other hypervisors, just to the one with the specific VM.

[oxment]

sorry I am not expret on networking but I have a question here.

For example if I have one VM1 on ESX1 and VM10 on ESX10.

I assign VNIC1 of VM1 to one port group on ESX1 and lable it as VLAN10.

At the same time I assign VNIC1 of VM10 to one port group on ESX10 and set it as VLAN10.

Then I guess I need to set all the switchs as trunk.

When I want to send a packet from VM1 to VM10 what will happen?

This is my scenario:

switch1 takes the packe and send it to all ports and then switch 2 take it and send it to all ports of its own ports and this continues in all switches.( how switches know about VLAN configuration inside ESX) and I think it make extra traffic on switches.

[rickardnobel · Accepted solution]

oxment wrote:

sorry I am not expret on networking but I have a question here.

For example if I have one VM1 on ESX1 and VM10 on ESX10.

I assign VNIC1 of VM1 to one port group on ESX1 and lable it as VLAN10.

At the same time I assign VNIC1 of VM10 to one port group on ESX10 and set it as VLAN10.

Then I guess I need to set all the switchs as trunk.

When I want to send a packet from VM1 to VM10 what will happen?

This is my scenario:

switch1 takes the packe and send it to all ports and then switch 2 take it and send it to all ports of its own ports and this continues in all switches.( how switches know about VLAN configuration inside ESX) and I think it make extra traffic on switches.

No, the frame will be sent only once from start-to-end and will not appear on any other switch ports than those. That is actually the main features of switches compared to hubs. There will be minimal impact on the network, i.e. just the same as sending any random frame between two networking devices.

[oxment]

wowww

you are right, I just got confused by myself !!! :smileygrin:

thank you very much