Vlan routing - best practise?

[m1kkel84]

Hey guys.

We want to seperate all our servers into vlans, on different subnets, but we want to be able to communicate with one firewall (ofcourse) but we also want to be able to connect to each server, from a central server (for monitoring etc.)

So we have to do some vlan routing. What is the best thing to do, vlan routing in our L3 switch? Or in the firewall (cisco asa) ? Switch can handle 64 ip routing domains, so it will take some time to run out of ip domains to route...

let me know.

best regars Mikkel

[peterdabr]

I would use L3 switch to keep default gateways for each network/vlan configured on the ESX hosts (I assume we are talking about ESX hosts here ) and perform routing and firewalling between routed networks using extended ACLs. ESX hosts would be either directly connected to it or connected to a intermediate zone switch that then uplinks to L3 switch. I would keep ASA strictly as a firewall/VPN device for the external traffic. Besides, I believe ASAs like 5510 have only 3 NICs and you would need to 'partition' one of the nics into multiple vlan logical interfaces, essentially dividing 1Gb bandwidth between multiple networks, whereas L3 switch would provide full 1 Gb link between any given pair of routed networks.

Best,

Peter D.

[chadwickking]

Peters reply is right on. That is mirror of our production environment except for our back bones and switching is alot more complicated. but overall good solid practice.

Cheers,

Chad King

VCP-410 | Server+

Twitter:

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful

[m1kkel84]

Well let me specify how my setup should be, if you have better ideas, please comment!

Router - ASA5510 default vlan 1 1.1

VM1 - Customer 1 - 10.2 - Vlan 10

VM2 - Customer 2 - 11.2 - Vlan 11

VM3 - Customer 3 - 12.2 - Vlan 12

VM4 - Customer 4 - 13.2 - Vlan 13

VM5 - Monitoring - 14.2 - Vlan 14

So All vm's must be able to reach router, and not each other, and VM5 must be able to reach all vm's regardless of vlan.

Beaiclly i want to seperate customers into different vlans.

The customers are placed on 4 esx 4 hosts, connected to a zyxel GS2750 L3+ switch. where the asa also is connected to.

After getting off support with zyxel, here is what they told me:

When i create a routing domain for each subnet, each subnet will be able to reach each other. Witch is not the plan. They told me that i need a firewall to be able to control who can reach who, but i am sure a cisco L3 switch can control those things ?

Then i tried tagging my way throught this, by creating trunks to vSwitch, but i can not get traffic from VM on vlan 6 to router on vlan 5 for example. (through the trunk) then zyxel told me that i need to do tx tagging in both ends, and i cannot do tx tagging in a virtual switch I was able to get traffic from VM on vlan 6, to physical machine on vlan 6, through trunk. But no to another vlan.

Please help, and advise. This should be pretty common issue. Maybe i need to replace switch to cisco, eh ?

[peterdabr]

I'm not familar with Zyxel L3 devices but, judging from their response, it doesn't have firewall capabilities to create ACLs for different vlan interfaces or it hasn't been configured properly yet.

If you have a budget to buy Cisco, I would recommend switching to Cisco L3 switch, like 3750 (max of 1024 vlans, I believe). You could find tons of documentation on how to configure it.

With Cisco routing switch, you would simply create a vlan interface, acting as a gw for each of the customers network and then use Access Control List to restict traffic between customers. You would then configure each port on the physical switch each ESX host connect to as a trunk port with all customers vlans on it. Also, setting up VLAN 14 for monitoring wouldn't be a problem at all.

"zyxel told me that i need to do tx tagging in both ends, and i cannot do tx tagging in a virtual switch" - that's not true. You can configure port group on vSwitch to be an 'access port' with vlan tagging (in that case you don't have to specify vlan # on the port group, but just make sure physical switchport that ESX host uplinks to is set to 'access mode' with proper customer vlan) or, in your case with many customers sharing the architecture, and each port group on vswitch having its own vlan # and pNICs on ESX hosts uplinked to the physical switchports configured in 'trunk mode' with all vlan #s specified.

Lastly, if you can spend more money, avoid single point of failure (with only one L3 switch) by investing into two of them. Then you would simply configure HSRP for each vlan interface (client network gateway) to make it fully redundant and highly available. Again, there is tons of documentation on how to configure HSRP on Cisco devices.

Cheers,

Peter D

[m1kkel84]

Cisco is indeed the way to go, but its so darn expensive. Buying 100 mbit 48 port cisco layer3 switch, is much cheaper than 1Gbit, But 100 mbit for the entire infrastructure is a step backwards i think.

Currently i have tested creating a port group for each customer, each portgroup have a specefied vlan eg. 10, 11 or 12 etc. portgroup in vswitch, and vswitch in pswitch, where the connection from vSwitch to pSwitch have been set to trunk mode. This also works through the trunk. see this:

Customer 1 - vlan 6 - virtual machine 7.3

Laptop - vlan 6 - physical machine. 7.2

There fine connection between the two, through my zyxel switch, trunk etc.

When laptop tries to access router 7.1 , on vlan 5, everything works fine (here we are not groing through trunk)

BUT when Customer1 - vlan 6 - virtual machine 7.3 tries to access router on vlan 5, 7.1, it fails. So going through trunk, into a different vlan, does not seem to work. Doing it just from physical machine to router from vlan 6 to 5, it works!

[peterdabr]

So it seems that only sending traffic between two different vlans, and only from an ESX host doesn't work.Try setting up vswitch<->pswitch link to access mode with only 1 vlan just to see if you could communicate with another vlan/network. Also, I'm not sure if you use vswitch or dvswitch, if the latter then make sure Uplink group has all vlans specififed there.

I would normally ask for a snippet for the switch config (althought his might not be a proper forum to do so ), but since it is non-Cisco, I can't be of any help...

Keep us all posted if you find the solution.

Best,

Peter D.

[chadwickking]

I too wish I could be of more help but all I have ever used are cisco. Though they are expensive the reliability, support, and performance are truly great. No to mention all the great forums support you can get as well and many other types of information. I would just ensure that all your tagging is correct on the portgroups as well and that trunking is set up correctly. I used a zyxel gateway device once but that was it

Cheers,

Chad King

VCP-410 | Server+

Twitter: http://twitter.com/cwjking

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful

[m1kkel84]

Yeah i am really impressed with the support we get on our cisco asa! They really know what they are talking about, compared to zyxel!

I will buy a cisco switch instead, also because everytime i read a whitepaper from vmware,everything is written "in case" you have cisco equipment.

I will see if i can find a used 1 gbit switch, with 48 ports, layer 3. Then maybe, i might get back. My experience with cisco is evry poor.

I use vswitch, and not dvswitch, by the way!

[rickardnobel]

I will see if i can find a used 1 gbit switch, with 48 ports, layer 3.

You could also look at other vendors than Cisco, for example HP. Here is one model that I have been working with, it has 48 ports and L3 routing with ACL support if needed, among many other things.

http://h10144.www1.hp.com/products/switches/HP_ProCurve_Switch_3500yl_Series/overview.htm#J9473A

There are of course many other brands of 48 ports switches from HP, and from other vendors too.