jhunter11
Contributor
Contributor

Layer 3 Networking in VCD

Hi Guys,

So, here's my issue...

We have an organization with 3 ORG Networks inside it.  Two of them are attached to an external NAT routed network and the other one is internal only.

The problem I'm having is that everything on any one ORG network can talk to everything else on that org network, but they networks can't talk to eachother.  Basically, I have no layer 3 network telling my layer 2 org networks that they can talk to eachother.  How do I fix this?  Is this something I would achieve with the static routing tab in the org network configure services menu?  Or something in vshield?

Any help would be greatly appreciated, thanks!

James

0 Kudos
14 Replies
_morpheus_
Expert
Expert

You can use static routing or add NAT rules

jhunter11
Contributor
Contributor

I do see the "static routing" tab in my organizational networks menu, but if i add static routing there, where does it add the route?  To the VSE that's acting as the gateway?  How do I know it keeps that route inside the organization?

What do you mean by NAT rules?  Where would I do that at?

0 Kudos
_morpheus_
Expert
Expert

It adds a route to the vShield Edge for that network. I don't know what you mean "keeps that route inside the organization"

You can add NAT rules to the routed org net for the VMs that you want to be reachable from outside the org net

0 Kudos
jhunter11
Contributor
Contributor

OK, so here's my situation.  We are using VCD to give our developers clones of our production environment to build in.

We are cloning our production VMs and keeping IPs all the same as it is in production.  Here's a quick diagram of our production networking:

Inside: 10.9.8.0/21

DMZ: 192.168.192.0/21

DMZ VIP 192.168.200.0/21

I have an org network for each of these networks that is setup with the same IP range.

So, what I meant by I don't want these routes to get outside the organization is that I don't want to add a route to the VSE that is acting as 10.9.8.1 to route to the real production 192.168.192.X.  Does that make sense?

0 Kudos
_morpheus_
Expert
Expert

Not really. Can you make a graphical diagram?

0 Kudos
jhunter11
Contributor
Contributor

I have a support case open to figure out layer 3 networking where my layer 2 is on the org network level.  However, we weren't sure that was an option so we've switched to a new design.  We now have one big vApp with 4 vApp networks all connected to one org network, which has static routing enabled and routes to each vApp network so they can all talk to eachother.  This has solved our issue with networks being able to talk to eachother at the org level we were having previously.  We have run into a small snag though.

In our current vApp, we have almost 50 VMs, most of which have 2 NICs on different networks.  The server to server communication is allowed by the static routing we have in place on the org level, but unless we put a rule for every IP to get to every other IP in the firewall, they can't talk to eachother.  Obviously, we can just have two firewall rules: allow all in and allow all out, but that brought up a new issue.  4 of the 50 or so VMs we need to access from the outside world, so we have NATs in place on the org network that translate to the external IP of the VM and allows us in.  The problem we are experiencing is that with the "Allow ALL OUT" rule, even VMs without NATs are getting out to the outside world.  Is there any way to put rules in the firewall for subnets and not just IPs?  I can't put in 10.9.8.X and I tried 10.9.8.0 and that didn't help either.  Let me know, thanks.

0 Kudos
_morpheus_
Expert
Expert

This is an issue that many people complain about. Unfortunately, VCD 1.5 doesn't support CIDR or ranges for firewall rules, and only accepts single IPs.

0 Kudos
jhunter11
Contributor
Contributor

I figured that was the case.  So, one way we found for it all to work is setting up two rules to allow all in and allow all out.  This solved our server to server communication just fine, but the one thing we didn't expect is all of our VMs could get out of the edge device to the outside world.  I guess I assumed that a VM inside a vApp which has, let's say, an internal IP of 10.0.0.1 and an external IP of 192.168.3.30, wouldn't be able to get out to the External Network (192.168.100.X) unless it had a NAT setup on the Org Network to the External network.  That isn't the case.  The NATs take care of getting into those VMs from the outside, but it appears that the VM's external IP hits it's VSE which goes to the Org Network and then gets out of the Org Network VSE which has the vApp External range of 192.168.3.X and the vCD External range of 192.168.100.X.  Is there any way to stop the traffic from going out the vCD External network's edge device?  Does that make any sense?

0 Kudos
_morpheus_
Expert
Expert

The edge will route (forward) any packets that are allowed by the firewall and where it knows how to reach the destination

0 Kudos
sunvmman
Enthusiast
Enthusiast

I went through this issue . I talk a Linux box and set it up as a router.

just connect to all networks in the vcs and you be fine. 

0 Kudos
jhunter11
Contributor
Contributor

When I got on with support, I was informed that you can add ranges to the firewall if you go into the VSM itself.  I added my ranges and everything is working now.  He says that CIDR ranges will be added in version 1.5.1 to VCD so you don't have to go into VSM directly, he wasn't sure why it wasn't added in the first place.

0 Kudos
markatLM
Contributor
Contributor

From: jhunter1

Sent: Tuesday, January 31, 2012 01:04 PM

To: Nelson, Mark

Subject: EXTERNAL: New message: "Layer 3 Networking in VCD"

VMware Communities<http://communities.vmware.com/index.jspa>

Layer 3 Networking in VCD

reply from jhunter1<http://communities.vmware.com/people/jhunter1> in VMware vCloud Director - View the full discussion<http://communities.vmware.com/message/1903385#1903385

0 Kudos
_morpheus_
Expert
Expert

You can add that to VSM directly, but the next time you touch that network in VCD, the change will be overwritten by VCD

0 Kudos
jhunter11
Contributor
Contributor

Yeah, he made us aware of that aspect of it as well.  We simply did the rules for the static routing and the NATing first, then added the firewall rules in VSM and it all seemed to work.  Smiley Happy

0 Kudos