External Networks, Organizational Networks and Network Pools help

[CSIEnvironments]

Hi,

I've been messing around the entire day trying to get this working. We currently use LabManager and are in the process of moving over to vCloud Director.

How we currently have LabManager setup:

The configuration is fenced.

The configuration's ip range in 192.168.0.20 - 192.168.0.30 (these are static ip addresses the machines use to communicate with each other)

This is then natted to VLAN800 and the IP's 10.1.180.10 - 10.1.180.200 (this an ip pool, we use these IP's to RDP and connect to this configuration from our work desktops)

This configurations is not accessible via the internet (public network)

For some reason I cannot get the same setup working in vCD. Is there any chance someoen could post some pics of what the External Networks, Organizational Networks and Network Pools should look like for the setup. I've tried every combination I can think of but can't get it works.

Please help!

Thanks!

[thedafa]

Hi,

I never used LabManager, so I cant really compare.

But I've managed to get vCloud to deploy solutions, where the vApp has its own internal network (192.168.0.x) and inside VMs are then NAT'ed (fenced) behind a vShield VM.

So all VMs in the vApp gets their own external IPs from a pool... Automagically.

Does this sound like your requirements? Then I can make a little doc/drawing describing our setup.

[CSIEnvironments]

Hi,

Yip that's exactly what we trying to achieve here

The VM's in the vAPP must have their own internal network on the 192.168.1.xx range.

Then each machine's internal ip must be natted to an IP from a pool on our VLAN800. (10.1.180.10 - 10.1.180.200)

Very interested to see your setup. Need to know what belongs in "External Networks, Organizational Networks and Network Pools".

Thanks

[thedafa]

Hi again,

I tried making a Visio of our setup.

Your "external" IP scope goes into External network (EXT). That is your VLAN800.

The Organisation network (ORG) should be External and Direct, no NAT'ting.

On the vApp create an internal network, and connect this to your Org. network. While fencing not being selected, it seems that fencing (NAT'ting) are being done, and the IP's given in your vApp network will be automatically forwarded (NAT'ted) to an external IP in the external scope.

Network pool is not related to networking out of the vApp, but is related to how VM's in the same vApp communicates with each other, if they end up on different ESX hosts. We first tried using PortGroupBased network pool, but ended up using VCD-NI mode and buying Enterprise Plus licenses for the involved ESX hosts. This is only a 'problem' if you dont use Enterprise Plus licenses normally. Basicly if you select VLAN og PortGroup based, you need to have access to a bunch of VLANs already configured on your entire network. If you select VCD-NI you only need one VLAN for this same thing.

Hope this helps you. I spend a couple of weeks trying to figure this out.

Dont hesitate to ask questions.

[JayhawkEric]

We use Lab Manager and are in the middle of migrating to VCD as well.  We use the same type of configurations you described and our setup for VCD is exactly as described by "thedafa" above. 

You'll need to make sure your Network Pool is on a different VLAN than your External Network, as shown in the diagram, or "Host Spanning" of your configurations won't work properly.  This showed up in our dev environment when the VSE was on one host and the VM's within the vAPP are on another.  It works a little different than using distributed switches within Lab Manager.

[bparlier]

Sounds like you have it all under control, for some visios, write up, etc... and maybe a little better understanding of how they work, see the post below. Might be useful.

http://it20.info/2010/09/vcloud-director-networking-for-dummies/

[CSIEnvironments]

Thanks for all the responses.

As I said to thedafa, we have enterprise plus and would  like to use the vDS for cross  host fencing and vmotioning. I tried to  replicate the thedafa's setup  exactly. I have switched to using VLAN400. Here is my setup. Please tell  me if you see something obviously wrong:

http://imageshack.us/photo/my-images/715/mysetupf.png/

Now  the issue comes in when I try build a new vApp. I get an  when I try  create the vApp network, it says this organization does not  have the  resources to deploy a vApp in fenced/nat. I assume i've done  something  wrong in the setup:

http://imageshack.us/photo/my-images/803/pic1c.png/

Lastly why do you need to have a VLAN for the internal pool of IP's (192.168.1.x range) ? In theory this range won't be routed anywhere so why does it need to be tagged with a VLAN id?

Thanks

[thedafa · Accepted solution]

Hi,

Under System tab -> Cloud Resources (left menu) -> Organization vDCs (left menu) -> "your ORG name"-right click -> Network Pool tab

Have you selected your Network Pool here?

[thedafa]

CSIEnvironments wrote:

Lastly why do you need to have a VLAN for the internal pool of IP's (192.168.1.x range) ? In theory this range won't be routed anywhere so why does it need to be tagged with a VLAN id?

If you have an vApp that contains more than one VM (and that would be all, because of the vShield thing that are created with the vApp).

Now this vApp gets deployed, and the two VMs end up on different ESX hosts. Theese two VM need to be able to communicate to each other via their 192.168.0.x net without being routed/natted out.

vCloud solves this using a technology called VCD-NI, that basicly creates a VPN (or an extra layer of VLAN, using mac-in-mac encapsulation) between the ESX hosts.

[CSIEnvironments]

Thanks thedafa Its working as expected now! I really appreciate your efforts!

Question: With regards to the internal network, if we set it to use a portgroup on the VDS instead of tagging it with a VLAN ID, will the machines not be able to contact each other if we do cross host deployments? I thought the dVS would manage this and allow for communication? I ask because we have only been given one VLAN from our security department.

Thanks again for the help so far :smileygrin:

[thedafa]

Great! 🙂 Happy to help.

Hmmm... It sounds plausible.

We usually dont use Enterprise Plus and vDS here, just Enterprise and normal virtual switches. So I am afraid your just have to try it out to test if it works 🙂 Would be great if you could try it and post it back here.

Have a nice weekend.

[JayhawkEric]

Lab Manager uses a Service VM to route traffic for fenced VM's between hosts in the same configuration where as VCD is going to use this secondary VLAN (or VLAN's/Port Groups depending on what type of Network Pool you want to use). 

In theory you shouldn't need to assign a VLAN to the Network Pool for the vApps but in our testing this is the only way we could get it working.  I believe this is b/c I administratively disable VLAN 1 on my switches and that's what it is trying to use.  Once I recreated the Network Pool and assigned the traffic to an available VLAN it worked perfectly.

Eric

[CSIEnvironments]

Looks like you are onto something JayhawkEric, unfortunately IT Security has only given us 1 x VLAN to make use of. But I think you right because I deployed a vApp in a cluster of just 2 hosts, the "service vm" (edge vm) was deployed to the 1 host and the 2 vm's in the vApp were deployed to the other. Once customization was done I could not ping either of the vm's....I then vmotioned the vm's to the same host as the edge vm and networking suddenly worked. (Strangely vmotioning the edge vm to the host with the 2 vm's did not work)

I have a few more configurations to test out begore I throw in the towel and ask for another VLAN from IT Security, namely the vSphere port-group backed network pool.

Thanks!

[thedafa]

For portgroup based, you also need a dedicated vlan per portgrop you pre-create. I had the same idea thinking it maybe would use some of the same mac-in-mac magic, but it didnt and we got a lot of conflicts and strange behaviour.

So... you are gonna wanna get down on your knees for IT Security and get that second VLAN

[JayhawkEric]

Hehe....

You just need another valid VLAN that is different than the one you're using for your external networks. I just used the VLAN ID we have for VM's that are not in VCD since it is already on the switches. Ours isn't public facing or anything like that so I'm not worried about network security. No sense in wasting an entire IP range.

Eric

[CSIEnvironments]

Thanks guys! I've surrendered and ask IT Security for another VLAN. Hopefully this will be resolved now, will report back once tested.

[CSIEnvironments]

So managed to get this sorted with 1 VLAN. This is the same setup as we used in our LabManager fenced configurations. This enables cross host deployments and vmotion between hosts. Connectivity between VM's remains up regardless of which host the vm's in the vApp sit.

External Network:

Connected to VLAN400

Name: ext_vlan400

Network Pool:

Network isolation-backed

VLAN ID: 400

(you need to tag this with the same vlan as the external network. This is so that the dvs created per vApp in VC is vlan tagged and traffic flows eg: dvs.VCDVSBoth_int_ext-77b361e8-e471-4a09-9b93-62064a182041)

dvSwitch

Name: NetPool_NIB_tagged_vlan400

Organization Network

External Network Only

Direct Connection

Name: org_ext_direct_vlan400

Network on the vApp

vApp Network

192.168.1.xx range

Conection org_ext_direct_vlan400 (NAT, No Firewall)

Name: Both_Int_Ext

PLEASE NOTE:

This only works if you have one nic in a Standard Switch connected to VLAN400

And one nic in a dVS. Having 2 nics in a dVS poses a problem as JayhawkEric mentioned.