VMware Cloud Community
Sly
Contributor
Contributor
Jump to solution

VLAN on a Cisco 3750G

Is a VLAN created on a Cisco 3750G with the latest IOS a "good" way to secure a vmware network? In this case I am hiding vmotion traffic, and the entire network is behind a firewall. I realize it would be better to have the switch dedicated and isolated, but is a VLAN on a Cisco switch reliable and secure? Or does the security lie elsewhere, such as encrypting the vmotion traffic or creating solid ACLs?

Reply
0 Kudos
1 Solution

Accepted Solutions
vSerge
Enthusiast
Enthusiast
Jump to solution

Sly -

I think you nailed it in your post - there are some things you have to do when using VLANs to avoid running into trouble. The vulnerability Tom referred to has to do with IOS/CatOS decoding of VTP frames - just like we see in Windows RPC/NetbIOS or SMB/CIFS vulnerabilities or other remotely exploitable vulnerabilities, it's possible to craft a frame with malicious content that could overflow a buffer, string handling (unvalidated input), double-frees, etc. These type of vulns are often found by "fuzzing" where you create bad frames or partially bad frames and feed them into the device under test, hoping to find a crash or a create a DoS. I remember running simple tools like ISIC (IP Stack Integrity Checker) to validate equipment and every so often you'd cause a switch to crash, especially older IOS. So this isn't limited to just control plane protocols like VTP, this can happen in the data plane too. The data plane is much more robust because it's attack surface area is much more exposed to attacks than protocols like VTP are and a lot of problems have been fixed. If you look back in history, there were tons of security issues in the data plane of Cisco and other gear in lesser used features like IP options, fragment handling, ICMP types/codes rarely used, TCP sequence overflows. Now, I bet if the security research community focused early on protocols like CDP, VTP and STP - you would have seen more vulnerabilities earlier.

So to say "do not use VLANs otherwise you are vulnerable because of a VTP vulnerability" is equivalent to saying do not run IP using Cisco routers/switches when the IP options and ICMP vulnerabilities existed in the data plane.

Now, if you had followed what Cisco and other L2 switch vendors recommend to do, you would not be exposing your VTP domain for such attacks and therefore you would not be vulnerable. Just like you would not expose your switches to recieve Spanning Tree BPDUs or dynamic routing protocol packets like OSPF, IS-IS or BGP from untrusted speakers. Take a look at a blog that I posted w/r/t this subject:

There are lot of fear created in the community about L2 attacks because networks and network devices are often a mistery to server folks and an improperly setup L2 could be a source of security and stability problems. It's important to educate the community on the possible exposures, and VMware and other market leaders like Cisco are taking responsibility to do so.

Disclosure on my part - I'm speaking from and have had the operational experience of setting up and maintaining one of the largest global datacenter networks in the world (Global Crossing/GlobalCenter->later became Exodus->Savvis) as one of the senior network engineers and even 10 years back we'd have datacenters with massive switch fabrics that homed customers like Yahoo, Ask Jeeves, etc - isolated and segmented using VLANs. If you go into a large hosted datacenter today, you certainly would not get your own physical switch and a backbone uplink - you'd be sharing a 6500, a large Extreme or Foundry for often 100+ other customers.

View solution in original post

Reply
0 Kudos
13 Replies
Rubeck
Virtuoso
Virtuoso
Jump to solution

Seperation using VLANs is in most cases adequate, IMO. But take a look at the linked NSA document... although the is for ESX 3, I would say the genral terms still applies..

Hope it helps....

/Rubeck

Reply
0 Kudos
TomHowarth
Leadership
Leadership
Jump to solution

No VLANs are not an security mechanism, it is a traffic separation technique. but nothing less that physical separation is enough to guarantee security.

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

Tom Howarth VCP / vExpert

VMware Communities User Moderator

Blog: www.planetvm.net

Contributing author for the upcoming book "[VMware vSphere and Virtual Infrastructure Security: Securing ESX and the Virtual Environment|http://my.safaribooksonline.com/9780136083214]". Currently available on roughcuts

Tom Howarth VCP / VCAP / vExpert
VMware Communities User Moderator
Blog: http://www.planetvm.net
Contributing author on VMware vSphere and Virtual Infrastructure Security: Securing ESX and the Virtual Environment
Contributing author on VCP VMware Certified Professional on VSphere 4 Study Guide: Exam VCP-410
Sly
Contributor
Contributor
Jump to solution

But from a cost perspective not all connections can be easily physically seperated. If you have 4 connections to a VLAN, it may not justify buying another switch. (and I know an argument could be made here about the cost of a security breech) I agree that adding a switch and physically separating it is the best practice, but what other steps can you take on this brand and model of switch to improve your security or is physical isolation the only way?

Reply
0 Kudos
howie
Enthusiast
Enthusiast
Jump to solution

> but nothing less that physical separation is enough to guarantee security.

1) isolation != security. physical separation is still an isolation, but not necessary security.

2) i agree VLAN may not provide enough characteristics some people look for, but then i don't agree physical as opposed to virtual separation is necessary in that vSphere is providing a platform to show "virtual is better than physical" in aspects including separation.:)

Reply
0 Kudos
TomHowarth
Leadership
Leadership
Jump to solution

Interesting comments, but still no cigar. the customers I work for demand seperation not isolation, at this moment in time Zones and vShields do not cut the mustard. they are a step in the right direction but until the limitations for VLAN technology is overcome secruity does equal seperation.

To the OP I will ask your question to a CCIE friend of mine.

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

Tom Howarth VCP / vExpert

VMware Communities User Moderator

Blog: www.planetvm.net

Contributing author for the upcoming book "[VMware vSphere and Virtual Infrastructure Security: Securing ESX and the Virtual Environment|http://my.safaribooksonline.com/9780136083214]”. Currently available on roughcuts

Tom Howarth VCP / VCAP / vExpert
VMware Communities User Moderator
Blog: http://www.planetvm.net
Contributing author on VMware vSphere and Virtual Infrastructure Security: Securing ESX and the Virtual Environment
Contributing author on VCP VMware Certified Professional on VSphere 4 Study Guide: Exam VCP-410
Reply
0 Kudos
howie
Enthusiast
Enthusiast
Jump to solution

> until the limitations for VLAN technology is overcome

what's the limitation of VLAN? what's not enough from vShield? can you spell it out? thanks.

Reply
0 Kudos
Sly
Contributor
Contributor
Jump to solution

Tom, I will be interested to hear what your friend has to say. I have done some additional reading about VLAN security and I have come up with the following opinion. VLANs can be as secure as physical isolation if you really know what you are doing and if you spend the time keeping your switch security current. And herein lies the danger. If not properly configured, VLAN traffic can be hacked. The following Cisco whitepaper ( here) included but was not limited to the following: setting up the proper isolation on your ports (i.e. limit promiscuous ports), set up ACLs and IP filters, turn off unneeded protocols, isolate unused ports in a separate VLAN, turn on rootguard, turn off VTP, and the list goes on... When all was said and done, you better have your best network engineer scrutinize the setup to give it the same reliability as a physically isolated network.

So Tom, I can see your point that isolation is the best way, as it is not as prone to human error and because there will always be another way to hack through the system. And if I was a consultant who didn't want to have to worry about getting sued because I missed something, I would definitely advise physical isolation.

But I think some risk management is needed here, because the more you physically isolate your network, the more you increase the cost for the customer. I do think that it would be extremely difficult for someone to hack traffic on a well configured VLAN. So for those who can accept the risk, I think VLANs are a great option, especially coupled with some of the new security features in vSphere 4.

Reply
0 Kudos
TomHowarth
Leadership
Leadership
Jump to solution

I agree completely with your comment of risk management. but the fact that a VLAN vulnerability can only be mitigated is still a risk too far for some clients. As you have already stated VLANs can be relatively secure if you actively manager and keep upto date your Switch and router firmware. However there is still the period between a attack being found and the breach being patched. for 95% of customers VLANs are adequate risk when mitigated against cost and data type. I raised the point only to state that VLANs are not a security mechanism persay, but a method of seperating traffic flow.

for example a recent VLAN vulnerability against CISCO was dated 5th November 2008. these are not "Old" issues that have gone away and new attacks are being crafted all the time. People really do not take IT security seriously and believe the hype too much. A secruity expert i knew explained VLANs being used as a security mechanism as a castle having a moat, with the drawbridge down and the gate guarded by a half asleep soldier.

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

Tom Howarth VCP / vExpert

VMware Communities User Moderator

Blog: www.planetvm.net

Contributing author for the upcoming book "[VMware vSphere and Virtual Infrastructure Security: Securing ESX and the Virtual Environment|http://my.safaribooksonline.com/9780136083214]". Currently available on roughcuts

Tom Howarth VCP / VCAP / vExpert
VMware Communities User Moderator
Blog: http://www.planetvm.net
Contributing author on VMware vSphere and Virtual Infrastructure Security: Securing ESX and the Virtual Environment
Contributing author on VCP VMware Certified Professional on VSphere 4 Study Guide: Exam VCP-410
Reply
0 Kudos
TomHowarth
Leadership
Leadership
Jump to solution

All I will say to you howie is google VLAN vulnerabilities there is ample evidence to show it is not a security mechanism. As regards vShields interesting technology, but until you can show me a certification under at least common critiera. I do not trust it for sensitive data.

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

Tom Howarth VCP / vExpert

VMware Communities User Moderator

Blog: www.planetvm.net

Contributing author for the upcoming book "[VMware vSphere and Virtual Infrastructure Security: Securing ESX and the Virtual Environment|http://my.safaribooksonline.com/9780136083214]”. Currently available on roughcuts

Tom Howarth VCP / VCAP / vExpert
VMware Communities User Moderator
Blog: http://www.planetvm.net
Contributing author on VMware vSphere and Virtual Infrastructure Security: Securing ESX and the Virtual Environment
Contributing author on VCP VMware Certified Professional on VSphere 4 Study Guide: Exam VCP-410
Reply
0 Kudos
vSerge
Enthusiast
Enthusiast
Jump to solution

Sly -

I think you nailed it in your post - there are some things you have to do when using VLANs to avoid running into trouble. The vulnerability Tom referred to has to do with IOS/CatOS decoding of VTP frames - just like we see in Windows RPC/NetbIOS or SMB/CIFS vulnerabilities or other remotely exploitable vulnerabilities, it's possible to craft a frame with malicious content that could overflow a buffer, string handling (unvalidated input), double-frees, etc. These type of vulns are often found by "fuzzing" where you create bad frames or partially bad frames and feed them into the device under test, hoping to find a crash or a create a DoS. I remember running simple tools like ISIC (IP Stack Integrity Checker) to validate equipment and every so often you'd cause a switch to crash, especially older IOS. So this isn't limited to just control plane protocols like VTP, this can happen in the data plane too. The data plane is much more robust because it's attack surface area is much more exposed to attacks than protocols like VTP are and a lot of problems have been fixed. If you look back in history, there were tons of security issues in the data plane of Cisco and other gear in lesser used features like IP options, fragment handling, ICMP types/codes rarely used, TCP sequence overflows. Now, I bet if the security research community focused early on protocols like CDP, VTP and STP - you would have seen more vulnerabilities earlier.

So to say "do not use VLANs otherwise you are vulnerable because of a VTP vulnerability" is equivalent to saying do not run IP using Cisco routers/switches when the IP options and ICMP vulnerabilities existed in the data plane.

Now, if you had followed what Cisco and other L2 switch vendors recommend to do, you would not be exposing your VTP domain for such attacks and therefore you would not be vulnerable. Just like you would not expose your switches to recieve Spanning Tree BPDUs or dynamic routing protocol packets like OSPF, IS-IS or BGP from untrusted speakers. Take a look at a blog that I posted w/r/t this subject:

There are lot of fear created in the community about L2 attacks because networks and network devices are often a mistery to server folks and an improperly setup L2 could be a source of security and stability problems. It's important to educate the community on the possible exposures, and VMware and other market leaders like Cisco are taking responsibility to do so.

Disclosure on my part - I'm speaking from and have had the operational experience of setting up and maintaining one of the largest global datacenter networks in the world (Global Crossing/GlobalCenter->later became Exodus->Savvis) as one of the senior network engineers and even 10 years back we'd have datacenters with massive switch fabrics that homed customers like Yahoo, Ask Jeeves, etc - isolated and segmented using VLANs. If you go into a large hosted datacenter today, you certainly would not get your own physical switch and a backbone uplink - you'd be sharing a 6500, a large Extreme or Foundry for often 100+ other customers.

Reply
0 Kudos
Sly
Contributor
Contributor
Jump to solution

vSerge - I agree with the points that both you and Tom have made and I am glad to hear an example of where VLANs are used securely on a large scale. It seems that there is a balance with using this technology. If you are managing the infrastructure at a large hosting company, the cost would not be justified in isolating connections physically unless the customer paid for it. In addition, you sound like you know the ins and outs of network security, so you are probably keeping your switches up to date. But there are a few cases where I think VLAN isolation is still not a good idea.

1) You are not encrypting your Vmotion traffic and do not have the staff or the expertise to maintain the network security properly. (Today most people are still using ESX 3.x, which does have the ability to encrypt Vmotion data.)

2) If you are a consultant who is setting up a VMotion network, I would not trust the customer to keep the security current. I think in this case I would push for physical isolation. Hopefully if you are a consultant you have an attorney that has helped you write up and review the contract to protecct from liability.

3) The company or agency you work at is targeted by hackers more often and the data that is being targeted is more sensitive. So examples here would be most federal government agencies and banking firms.

I will be encrypting the data and I will be using the latest IOS and setting up the switch according to Cisco's recommendations. So I feel pretty good about using a VLAN in this case. In general though, I think many in the community think that VLANs are more secure than they really are and so I agree with your point that leaders in the industry need to continue to educate the public.

Let me know what you think...

Thank you,

David.

Reply
0 Kudos
vSerge
Enthusiast
Enthusiast
Jump to solution

Sly - agreed on all points. I can see how a consultant may be worried about liability when working with a customer that has limited expertise on the security front and maintains their own environment. We at VMware actually advocate using dedicated pNICs for vMotion traffic and to separate the ESX mgmt traffic away from the VMs' application data plane. Ultimately, VLANs aren't considered a security featureset - they are an excellent way to isolate and segment traffic on shared L2 fabric and create broadcast domains on top of which you implement multiple layers of security controls - network edge firewalls, web-proxies, etc, virtual networking layer ESX firewalling solutions, network IPS/IDS end-point VM security solutions...

Reply
0 Kudos
TomHowarth
Leadership
Leadership
Jump to solution

Ultimately, VLANs aren't considered a security featureset

At last something this is the point I was making.

they are an excellent way to isolate and segment traffic on shared L2 fabric and create broadcast domains

Agreed, that is all they are

on top of which you implement multiple layers of security controls - network edge firewalls, web-proxies, etc, virtual networking layer ESX firewalling solutions, network IPS/IDS end-point VM security solutions...

Finish off the security layer.

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

Tom Howarth VCP / vExpert

VMware Communities User Moderator

Blog: www.planetvm.net

Contributing author for the upcoming book "[VMware vSphere and Virtual Infrastructure Security: Securing ESX and the Virtual Environment|http://my.safaribooksonline.com/9780136083214]”. Currently available on roughcuts

Tom Howarth VCP / VCAP / vExpert
VMware Communities User Moderator
Blog: http://www.planetvm.net
Contributing author on VMware vSphere and Virtual Infrastructure Security: Securing ESX and the Virtual Environment
Contributing author on VCP VMware Certified Professional on VSphere 4 Study Guide: Exam VCP-410
Reply
0 Kudos