mattjk
Enthusiast
Enthusiast

How important is iSCSI switch isolation?

Jump to solution

Hi all,

How important is switch isolation when using iSCSI with ESX (i.e., having dedicated switches for iSCSI traffic), as opposed to sharing switches (iSCSI & regular network on the same switches) and using VLANs to segregate traffic? Is the general recommendation that iSCSI switch isolation is a good idea due to security, or performance, or what?

We are looking at a NetApp FAS20x0 to use with our ESX deployment, and want to take advantage of it's NFS features (e.g. users directly accessing older snapshots of data on RDMs) and IP-based replication.

To use these features the NetApp unit would have to be physically connected to the main LAN. We can achieve this by connecting the LAN and iSCSI switches together, but this seems a bit silly if the main purpose of iSCSI switch isolation is physical security.

So, I'm wondering if switch isolation for iSCSI is really necessary - using the same switches for LAN & iSCSI (with VLANs) would be simpler, and saves us the cost of two dedicated switches for iSCSI.

Thoughts? Answers? Opinions?

Thanks in advance.

Cheers, Matt
0 Kudos
1 Solution

Accepted Solutions
mike_laspina
Champion
Champion

Hi,

The separation of iSCSI is very important I did a short blog on it if your interested.

http://blog.laspina.ca/roller/Ubiquitous/entry/iscsi_security_basics

http://blog.laspina.ca/ vExpert 2009

View solution in original post

0 Kudos
29 Replies
kukacz
Enthusiast
Enthusiast

In my opinion there is no reason to physically separate switches for iSCSI traffic. It's costly, it's not too flexible.

You should separate the LAN and iSCSI traffic using VLANs to be safe. As long as you use enterprise-class switches you can stay calm with the performance and feature requirements.

--

Lukas Kubin

Texiwill
Leadership
Leadership

Hello,

Whether to Isolate iSCSI is really a security call and not a performance or redundancy issue. Please remember that for VI3/ESX iSCSI requires the participation of the SC, so isolation is limited anyways. If the SC (Administration Network) must participate in the iSCSI Network then the most secure method is to use a firewall to allow CHAP and protect the iSCSI network from the SC.

You do however want to isolate your Storage Network from VMs and other systems that are not using it. The reasons for this are two fold. 1 is to limit cross chatter caused by broadcasts sent out by some Microsoft Windows installations (They are noisy) and the other is to limit access from a Security point of view. Most iSCSI implementations use CHAP for Authentication (which is encrypted) and this is what the SC must be able to do, whether you use CHAP or not. BUT the non-authentication traffic is not encrypted. That part of the iSCSI Spec is not implemented by many vendors, so it is a clear text protocol and there are several whitepapers out there that discuss how to intercept this traffic using pretty standard MiTM attacks.

Given that NFS, and iSCSI are generally clear text, this is why you want isolation. You would not want a hacker (internal or external) to get a hold of credentials, critical data or anything else.

Also NOTE, that VLANs only protect you so far. There are a number of VLAN Jumping (encapsulation) attacks that work against most physical switches. Granted Cisco and the other switch vendors are working to close these gaps, they can not be fully closed til IPV6 w/IPSec is widely in use within the datacenter. Also, VLAN attacks are a hot subject of research in the black hat world.

It all boils down to three things.... Your Written Security Policy, The Levels of Trust within your company, and any Organization, Federal ,or State Guidelines that must be met for your line of work... I.E. SOX/Hippa/PCI, etc....


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XII: 2009-2020,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
jeremypage
Enthusiast
Enthusiast

Note:It can become a performance issue if the switches' backplanes get overloaded. With most new boxes that's not going to be an issue, but if you have older hardware it would be a good idea to see what their total IO is.

0 Kudos
pdrace
Hot Shot
Hot Shot

My servers are on physically seperate switches mostly for security reasons as Texiwill has explained. I'm using 2 HP switches with switch meshing enabled and it's worked well.

We also do this with our Oracle NFS mounts on Netapp.

0 Kudos
mattjk
Enthusiast
Enthusiast

Thanks for the info guys - sorry to ask a question and then just disappear for a week, it's been a hectic one Smiley Sad

Doesn't sound like there's a definitive answer (other than use good switches and VLANs at the very least).

I guess we're just going to have to consider the pros and cons and make a call on it - we're a fairly small company so we don't have any formal security policies to guide us, and we're not covered by any particular legislative requirements either.

The main benefit for us not physically seperating iSCSI traffic is that we'll be able to afford slightly better switches. For example, rather than getting 4 x ProCurve 2810-24G's for iSCSI & our new network "core", we could get 2 x ProCurve 2848's or 2 x Catalyst 2960G-48TC-L's instead.

Does anyone have an opinion on whether the better switches is likely to be of bigger benefit than physically seperating iSCSI?

Thanks.

Cheers, Matt
0 Kudos
mike_laspina
Champion
Champion

Hi,

The separation of iSCSI is very important I did a short blog on it if your interested.

http://blog.laspina.ca/roller/Ubiquitous/entry/iscsi_security_basics

http://blog.laspina.ca/ vExpert 2009

View solution in original post

0 Kudos
Texiwill
Leadership
Leadership

Hello,

Mike is correct, there is a definite response. Separation should be required as iSCSI non-CHAP traffic is clear text.... A VM for example should not be able to access a VMFS iSCSI target.


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XII: 2009-2020,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos
kukacz
Enthusiast
Enthusiast

Texiwill, I don't understand why you stricly force people in this forum to use physical switch separation for iSCSI traffic.

Sure, iSCSI is a clear-text protocol. But, 1) switch is not a hub, there are only 2 ports involved in each iSCSI frame's path, 2) VLANs allow for another kind of traffic separation, 3) there might be lots of unsafe protocols on the LAN side of ESX envirnonment too, so securing the iSCSI only would have no effect on overall security.

Of course, there are attacks. To break shared-switch security and make some benefit of stealing iSCSI frames, the attacker would have to put on a great effort. Also, there are sorts of attack which even physical switch separation would not prevent.

The "never say never" rule applies (not so) surprisingly often and I don't agree physical switch separaration is a must for any iSCSI installation. It's always on the system architect to decide between various factors and to what scale they apply.

--

Lukas Kubin

0 Kudos
Texiwill
Leadership
Leadership

Hello,

There are a number of existing attacks that would jeopardize your iSCSI/NFS/Administrative physical switch configurations. Unfortunately, this will continue until IPV6 w/IPSec is implemented within the datacenter. I do not see that happening any time soon, but the poster stated there was no clear cut statement that it should be isolated. I think there is a clear cut statement that it should be. I treat iSCSI/NFS just like I would SAN and keep everything separate.

However, whether or not it should be or should not be depends quite a bit on your Security Policy, security requirements (which should be in the security Policy), regulatory requirements, and your levels of Trust within an organization.

You are correct, it would take quite a bit to sniff iSCSI traffic on most networks. However, I know plenty of disgruntled employees and hackers that will go to great and even convoluted lengths to either hack a system, or access data they have no right to access. Bastion hosts will hopefully protect the inside from the outside and further bastions prevent Pivot attacks, but it is really the inside that is the greatest cause for security concern. 70% of all attacks come from the inside.

But once more this truly depends on your levels of Trust, your Security Policy, and any other required levels of Security either regulatory, or dictated. As you state the system Architect should talk to Security and design with this in mind. Not every industry requires this level of Security. Nor can everybody afford it...


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XII: 2009-2020,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos
mike_laspina
Champion
Champion

Hi,

Security is not only about someone hacking into the switch. For example this real world example really happened.

Lets say you need to upgrade your switch firmware to support a specific STP protocol and your iSCSI network shares the same hardware.

Unfortunately you discover that this firmware has issues that destabilized the iSCSI network and drops some servers.

This is just one of many possible issues.

Now consider that you are running 10 Vmware hosts on this environment.

This is an uneccessary risk that does not need to happen.

The cost of this event was over $100,000. The cost of two switches ~$8,000.

The value of separating the iSCSI switch environment sometimes has to be experienced, then it is truly understood and appreciated.

Hopefully you will not have to find out the hard way.

http://blog.laspina.ca/ vExpert 2009
0 Kudos
Texiwill
Leadership
Leadership

Hello

Mike may I use this as an example, with appropriate credit, in some works I am writing? This is a very good case for separation and a way to protect from unknown but very possible disasters and impacts on business continuity. Security is not always about protecting from the wily hacker, but protecting against any type of failure. For those who think BC/DR is separate from Security, they should look at the Common Body of Knowledge required to pass the CISSP certification.


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XII: 2009-2020,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos
mike_laspina
Champion
Champion

You are welcome to use it.

http://blog.laspina.ca/ vExpert 2009
0 Kudos
kukacz
Enthusiast
Enthusiast

Mike, your example doesn't fit well into this thread. You wrote a case about switch - storage paths - redundancy while we are discussing iSCSI and LAN switch separation.

In my opinion (also accepting some of the arguments written in posts above) there is a scale of levels of security and business continuity for everyone to choose from. Applied to number of switches used, this scale should probably look like the following (sorted from insecure to more secure):

  1. Single switch for both LAN and iSCSI traffic. Single point of failure for all traffic; vulnerable to switch attacks.

  2. Two switches for separated traffic. Single point of failure for both sorts of traffic; less vulnerable to switch attacks.

  3. Two switches for mixed traffic. Redundant paths - no single point of failure for all traffic; vulnerable to switch attacks.

  4. Four switches for separated traffic. Redundant paths - no single point of failure for all traffic; less vulnerable to switch attacks.

--

Lukas Kubin

0 Kudos
mike_laspina
Champion
Champion

The original Q.

How important is switch isolation when using iSCSI with ESX (i.e., having dedicated switches for iSCSI traffic), as opposed to sharing switches (iSCSI & regular network on the same switches)

My opinion.

The separation of iSCSI is very important.

Your opinion.

In my opinion there is no reason to physically separate switches for iSCSI traffic.

Enough said as far as I am concerned.

You have your view and I have mine.

http://blog.laspina.ca/ vExpert 2009
0 Kudos
jeremypage
Enthusiast
Enthusiast

I understand what you are saying (and in fact have redudant 3850's dedicated for back end storage traffic only in my environment) but realize that for smaller shops the questions is "do I buy a switch for iSCSI (or NFS as the case may be) or use my existing switches"?

In my opinion redundancy is more important than isolated hardware. So I'd rather use a dedicated VLAN for iSCSI across multiple switches then a single switch for iSCSI only.

Obviously if you can afford redundancy and it makes business sense to pay for it, do so. Keep in mind that the VMware best practices are written from VMware's point of view, i.e. what's going to make their product as bulletproof as possible. That does not always equate to what's best for your business, which is why there is so much discussion on the topic.

0 Kudos
mike_laspina
Champion
Champion

For small shops there are significant reasons that the whole availability/confidentially/integrity part would be mush less important. But in the same light these companies still have an obligation to the information protection acts. And there are very inexpensive switches out there in the $300 to 1500 range which solve the issue of cost for small shops. The smaller shop is less likely to be using a system that needs this type of configuration. But in the long run what it comes down to is risk verses cost how much would it cost if the system went down verses how much does it cost to mitigate the risk element and what is the probability of the failure event. The challenge is not the actual hardware fault e.g. the power supply fails in the switch. It is the combination of elements that can lead to failure.

If the switch was separate what is the probability that an error will happen during a network change that is unrelated to the SAN.

What is the probability that a port will be plugged into the wrong network etc.

http://blog.laspina.ca/ vExpert 2009
0 Kudos
mattjk
Enthusiast
Enthusiast

Thanks for the extra input everyone - looks like I opened a bit of a can of worms here Smiley Happy

Mike is correct, there is a definite response. Separation should be required as iSCSI non-CHAP traffic is clear text.... A VM for example should not be able to access a VMFS iSCSI target.

Looks like I mis-read your original post - thanks for clarifying.

We can afford the extra switches - even if it is a stretch - so we'll go down that route.

Another quick question I'm hoping the gurus can help me with:

Given we will have a very simple iSCSI network (a couple of servers + a SAN), is there any problem using cheaper switches for iSCSI? e.g. ProCurve 1800-24G or 1400-24G?

They don't have the same level of features of the 2810-24G / 2824 / 2900-24G, but still appear to offer full port speed on all ports (48 Gb/s switching capacity) + jumbo frames + flow control, and I don't think we need full-on L2/L3 managed switches for our rather simple iSCSI network.

Opinions?

Cheers, Matt
0 Kudos
mike_laspina
Champion
Champion

I use the HP 2810 and 2900 series. The 1800 series are also a very good cost effective solution and will serve well.All the HP switches are reliable. I see no problem at all.

http://blog.laspina.ca/ vExpert 2009
0 Kudos
mattjk
Enthusiast
Enthusiast

Great, thanks for the info Mike.

We'll probably save some money and use 1800-24G's for iSCSI, which then frees up some budget to buy better LAN-side switches.

Now I just need to figure out whether the 2810, 2824 or 2900 is what we need on the LAN side - networking is not my strong point :-S

Cheers, Matt
0 Kudos