timcoote
Contributor
Contributor

Virtual Switches

Jump to solution

Hullo

Are virtual switches a necessary component of the VMWare infrastructure model? I'm thinking of banning them from my enterprise architecture and i'd like to know what I lose. I know that I'll gain a simpler deployment architecture and at least have a chance of being able to identify biz application to physical hardware dependencies, but I"m sure that I'm going to lose something.

Hope someone can help me.

cheers

Tim

0 Kudos
31 Replies
Rodos
Expert
Expert

Man what a thread. Tim, it looks like you are trying to really build something reasonably large and serious here. Some very smart and experienced VMware people have been answering your questions but a public internet forum is probably reaching its limits (we need a virtual whiteboard).

I would strongly suggest you engage an experienced VMware consultant who has experience in operational readiness to give you some guidance in possible architectures and designs. Alternatively get in contact with your local VMware sales office or a VMware partner and have a session with one of their experienced Systems Engineers.

Just reading this I worry that one of two things might happen. One you may get turned off virtualisation as a technology and not end up reaping the amazing benefits it can bring to what you are trying to do. Alternatively you may implement it and end up with such a whacked implementation that it will be worse than if you had not done it at all, costing you more in hardware and operational costs.

We would all love to hear how you get on, so post back and communicate. But having just read through all of this you will get great value of some face to face time with someone who is experienced in this field and walked the path before you. If you have already done that, find someone else, they did not do a very good job.

Rodos

Consider the use of the helpful or correct buttons to award points. Blog: http://rodos.haywood.org/

Rodos {size:10px}{color:gray}Consider the use of the helpful or correct buttons to award points. Blog: http://rodos.haywood.org/{color}{size}
0 Kudos
Ken_Cline
Champion
Champion

I know about those tools - I was a founder of one of those companies Smiley Wink

What company - it would help to understand more about your background so we can better phrase our response to your vernacular.

Not sure where the spof is: if I allocate one switch per pnic, I've got a one to one mapping of vSwitch to physical switch. I then do whatever I normally do to ensure diverse routeing with physical servers and ensure that each VM vnic is connected to two vswitches. Why would I want to have a different configuration approach for my virtual servers and my physical servers?

OK...basic misunderstanding of the way that a vSwitch works. In a virtual environment, you are removing the responsibility for redundancy from the guest OS. That is a significant simplification of your environment. Instead, you place the responsibility for redundancy at the vSwitch level. Your VM will have a single connection to a single vSwitch, which will - in turn - have multiple, redundant connections to your physical infrastructure. The vSwitch will detect path failures and automatically failover to an alternate path...totally transparently to your VM. Additionally, since the vSwitch is a layer 2 device and does not participate in the spanning tree protocol, there is NO POSSIBILITY of the introduction of a spanning tree loop (unless you do as you're suggesting and have a VM bridge between two vSwitches...)

How would I coordinate this switch naming across the hundreds of ESX servers in my estate? I try to avoid VLANs as they complicate the configuration, usually unnecessarily and catch out the unwary.

The beauty of it is that you WOULDN'T coordinate the switch naming across hundreds of ESX servers. The vSwitches do not have names in the traditional sense of the network world. Your pSwitch will view them essentially as a host (albeit a host with a lot of connections)

All of your ESX hosts (within a particular environment) would be configured IDENTICALLY. All the vSwitches would be named the same, they would all be managed from a single, centralized management console (vCenter Server, nee VirtualCenter).

>This is simply a design concept that you're going to have to accept.

Yup. Would connecting all VMs to one vSwitch work better?

There are many users who do. It depends on the complexity of your environment. If you have a single, flat IP network, then a single vSwitch could very well be a viable option.

>Well, I'll have to disagree that there is no business value to a vSwitch. Without the vSwitch (or similar) technology, an ESX host would be severely limited on the number of virtual machines it could host. This would drive >down the ROI that could be realized (and drive up the TCO). Also, bringing the flexibility to reconfigure portions of your network with the click of a mouse rather than the movement of a cable is a significant business driver. >Many, many server outages are caused by a technician either moving the wrong cable or inadvertently plugging the right cable into the wrong port.

I'm sure that there is value. I just want to quantify it. All I'm seeing at the moment is cost and risk. I don't really follow your logic here, I'm afraid.

The basic logic is this: a typical Windows-based x86 server is running at less than 10% utilization - that, in effect, means that you are realizing only 10% of the benefit of your investment (and that doesn't count all the additional overhead of power, cooling, footprint, HW maintenance, etc.). The goal is to consolidate as many of these "low level" workloads as practical onto a single robust server - driving the utilization of the platform up while lowering the overall infrastructure costs. In a typical ESX environment, we're seeing 20:1 or higher consolidation ratios. Those levels would be impossible using your proposed two pNICs per VM approach.

And I really don't want a gui to drive it as it makes automatic backout of changes more than fragile.

Changes can be implemented via scripts - and with appropriate change control processes in place, I would assume that a change would not be authorized without a tested fall-back plan. The technology, by itself, does not solve a problem - it is merely a tool to assist you in managing your environment.

I'm sure you're right. I want to see how these pieces of value come about. There's clearly extra costs around training and maybe team size. There will be a trade off against server deployment costs and economies of scale around platform consistency, etc.

There will be increased costs for training and for storage. In a typical deployment, there are reductions in cost for physical footprint, power, cooling, HW maintenance, network infrastructure, server lifecycle management (acquisition, provisioning, operations, decommissioning, disposal) and (frequently) staffing. Storage costs will typical increase because, to derive maximum benifit from virtualization, you will likely need to provision additional SAN-based storage at what could be a significant expense.

A lesson that I've learned about IT Operations is that it is full of technology that destroys value (shelfware, products deployed and not used, overlapping and even competing solutions from different technology groups). Most of this destruction comes from where technology is used in business processes that do not properly span the IT organisational silos, eg network team and server team, or server team and storage team or firewall team and application team. Technology per se does not do anything. You have to get the People and Process pieces right, too.

I agree with this 100%. This is a clear demonstration of how process needs to drive technology, rather than the other way around. If you adopt technology simply because it's "cool" then you're asking for trouble.

Just because I can put in a complex switching fabric inside every ESX server doesn't mean that that I should.

I would NEVER recommend establishing a complex switching fabric inside any ESX host - unless that was the best solution to a problem. Most of the time, complexity breeds complexity, which breeds fragility and failure. I can't remember a single implementation that I've been responsible for that had a complex intra-ESX switching environment. Although, I guess complexity is a matter of perspective...

>Since the vSwitch is an integral part of the virtual infrastructure, you cannot realize these benefits without the use of vSwitches (or similar technologies). There is a learning curve - and I would encourage you to make an >investment in the education of your support staff. It is an investment that will be returned many times over.

I think that your first point is a non-sequitur. That's the point that I want to bottom out.

Well...in a VMware ESX environment, it is not a non-sequitur. You MUST use vSwitches if you want your ESX host to be connected to the external network - there is no alternative.

I don't see what having that extra level of configuration items in my estate gives me.

The use of vSwitches allows you to REDUCE the complexity of your overall environment. I know you don't believe me (yet), but you will Smiley Wink

I don't think that it's a common architectural necessity across all virtualisation approaches.

The vSwitch is unique to ESX; however, all virtualization platforms use some sort of abstraction of the pNIC.

It may well be essential to scale up an individual ESX server that I spend the time tuning the internal switch configurations for some IT service patterns.

However, at scale, I'd rather avoid the variability if I can. I can allow my desktop users to futz around with their configurations, and when the world was like that, Gartner were estimating the annual cost of supporting such a desktop as 25k usd. Smiley Happy

At scale, the use of consistently defined vSwitches across your ESX estate actually reduces variability.

Ken Cline

Technical Director, Virtualization

Wells Landers

TVAR Solutions, A Wells Landers Group Company

VMware Communities User Moderator

Ken Cline VMware vExpert 2009 VMware Communities User Moderator Blogging at: http://KensVirtualReality.wordpress.com/
0 Kudos
Texiwill
Leadership
Leadership

Hello,

As Rodos stated I would talk one on one with either VMware or a consultant that has implemented VMware ESX. I know when I talk to customers we spend quite a bit of time on the virtual networking. It is a critical component of the virtual environment and is often quite confusing. As Ken stated, once you define your virtual switches, then each VMware ESX host will have an identical configuration. When Distributed Virtual Switches are available this would be handled for you at the VMware CLuster level (above the ESX hosts). But that is a future item.... All it does is make the management of the virtual network easier.

The complexity of your virtual network will be determined by the number of security zones within your existing network that you wish to virtualize. The one thing you can not do within the virtual network is directly layer virtual switches, so things are generally relatively flat. Also note that ESX has at least 4 of its own virtual networking security zones.


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll

Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links

--
Edward L. Haletky
vExpert XIV: 2009-2022,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos
timcoote
Contributor
Contributor

Thanks to all for the input. I've learned a lot from this interaction. However, I sense that I'm annoying people and I don't wish to outstay my welcome in this forum, so I suggest that if anyone wishes to discuss the issues further, that we do it via email: tim+vmware.com@coote.org

Cheers

Tim

0 Kudos
Ken_Cline
Champion
Champion

Tim,

No harm, no foul. Noone here has thin skin Smiley Happy

I think this is a valuable discussion that will benefit lots of folks - it's not often that there is a business value discussion here - it's almost always technical. Let's keep it going here for the benefit of all...

Ken Cline

Technical Director, Virtualization

Wells Landers

TVAR Solutions, A Wells Landers Group Company

VMware Communities User Moderator

Ken Cline VMware vExpert 2009 VMware Communities User Moderator Blogging at: http://KensVirtualReality.wordpress.com/
0 Kudos
Texiwill
Leadership
Leadership

Hello,

Tim you have not out stayed your welcome, and no one is as Ken stated 'thin skinned' on the forums. It is a very good discussion one that is very important to have. You are not asking bad or what others would could consider wrong questions. These questions are valuable and important to answer.

Keep asking them!


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll

Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links

--
Edward L. Haletky
vExpert XIV: 2009-2022,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos
Rodos
Expert
Expert

Tim, there is no annoying, we are a friendly bunch. We are all keen for your success. I really think if you spend some time in front of someone with a whiteboard and work all of this through you are going to fast pace some deeper understanding, which can be hard to do in a forum to a certain level.

But keep the discussion going. We will all be interested in your progress. Everyone has contributions to make, including you.

Rodos

Consider the use of the helpful or correct buttons to award points. Blog: http://rodos.haywood.org/

Rodos {size:10px}{color:gray}Consider the use of the helpful or correct buttons to award points. Blog: http://rodos.haywood.org/{color}{size}
0 Kudos
timcoote
Contributor
Contributor

Thanks all. I need to get more concrete requirements so that I can work through the details.

I'll share as I get further and better particulars, but don't hold your breath as I'm currently building value propositions/scenarios, based on my experience rather than saving cash for a customer, so I don't have the detailed numbers to hand.

One thing that I have noticed as a dig around more is that the primary market for VMWare server virtualisation is mopping up the 'one app per server' Wintel deployment model. I'm taking that value as a given and quite easy to capture.

0 Kudos
mreferre
Champion
Champion

the primary market for VMWare server virtualisation is mopping up the 'one app per server' Wintel deployment model

That's true. Fact is that it's (practically) the only Wintel deployment model....

Massimo.

Massimo Re Ferre' VMware vCloud Architect twitter.com/mreferre www.it20.info
0 Kudos
Texiwill
Leadership
Leadership

Hello,

If customers do have more than one app per system often they split them into separate VMs when they virtualize. This alleviates many issues that spring up from application interactions, etc.


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll

Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links

--
Edward L. Haletky
vExpert XIV: 2009-2022,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos
mreferre
Champion
Champion

Ed,

you don't have to sell me on that.... Smiley Wink

http://it20.info/blogs/main/archive/2007/06/25/32.aspx

Massimo.

Massimo Re Ferre' VMware vCloud Architect twitter.com/mreferre www.it20.info
0 Kudos
nabsltd
Enthusiast
Enthusiast

That's all fine and dandy, but why do I have these extra pieces of technology in my estate?

Think of your VMs as if each one was a physical machine sitting in a rack. Each VM has the same sort of hardware a physical machine would have...hard drives, video cards, network cards, etc.

So, like a physical machine, if you want a VM to connect to the rest of the machines on your network, the VM needs a network card, and that network card needs to be plugged into a switch. You can't plug a VM into a physical switch because you don't have the right cable, so you plug it into a vSwitch. VMware provides an infinite number of the cables to do this plugging for free with each ESX server license.

Then, to connect your vSwitch to the rest of your network, you need an uplink cable between the vSwitch and a physical switch. This role is served by the physical NICs in the ESX host.

0 Kudos