Securing Sprawling Virtual Machines from Vulnerability-based Attacks

[SecurityJunkie]

Does anyone have any thoughts on the impact of virtualization on server security? I was chatting with a security expert (who will be apparently speaking on an Interop panel in May) and he was genuinely concerned with the security impacts of: decoupled software and hardware; VM sprawl; software updates; and complex server stacks.

He also said that HIPS/NIPS/Firewalls were never designed to protect these kinds of sprawling (hard to manage) environments. Some of their functionality will continue to function, but any features tied to hardware-based signature processing (very common in mature security solutions) would be rendered "virtually irrelevant."

Anyone have any thoughts? Suggestions?

[Pisapatis]

Please be specific on your security concerns so that the vendors can analyze and address the specific threat due to lack of such protection. For example, if there is a hardware signature requirement, let virtualization vendor know the deficiencies of the product and understand how to implement such security requirements to minimize the threats. I see many security experts express their concerns, but not ready to help the community. I am also looking for security scanning tools so that my security folks can test my environment for any flaws before I go for production.

[SecurityJunkie]

More specifically, what do we do with all of the "moth-balled" servers as software patches/updates are made available? I cannot patch instances; and I'm similarly concerned that the sprawl of virtual machines will expand beyond my partitions....

[kix1979]

Does anyone have any thoughts on the impact of

virtualization on server security? I was chatting

with a security expert (who will be apparently

speaking on an Interop panel in May) and he was

genuinely concerned with the security impacts of:

decoupled software and hardware; VM sprawl; software

updates; and complex server stacks.

How is VM sprawl any different from physical server sprawl and all of the issues it presents? I don't see virtualization creating any new headaches in this realm, only perhaps the same ones people have in the physical world.

He also said that HIPS/NIPS/Firewalls were never

designed to protect these kinds of sprawling (hard to

manage) environments. Some of their functionality

will continue to function, but any features tied to

hardware-based signature processing (very common in

mature security solutions) would be rendered

"virtually irrelevant."

How is a virtual environment hard to manage? Treat VMs similar to a physical machine. You still need to patch the OS, maintain the software, update AV definitions etc...

[SecurityJunkie]

How is virtual machine sprawl any different from physical server sprawl?[/i]

At least from our standpoint its much easier and faster to create a virtual machine. We're a fairly small company... and within 8 months we've created almost 300 virtual machines. Do I now have to update each VM? Is that no different than updating/tracking managing our hardware? For us it definitely is. We're still not sure about patching instances of an application on a monthly (or more often) basis. I'm sure larger enterprises have armies of people dedicated to this, so its not as big a problem.

We're also seeing people create instances outside of partitions, that then connect inside. So from our perspective keeping track of instances is more complex (like herding cats) versus finding an extra server on a rack.

If we treat VMs just like physical machines (as you advised) I think our patch cycle just got ten times more complicated? Please advise...

Thanks for your comments.

[SecurityJunkie]

http://www.networkworld.com/newsletters/datacenter/2006/1030datacenter1.html

Here is an article on sprawl... I've only pasted the first page. You can get the whole story at the link.

Virtual servers may be too easy to deploy

Beware of virtual server sprawl

New Data Center Strategies Newsletter By Andreas M. Antonopoulos, Network World, 10/31/06

Scientists conducting experiments on addiction have shown that when mice are allowed to self-administer narcotics, such as cocaine, by pressing a lever, they will very quickly develop addictive behavior patterns - pressing that lever repeatedly, even until death by overdose. At a recent technology conference, an IT director described a very similar behavior that immediately reminded me of the addiction studies. In this case, however, the drug of choice was a virtual machine.

A virtual machine could be “spawned” with a few clicks, and, in a matter of minutes (instead of days, as with physical servers) lead to rapid sprawl of virtual machines. This comparison is only fanciful and not meant to downplay addiction, but the challenge of sprawling virtual machines is real. As companies consolidate servers to save on hardware, many are seeing an explosion in the deployment of virtual machines - often for trivial uses - replacing server sprawl with virtual server sprawl.

Unfortunately, virtual server sprawl is not as harmless as it might sound...

[kix1979]

At least from our standpoint its much easier and

faster to create a virtual machine. We're a fairly

small company... and within 8 months we've created

almost 300 virtual machines. Do I now have to update

each VM? Is that no different than updating/tracking

managing our hardware? For us it definitely is.

We're still not sure about patching instances of an

application on a monthly (or more often) basis. I'm

sure larger enterprises have armies of people

dedicated to this, so its not as big a problem.

Use Windows Software Update Services. It's free and pushes updates, it's got a GPO plug-in as well. The rest is part of your normal business processes. Virtual or not you will have the same issue.

We're also seeing people create instances outside of

partitions, that then connect inside. So from our

perspective keeping track of instances is more

complex (like herding cats) versus finding an extra

server on a rack.

So you are giving people free reign to create VMs? Again, if it were physical would you let anyone buy a server and put it in a rack? Probably not. Just because it is easy to give people control to create VMs, doesn't mean you should do it. You still need to have someone control access and manage the farm.

[kix1979]

Again, if people manage their environment this is not an issue. That article talks about people that I call lazy. If you manage the environment and put some standard constraints in place, like who can add VMs, when is a new VM required, etc... this isn't an issue.

[SecurityJunkie]

It sounds like your team has done an excellent job keeping VMs under control AND patching server apps/OSs without breaking anything. Patching clients goes pretty smoothly... but applying patches to servers has been a whole new ball game on our end.

Thanks for your thoughts.

[SecurityJunkie]

Some great blogs are coming out on this topic-FYI.

Rational Security: http://rationalsecurity.typepad.com/blog/2007/02/virtualization_.html

Always On: http://alwayson.goingon.com/permalink/post/9944

Matasano: http://www.matasano.com/log/708/dark-reading-on-virtualization-security/