Security usage scenario - thoughts please

[RomeoJava]

Hello,

As you guys seem to be the best people to ask, I was wondering if you can help me think through a scenario and look at the security risks:

The scenario involves two domains with differing security sensitivity, and the requirement for a barrier between them. It will only let a certain type of traffic through after content checking e.g. e-mail. The current solution would be a physical high-side firewall, a machine in the middle doing content checking, and then a physical low side firewall. The only way users interact with this is by sending e-mails to the network interfaces at each side.

If this were replaced with say ESX server running three virtual machines: two firewalls and a content checker, what do you perceive the risks to be? This is assuming that once the device is set up, only trusted admins will have access to the Guest OSs.

My thinking so far is that the VM escaping techniques proposed by some, if applicable to ESX (which I don't think they are) are not applicable in this scenario because users aren't running code on the guest OSs. One area that I'm not sure has been discussed but I'm interested in is, theoretically there could be a weakness in the device drivers used by the host OS. If, and that is a big if, there were a flaw in the NIC device driver, then I assume that a buffer overflow could lead to control of the hypervisor, and in turn all Guest OSs?

Any thoughts on this scenario welcome.

Regards,

Rj

[michael_40catbi]

I thought a picture would help, see below

Risks are:

1. Misconfiguration of either the high side "H" or low side "L" firewalls

2. Misconfiguration of the content filter

3. Misconfiguration of the hypervisor

4. Exploit of any of the above

You should consider running an IDS/IPS within ESX to enforce compliance with your policy, i.e. the nature and direction of traffic between these systems and networks. It is also a good idea to run HIDS on ESX. In this specific application, you should regularly send bad/good messages through the system and monitor that your filter is working as expected.

In a high security environment, assurance requires continuous validation. Whatever tools you use to monitor the behavior and configuration of this environment, make sure the security administrator who monitors the "validation" is not the same person responsible for the implementation of these controls. If your shop is too small to make this two people, then ensure that alerts for errors or changes to the system go to a third-party or manager who does not have admin access to the ESX system.

Compromise of the Hypervisor is not automatically catastrophic as long as you have a good process for detecting the event. Again, use standard controls on the external networks connected to your ESX system and monitor for unusual or forbidden traffic. For instance, you should strictly limit SSH and VIC traffic, so monitor these ports for illicit communication.

These discussions should help you:

Do you / How do you AUDIT your compliance to security policy

http://www.vmware.com/community/thread.jspa?threadID=98451&start=0&tstart=0

Interesting security articles on VM's escaping into the host OS.

http://www.vmware.com/community/thread.jspa?threadID=96234&start=0&tstart=0

ESX VI3 Host with Nics on both Internal and DMZ Networks

http://www.vmware.com/community/thread.jspa?threadID=96550&start=0&tstart=0

ESX Guest monitoring

http://www.vmware.com/community/thread.jspa?threadID=89526&start=0&tstart=0

Security concerns on corporate domain

http://www.vmware.com/community/thread.jspa?threadID=86098&start=0&tstart=0

\----

---

\ || ESX ||

\--

---

|| || \--

---

\ | | || VM1 VM2 VM3 || | |

\ | SENSITIVE| || || | PUBLIC |

\ | | || H C || | |

\ | | || H O || | |

\ | NETWORK | || H N L || | NETWORK |

\ | | || H T L || | |

\ | ----

---

| H E F L |----

---

|

\ | | || H N I L || | |

\ | | || H T L L || | |

\ | | || H T L || | |

\ | | || H E L || | |

\ | | || H R L || | |

\ | | || || | |

\--

---

\

---

\--

---

Michael

[RomeoJava]

Michael,

Thanks a lot for your reply, especially your diagram! I'm particularly interested in the 4th Risk you mentioned, exploitation, as I am assuming that the first three are n/a in my case as they \*should be* be implemented correctly.

I'm still a little confused about the Linux / vmkernel structure even after reading a fair amount in an ESX admin guide. If my understanding is correct, ESX Server is still relying on Linux device drivers to communicate with all the hardware but additionally on the vmkernel which needs to understand the hardware to communicate and take control of that driver. Do you have any thoughts on this?

Also you mentioned SSH and VIC traffic. What is VIC traffic? did you mean VNC? Although I haven't got to that chapter in the ESX admin guide, I assume all VMs can be directly accessed from the server so that no remote control software is required on the VMs and no remote control traffic is available on the network?

Anyway, thanks again for your reply.

Rj

[Texiwill]

Hello,

VIC traffic is over port 902/903 and is used to administer/configure the ESX server. In essence, you would at least have 2 ESX servers, so that you could use vMotion and the HA/DRS features of VI3. vMotion allows for the movement of VMs between ESX servers. DRS is the automation of this feature based on rules. And HA will bring up VMs on an alternative host just in case one Host dies. vMotion requires some form of remote storage and that all vSwitches in use are connected to some physical device.

Now: Host OS. The Host OS is ESX which is the vmkernel fall intents and purposes. THis black box kernel has its own drivers, etc and runs first. It creates a VM on boot that is used as the Service Console. The Service Console allows you to administer your network and it runs within the vmkernel, not the other way around in VI3.

Escaping a VM to the service console on ESX is only possible with a seriously flaws (non-default) Service Console configuration: You would have to make the Service COnsole a fileserver to make this even remotely possible and that is SERIOUSLY not recommended. There is currently no known way to escape to the vmkernel.

As for your desired configurations: YOu would have at least 6 networks involved:

1- Service Console/Administrative network locked down by network and host. This is an internal sensitive network.

2-vMotion network. This should be a private network/VLAN with no other hosts on it besides ESX servers.

3-Highside network (2 links for redundancy)

4-Internal network (to vMotion and use multiple hosts, this should be a private network between the VMs spanning multiple hosts) (2 links for redundancy)

5-Lowside network (2 links for redundancy)

6-Remote storage network (either SAN, iSCSI, or NFS) (2 links for redundancy)

If I was to implement this, I would use a mix of physical security and properly configure each network segment within ESX.

If anything in these network segments is misconfigured then you have the possibility of a breach in security. Having a misconfigured Administrative network/Service Console/vMotion/Storage network will open you up to a fair amount of possible attack paths.

Running another VM within the VM network would allow for an IDS, which would be extremely valuable.

Best regards,

Edward

[RomeoJava]

Edward,

Thanks for your response. What network adapters on your list have the VIC traffic on? Can I assume the SC network only?

Regards,

Rj

[Texiwill]

Hello,

That is correct, only the Administrative network has VIC traffic and hence only the SC.

Best regards,

Edward