Hello,
There is no internal documentation on this, it is the nature of the iptables rules that are in place. Every time you create a service console portgroup on a vSwitch you are not creating a new service console, but tying into the existing service console. The existing service console protects all input and output devices using iptables. A good iptables reference is: "LINUX iptables Pocket Reference" from O'Reilly and Associates. Also, you can look at www.netfilter.org.
Since there is only one instance of a service console, all you are doing by providing more than one Service Console port group is making more ethernet interfaces on the service consoles, these are the vswif devices.
You have not really found a bug in ESX, but the fact that there are 1001 ways to do networking with LINUX. Consider this, would you rather have a specific rule that says everything allowed on vswif0 and that vswif1 is absolutely wide open? If they were to implement interface level blocking on the default iptables rules, that is generally what results. Or would you like absolutely nothing on vswif1? THat is the other option. Note that the VIC and esxcfg-firewall commands do not know anything about the devices in use only the rules to be used. This is a General setup that meets most peoples needs.
So, for iptables you have 3 default chains on the primary table. Those chains are INPUT, FORWARD, and OUTPUT. INPUT covers everything coming in, FORWARD is for forwarding between interfaces and OUTPUT is for anything going out. There is also a nat, mangle, and some other tables. None of these are used by ESX and should not be used by ESX. For INPUT, the general rules are written like:
iptables -A INPUT -p with the state argument and the --state argument of NEW. This is matching on the state of a new connection only.
iptables -A INPUT -m state --state NEW -p tcp -m tcp --dport 902 -j ACCEPT
You can also have a sourcePort in use, a sourceAddr, destinationAddress, and a host of other elements at work including which interface upon which this rule applies. The interface lock down is NOT part of the ESX firewall rules, nor should it be as any vswif should be able to use all its services. If you want to change this then you will need to play with the firewall rules.
IPtables is extremely complex and incredibly powerful. To fully understand it you will need to do some reading of the manual page, a reference and play with it.
In addition there is only one default gateway. THere can not be more than one default gateway, however, it is trivially easy to add a gateway that is specific to a given subnet if you are using more than one vswif device. There is no cross routing between vswif0 and vswif1 if you look at your network routes (netstat -rn) you will see the default route plus a node specific route for your vswif1 device. Anything coming in on vswif1 will go back out vswif1.
As for your comment about it ties the vswitch portgroup to the Service Console, you are absolutely correct. If you are not using VLANs the traffic is not segmented. If you are allowing promiscuous mode ethernet devices on your vswitch then things are even worse. Virtual Networking has some basic rules to follow and you are crossing some of them.
1) vMotion should be its own private vSwitch or portgroup
2) Service Console should be its own private vSwitch or portgroup. This includes all possible service consoles ports, not just the primary (use VLANS to segment networks on the SC vSwitch). This is your Administrative network which should also be firewalled. VC is on this network. As is any management tool whether running as a VM or otherwise.
3) Virtual Machines should be their own private vSwitch or portgroups
4) Each should have redundancy (2 pNICS per vSwitch)
5) VLANs should be used when dealing with reduced number of ports
6) vMotion should be its own private network including routing if you are to doing this. Only ESX vMotion ports should be allowed on this network.
7) iSCSI requires that a service console vswif participate in its network, yet iSCSI for VMFS should be separated from the VM network.
8) NFS should be separate from the VM Network.
9) Never enable promiscous mode on your vswitches but you can for specific portgroups that only contain your IDS... No other VM should be in this portgroup.
10) Using VLANs will lower the amount of pNIC you require.
Ideally I like having a minimum of SIX physical network adapters so that I have everything segmented nicely even if I do use VLANs as well. This way I gain redundancy and functionality.
You have not found anything that is a problem with ESX, you have found that there are 1001 ways to do networking within LINUX and that some are better than others. Also you have found that generic firewalls like what ESX uses is only useful in generic cases, you have a SPECIFIC need and will have to adjust things to meet that SPECIFIC need. The iptables firewall is only part of the answer, you can also use /etc/hosts.allow and /etc/hosts.deny to implement tcpwrappers, or implement tcpwrappers in /etc/xinetd.conf; these are just some of the ways to limit access based on IP and or network and are a 2nd line of defense for all LINUX machines. Changing the default ESX security rules are not for the faint of heart, you can really mess things up extremely easily. Nor do I expect most people to do so, however, if they do not understand how the rules work, then they could easily mess up.
One thing you can do is to deny access to anything on vswif1 using the following in /etc/rc.d/rc.local
/sbin/iptables -I INPUT 1 -i vswif1 -j DROP
Which would insert this rule at the top of the chain denying all access to vswif1 from outside. Adding in specific rules requires even more tweaking and you need to really understand firewalls to do this plus iptables.
Then to allow port 902 or anything to come back in on vswif1 that you specifically start you can use:
/sbin/iptables -I INPUT 1 -i vswif1 -m state --state NEW -p tcp -m tcp --dport 902 -j ACCEPT
/sbin/iptables -I INPUT 1 -i vswif1 -m state --state ESTABLISHED,RELATED -j ACCEPT
You must use the 3 lines in the order given to get only port 902 and what you have started to go out vswif1.
If anyone does not understand these concepts or know how to implement them, then I suggest taking a LINUX Security Class, There is a very good version of this presented at each Linux World as well as the one SANS puts on. Also my upcoming book has a chapter just on security and how to implement some of the concepts I discussed. The last option is to hire a consultant that does understand these concepts.
How ESX 3i changes these above, I am unsure as I have yet to play with ESX 3i. I imagine it will to a certain extent but hopefully not very much. A firewall for the CLI is extremely important.
Best regards,
Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education.
