VMware

This Question is Possibly Answered

1 "correct" answer available (10 pts) 2 "helpful" answers available (6 pts)
1 2 3 4 Previous Next 54 Replies Last post: Feb 7, 2008 6:57 AM by Texiwill   Go to original post

Re: ESX 3.0.2 Service Console Security Issue

15. Nov 11, 2007 3:55 PM in response to: Texiwill
Click to view biniam's profile Novice 18 posts since
Jun 19, 2007

Hi Edward,

Could you please check the following figure if you could see if this will work? The majority of the design was based on your book. I am not sure how can I implement the DMZ ISCSI and is this going to work with vMotion?
Biniam

Attachments:

Re: ESX 3.0.2 Service Console Security Issue

16. Nov 12, 2007 6:12 AM in response to: biniam
Click to view Texiwill's profile Guru 10,236 posts since
Jan 13, 2004
Hello,

Assuming the DMZ VMs will not use iSCSI Initiators internally then you will not need the DMZ iSCSI Portgroups. These are not necessary as the Storage presentation is to the ESX server and not the VMs. So you only need one set of iSCSI networks. These are unique and no VM even knows the Backend storage, it just sees SCSI drives. SO those are not needed.

The only time those would be needed, is if the VMs will run iSCSI initiators. I would not have DMZ VMs do this if you only have one iSCSI server, networking being what it is, I would not let my DMZ VMs access anything but themselves, or present RDMs to them for larger storage areas. Granted some DMZ VMs have to access internal hosts....

General networks with DMZs look like this:

Internet <-> Firewall <-> DMZ <-> Firewall <-> Internal network

THe idea is to limit direct access to the internal network. Sometimes a DMZ machine must query an internal DB for some bits of data. Not often but this will allow for that in a protected fashion. The external firewall is slightly more open than the internal, perhaps allowing web traffic where the internal firewall does not, etc.

And not:

Internet <-> Firewall <-> DMZ
Internet <-> Firewall <-> Internal Network

This just allows at least 2x attack points. Force all traffic through the DMZ so that you can monitor everything within there. And have a secondary firewall to add more protection. It is sometimes quite hard to bust through firewalls but not impossible.

Best regards,
Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education. Available on Rough Cuts at http://safari.informit.com/9780132302074

Re: ESX 3.0.2 Service Console Security Issue

17. Nov 14, 2007 5:54 AM in response to: Texiwill
Click to view biniam's profile Novice 18 posts since
Jun 19, 2007

Hello,

I have only one ISCSI and with software Initiators. We will have our portal and Front End exchange server on dmz and i need these two servers with high availablity; vMotion

Regards

Biniam

Re: ESX 3.0.2 Service Console Security Issue

18. Nov 14, 2007 11:35 AM in response to: biniam
Click to view Texiwill's profile Guru 10,236 posts since
Jan 13, 2004
Hello,

So your VMs will be using Windows iSCSI initiators or will ESX be using the software initiator? This is the real question. I hope you imply the later as the former has several security issues.

Best regards,
Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education. Available on Rough Cuts at http://safari.informit.com/9780132302074

Re: ESX 3.0.2 Service Console Security Issue

19. Nov 20, 2007 3:21 PM in response to: Texiwill
Click to view biniam's profile Novice 18 posts since
Jun 19, 2007

Hello,

I am using ESX software initiator.

Regards

Biniam


Re: ESX 3.0.2 Service Console Security Issue

20. Nov 21, 2007 6:02 AM in response to: biniam
Click to view Texiwill's profile Guru 10,236 posts since
Jan 13, 2004
Hello,

Then there is no need for an extra set of 'DMZ' iSCSI ports as the storage network is from internal and the VMs have no direct access to it, therefore there is no method to get into the storage network.

Best regards,
Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education. Available on Rough Cuts at http://safari.informit.com/9780132302074

Re: ESX 3.0.2 Service Console Security Issue

21. Nov 26, 2007 2:23 PM in response to: Texiwill
Click to view biniam's profile Novice 18 posts since
Jun 19, 2007

On last question Edward,

The IP addresses that you set when you configure the SAN. Are these IP number the same when you setup vmware iSCSI?

I that possible to have the VLAN on pSwitch only?

VM VLAN on the pSwitch

vMotion VLAN on the pSwitch

SC VLAN on the pSwitch

ISCSI VLAN on the pSwitch

Regards Biniam

Re: ESX 3.0.2 Service Console Security Issue

22. Nov 28, 2007 5:12 AM in response to: biniam
Click to view Texiwill's profile Guru 10,236 posts since
Jan 13, 2004
Hello,

A VLAN to the pSwitch is called EST (External Switch Tagging) and from there you have multiple links to the ESX server. Yes this is fine as long as you also do not allow promicous mode on the pSwitch or if you do it is limited to just the IDS system.

Best regards,
Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education. Available on Rough Cuts at http://safari.informit.com/9780132302074

Re: ESX 3.0.2 Service Console Security Issue

23. Jan 26, 2008 2:37 PM in response to: Texiwill
Click to view biniam's profile Novice 18 posts since
Jun 19, 2007
Hi Edward and all,

I am back and I have all the kit to test what we discussed. I have some questions with regards to infrastructure and security.

I have installed the ESX hosts on 10.44.1.0/24 Vlan 10 also I have

2 pNIC for Service Console 10.44.1.0/24 Vlan 10
2 pNICs for VMs 10.44.2.0/24 Vlan 20
2 pNIC for vMotion 10.44.99.0/24 Vlan 99
2 pNICs for iSCSI 10.44.3.0/24 Vlan 30

I am Using EST (External Switch Tagging) with firewall controller access between the VLANs. Could you please tell me which VLANs need to see each, for VMware infrastructure to work in secure and efficient way?

Regards
Biniam

Re: ESX 3.0.2 Service Console Security Issue

24. Jan 26, 2008 4:09 PM in response to: biniam
Click to view JDLangdon's profile Master 981 posts since
Jun 30, 2006
biniam wrote:Hi Edward and all,
I have installed the ESX hosts on 10.44.1.0/24 Vlan 10 also I have

2 pNIC for Service Console 10.44.1.0/24 Vlan 10
2 pNICs for VMs 10.44.2.0/24 Vlan 20
2 pNIC for vMotion 10.44.99.0/24 Vlan 99
2 pNICs for iSCSI 10.44.3.0/24 Vlan 30

I am Using EST (External Switch Tagging) with firewall controller access between the VLANs. Could you please tell me which VLANs need to see each, for VMware infrastructure to work in secure and efficient way?


How many pNICs do you have per host?

Jason

Re: ESX 3.0.2 Service Console Security Issue

25. Jan 26, 2008 5:11 PM in response to: biniam
Click to view JDLangdon's profile Master 981 posts since
Jun 30, 2006
biniam wrote:Hi Edward and all,
I have installed the ESX hosts on 10.44.1.0/24 Vlan 10 also I have

2 pNIC for Service Console 10.44.1.0/24 Vlan 10
2 pNICs for VMs 10.44.2.0/24 Vlan 20
2 pNIC for vMotion 10.44.99.0/24 Vlan 99
2 pNICs for iSCSI 10.44.3.0/24 Vlan 30


Assuming you have eight pNICs per host here's what I would do.

1) I would use the same two pNICs for both the service console AND Vmotion and I would route all traffic over VLAN10.
2) I would install the iSCSI initiator on the ESX host server and use two pNICs and VLAN30 to route between the iSCSI SAN and the host server.
3) I would use two pNICs and VLAN20 to route traffic between all VM's and what I call "The outside world".
4) I would install the iSCSI initiator on the VM's and use two pNICS and VLAN99 to route iSCSI traffic between the iSCSI SAN and the VM's.
5) I would divide my iSCSI SAN into two sections. One section would be formatted as VMFS and would be used to store all .vmdk files. These .vmdk files would contain only the VM guest OS (C:\ drive). Assuming you are using Windows OS within the VM's, I would format the other section of the iSCSI SAN as NTFS and allocated this space to the individual VM's which would be used to store the data.

While this is not necessarily the correct way to configure this environment, it is what I would do. Attached is a .png file which outlines what I stated above.

Jason
Attachments:

Re: ESX 3.0.2 Service Console Security Issue

26. Jan 27, 2008 10:01 AM in response to: JDLangdon
Click to view biniam's profile Novice 18 posts since
Jun 19, 2007
Hi Jason,

Thanks for your input, very interesting solution, I have 10 pNICs in total. Two planned to be used for DMZ. I am using EMC NS20 which has multi-protocol support (ALL-IN-ONE box NAS/SAN, ISCSI/FIBRE). So we have CIFS for simple data. We also use RAW for our exchange and SQL logs.

1. In your recommendation you have putted the VM on two groups. Are these similar to DMZ and trusted networks?
2. Is that good practice to allocated four pNIC to VM?
3. What is the purpose of separating VM as the raw and vmfs data usage? We have exchange server that use the data and log on raw and the OS on VMFS.
4. My staff went to Deploy, Secure and Analyse training and there was a lot of discussion on security on mixing vMotion with SC or ISCSI. Do you thing mixing SC with vMotion bring some security concern?
Regards
Biniam

Re: ESX 3.0.2 Service Console Security Issue

27. Jan 27, 2008 10:10 AM in response to: biniam
Click to view Texiwill's profile Guru 10,236 posts since
Jan 13, 2004
Hello,

The bottom looks great.

2 pNIC for Service Console 10.44.1.0/24 Vlan 10
2 pNICs for VMs 10.44.2.0/24 Vlan 20
2 pNIC for vMotion 10.44.99.0/24 Vlan 99
2 pNICs for iSCSI 10.44.3.0/24 Vlan 30

I am Using EST (External Switch Tagging) with firewall controller access between the VLANs. Could you please tell me which VLANs need to see each, for VMware infrastructure to work in secure and efficient way?


The Service Console must partiticpate in the iSCSI network in order for authentication protocols to work. This is required whether you use hardware iSCSI- HBAs or network cards. So your iSCSI device must see your service console and visa versa.


Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

Re: ESX 3.0.2 Service Console Security Issue

28. Jan 27, 2008 12:08 PM in response to: biniam
Click to view JDLangdon's profile Master 981 posts since
Jun 30, 2006
biniam wrote:Hi Jason,
1. In your recommendation you have putted the VM on two groups. Are these similar to DMZ and trusted networks?
2. Is that good practice to allocated four pNIC to VM?
3. What is the purpose of separating VM as the raw and vmfs data usage? We have exchange server that use the data and log on raw and the OS on VMFS.
4. My staff went to Deploy, Secure and Analyse training and there was a lot of discussion on security on mixing vMotion with SC or ISCSI. Do you thing mixing SC with vMotion bring some security concern?

1. VLAN99 should be a trusted network in that the only devices on this VLAN should be the VM's that need access to the iSCSI, the EMCns20, and whatever PC's are used to manage this environment.

2. I haven't allocated four pNICs to a VM, I've attached two pNIC's to the vSwitch that controls VLAN99 (iSCSI) and 2 pNICs to the vSwitch that controls VLAN20. You could have just as easily routed both VLAN20 and VLAN99 over the same tow pNICs but in my environment, the iSCSI devices are on a private, non-route able network.

3. By storing the VM data on the iSCSI in native formate (non-vmdk) you have to option of attaching it to a physical server if the VM ever crashes. Also, while I have not experimented, I have been told by an iSCSI vendor that you should get better performance when using the iSCSI initatior inside of the guest OS to access the data.

4. If your staff attended the "Deploy, Secure and Analyse" training session, tell them to look at pages Module1 - Lesson 3 and Module2 - page 12. In my environment, all service console and Vmotion traffic will be on a secured VLAN that is only accessible from my laptop, an ESX host server, or the VirtualCenter server. This will remove all concerns about having SC and Vmotion on the same network.

Jason

Re: ESX 3.0.2 Service Console Security Issue

29. Jan 28, 2008 5:45 AM in response to: JDLangdon
Click to view Texiwill's profile Guru 10,236 posts since
Jan 13, 2004
Hello,

If you want your VMs to see your iSCSI network then they also must participate in this. I however recommend that the VM access and the Host access be segregated some how. If you only have your VMs access the iSCSI server and you have no other remote storage then vMotion will not work. Consider this:

VM only access to iSCSI using iSCSI initiator.... Where does the boot disk live? On local storage.... So therefore no vMotion.

VM and Host access to iSCSI using initiator within VMs and iscsi target within Host.... Could a VM then see any of the VMFS' on the remote storage? This depends on how the LUNs are presented, how the iSCSI server segregates such traffic. Does your iSCSI server support multiple network links? If so you may want to use a set of links for hosts and a set of links for initiators. If the Host uses iSCSI then an SC connection must be part of this network. How would such access impact disk performance at the Host level. Or impact the VM networks. With only 10 pNICs and 2 assigned to a DMZ, you could do this but you compromise security or redundancy on other networks. See previous discussions. If you have another set of pNICs then this would be a safe option. iSCSI is all about performance moving through the network to have VMs using iSCSI initiators they would need the network bandwidth for disk access. You can get the same backup/restoration behavior using Raw Disk Maps as well, which can use the Host based iSCSI target code. Either method works.

SC and vMotion in a truly secure environment should never be over the same vSwitch or pNICs. Yes people do it, but it really is insecure. However, this is mostly a trust related issue. If your administrators are allowed to login to the VMs then it is a moot issue as they have the credentials however if they are not allowed to login then allowing them access to the vMotion network could allow someone to gain those credentials quite easily just by capturing the data during a vMotion. Or if your SC is hacked and you are using remote login services a hacker can do the exact same thing. For full protection make it a private link.

4. If your staff attended the "Deploy, Secure and Analyse" training session, tell them to look at pages Module1 - Lesson 3 and Module2 - page 12. In my environment, all service console and Vmotion traffic will be on a secured VLAN that is only accessible from my laptop, an ESX host server, or the VirtualCenter server. This will remove all concerns about having SC and Vmotion on the same network.

Absolutely, lock down which machines have access to the Service Console, but nothing but an ESX server should have access to vMotion. If this laptop or any other host was compromised it can be setup to grab the memory image as it travels across the network during a vMotion and thereby gather credential data. I would not risk this.


Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

VMware Developer

SDKs, APIs, Videos, Learn and much more in the Developer community.

Learn More

Developer Sample Code

Increase your developer productivity with VMware API sample code.

Learn More

VMworld Sessions & Labs

Online access to the latest VMworld Sessions & Labs and online services.

Learn more

Purchase PSO Credits Online

Purchase credits to redeem training and consulting services online.

Buy Now

Community Hardware Software

View reported configurations or report your own.

Learn More

VMware vSphere

Come witness the next giant leap in virtualization.

Register Today

Communities