Contributor
Contributor

Good plan?

Hi,

I'm looking to setup a new iSCSI SAN with 3 ESX servers and I was hoping for some feedback on my plan and if anyone can help with some questions!

SAN:

  • 8x 400GB disks. My plan is to create a RAID5 array of 2.8TB. I'll then create a 2TB and an 800GB LUN and assign that to the ESX servers.

  • The SAN has 2 RAID modules with 3 NICs in each. I plan to use 2 switches for the SAN connecting each module to both. Should I look at aggregating/teaming these connections?

ESX:

  • Each server has 8 NICS. The NICS will be used in 4 vSwitches (3x iSCSI, 2x VM network & SC, 2x VMkernal, 1x SC).

  • I'll create a separate subnet for vmkernal and iSCSI. Should I look at using jumbo frames or flow control?

I'd be interested to hear view on whether this is a good plan!

Cheers,

H

0 Kudos
6 Replies
Enthusiast
Enthusiast

Although ESX supports 2TB LUNS, you may be better off creating 400/500GB LUNS.

Neil VCP
0 Kudos
Hot Shot
Hot Shot

Hello,

You might be better to look at the number of VMs you are going to have and thier Roles as well. As suggested earlier, it's better to create LUNs between 400 and 500 gig. Raid 10 or Raid 5!!! That's depends on the VMs you have and kind of arcticture you will be having. For examble, 350 gig LUN that's Raid 10 its porpuse only to host a C:\ Drive of the VMs and the number of VMs I have in this LUN around 11 VMs. This gives better Read/Write performance and faster booting.

If you have VMs that require to host Data, such as FileServer, ect ect, it's better to create another LUN that's Raid 5 and assign a VMDK Disk from this LUN to the VMs that require Data Drive. Offcourse, the C:\ Drive has to be mapped from LUN that's Raid 10.

Also, your C:\ Drive of the VM shouldn't be exceed 20 gig * by number of VMs + free LUN space for overhead and SnapShots. = your LUN Size.

Regarding your SAN connectivity, it's better to provide a spearate pSwitches to connect your SAN to be on it's own seperate network, this will give you better Security and better performance. But make sure that the iSCSI VMkernel PortGroup seprated of your vMotion and Service Console PortGroups.

Regarding your Networking connectivity, it's better to seperate each network from each other. Since you have 8 pNICs, If I were you I would do the following;

  • vmnic0 & vmnic1 --> vSwitch0 " Service Console PortGroup"

  • vmnic2 & vmnic3 --> vSwitch1 " vMotion PortGroup"

  • vmnic4 & vmnic5 --> vSwitch2 " VMs PortGroup"

  • vmnic6 & vmnic7 --> vSwitch3 " iSCSI - VMKernel PortGroup"

If those pNICs comes from seprate PCI or Mezzanien Cards. Then think of Cross-Over PCI to give you better redundancy. Such as 0 -> 7, 1 -> 3 ect ect.....

Make sure the Service Console Network i behind a Firewall that seprate between your Production and ESX Network. So, this will give your better security and limited ports opened between your ESX and Production Networks. If you want to manage your environment from production, just you have to use the Vi-Client to access the Hosts Directly or access the vCenter via Vi-Client. Since you will have a Seperate ESX Network, it's better to put the vCenter beside your hosts, so all the vCenter communication to your hosts will be faster and secured witing it's own network.

You can build a Virtual Firewall to have one vNic on ESX Network side, that's Internal and one vNIC on your Production Network Site, that's External. Create the allowed ports to be accessed and you are done.

Seprate the vMotion on it's own network, because the VMs movements are in clear text. So, hacker can sniff the data quickly. And the same applies on the iSCSI Network, put this has to be on a seprate pSwitch which will seprate the vMotion and iSCSI Networks off each other.

If you have got only two pSwitches that cascaded use VLANs to seprate the networks.

Best Regards,

Hussain Al Sayed

If you find this information useful, please award points for "correct" or "helpful".

Best Regards, Hussain Al Sayed Consider awarding points for "correct" or "helpful".
Contributor
Contributor

Thanks for your excellent response - much appreciated.

I was wondering if it's worth using RAID 10 on iSCSI (10k) - i.e. as the disk access speeds are relatively slow compared with FC? I also wondered if people use switch port teaming/aggregating for the SAN and if it's recommended.

Cheers,

H

0 Kudos
Hot Shot
Hot Shot

Hello,

Check this out, it might help you.

hxxp://blogs.vmware.com/performance/

hxxp://www.experts-exchange.com/Software/VMWare/Q_23603491.html

hxxp://communities.vmware.com/thread/74311

hxxp://blog.dnfcorp.com/2009/02/19/iscsi-performance-myths-explained/

hxxp://blog.dnfcorp.com/2009/02/04/tutorial-how-do-i-set-up-my-iscsi-san-with-vmware-for-optimal-performance/

Best Regards,

Hussain Al Sayed

If you find this information useful, please award points for "correct" or "helpful".

Best Regards, Hussain Al Sayed Consider awarding points for "correct" or "helpful".
0 Kudos
Contributor
Contributor

Thanks again. Very helpful.

I understand that creating multiple LUNs and tweaking the path to them in ESX can give you a performance benefit.

What I'm unclear about is the configuration of the arrays. Is there any benefit in creating 2 separate disk arrays (and distributing the LUNs between them) as opposed to 1 big array containing all LUNs?

Cheers,

H

0 Kudos
Leadership
Leadership

Hello,

For iSCSI using multiple LUNs presented through multiple ethernet links is better than presenting all through one link and just having failover mode.


Best regards, Edward L. Haletky VMware Communities User Moderator, VMware vExpert 2009, DABCC Analyst[/url]
Now Available on Rough-Cuts: 'VMware vSphere(TM) and Virtual Infrastructure Security: Securing ESX and the Virtual Environment'[/url]
Also available 'VMWare ESX Server in the Enterprise'[/url]
[url=http://www.astroarch.com/wiki/index.php/Blog_Roll]SearchVMware Pro[/url]|Blue Gears[/url]|Top Virtualization Security Links[/url]|Virtualization Security Round Table Podcast[/url]

--
Edward L. Haletky
vExpert XII: 2009-2020,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos