Best Practice for File Server with ISCSI SAN

[azev]

What is the best practice of setting up file server using ESX3.5 and ISCSI san ?

My current setup is 2 ESX 3.5 host with HA and DRS enabled on VC2.5.

One ISCSI san box with 3 LUN hosting about 20 VM's in the cluster.

I have about 2Terra left in the SAN that I would like to setup as file storage.

I wonder if its better to map the LUN to the ESX box or use Microsoft ISCSI initiator on the file server ?

Both SAN and ESX has many NIC that is not currently being use, so I can setup a dedicated connection just for this.

Suggestion please..

Thx

[Texiwill]

Hello,

EIther method will work. There have however been posts that state that an iSCSI Initiator within the VM is faster, by how much is uncertain.

One thing to be concerned about is your Backup and DR plans. How would using one over the other fit in with these plans?

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[eabeukes]

There have been a few threads on here that seem to point to the MS iSCSI initiator being a bit quicker. Some of the newer Intel cards can also do basic TCP offloading to speed things up. Mapping the LUN to the ESX will possibly give you more flexibility to dynamically grow the disk space on the VM, but going direct via the MS iSCSI would give you faster access.

[azev]

Thanks for the info everyone, I did just like what you guys suggested, and used microsoft ISCSI initiator on our windows 2008 file server and get great results.

a sequential transfer of 4GB files from the server to my vista desktop yields up to 450Mbps. One thing of concern is that the VM CPU utilizations are pegged to amost 100% during this transfer.

I wonder what it's going to look like when I have many users accessing the server at the same time. Do you guys think this would be a good candidate for a SMP VM ? or should I just increase the VM memory allocation ? right now it has 1GB memory on a Windows 2008 no hyperv version.

[raadek]

Hi azev,

High CPU utilisation is a flip side of the software iSCSI - vCPU is doing iSCSI (& the entire TCP/IP stack) processing.

My thoughts:

a) adding more memory won't help, it's just CPU

b) multi-vCPU VMs are not the best thing you can have - avoid them if possible; I'd rather increase this VM CPU reservation and/or limit

c) you may test VM-level iSCSI initiator vs. ESX-level if your VM is still struggling with CPU resources - although ESX-level iSCSI processing still requires some CPU cycles, so it's a kind of a chicken & egg dilemma...

Rgds.

[eabeukes]

Windows 2008 is quite hefty on CPU anyway so SMP VM would be a good idea however as above pushing the resources up may work better. Check the VM itself during transfers to see what is using all the CPU. Do you have the VMware Tools installed? The other 'best practice' idea would be to use a seperate dedicated NIC (something like one of the Intel Pro/1000 server NIC's) for iSCSI traffic. If you find your CPU use is still very high then QLogic do some good iSCSI HBA's, but at a price...!

Message was edited by: eabeukes - spelling

[raadek]

Dedicated NICs (ideally more than one for resiliency) are always a good idea.

Yet iSCSI HBA won't help when using software initiator within the VM - it sees just a virtual NIC, not an HBA.

Rgds.

[eabeukes]

The iSCSI HBA will offload all CPU intensive tasks and all the ESX system will see is a LUN presented. It will show up as an HBA (possibly even as a FC/SCSI HBA) in ESX.

[5474]

I noticed that too on our 2003 file server while I was testing with Iometer. Enabled Jumbo frames and it dropped about 30%

[azev]

Unfortunately cisco switches we used in this VM environtment are not Jumbo frame capable.

But, if jumbo frame really help, I guess I can see getting a cheap dell switch with jumbo frame enabled.

Or, maybe just a direct crossover cable from the VM to the ISCSI box.

Anyhow, how do you enable jumbo frame in my scenario ?

What do you guys think ?