Skip navigation
VMware

This Question is Possibly Answered

1 "correct" answer available (10 pts) 2 "helpful" answers available (6 pts)
1,895 Views 25 Replies Last post: Jul 14, 2010 2:51 PM by Andy Banta RSS
1 2 Previous Next
markokobal Enthusiast 131 posts since
Aug 7, 2007
Currently Being Moderated

May 24, 2010 1:58 PM

Multipathing for Software iSCSI - Load Balancing

Hi,

 

I would like to fully understand iSCSI Load Balancing with Software iSCSI and I would like to make the best of an low-budget Software iSCSI system. Se my configuration below:

 

 

On my storage I have enabled multi path to single LUN.

 

On both ESX servers I have enabled Multipathing with Round Robin Selection Policy on the iSCSI Paths (have done everything exactly as it is shown in "iSCSI SAN Configuration Guide" for ESX 4.0 u1. And It works, I would just like to check if I got this correct: so the Load Balancing function for the software iSCSI configuration can not provide more than 1 Gb throughput for single ESX server, no matter how many connections are used for multipathing, is this corret? Because as I have seen the traffic goes like this:

 

A = path 1 (first cable to the switch and on to Storage)

B = path 2 (second cable to the switch and on to Storage)

x = traffic from ESX 01

 

traffic:

 

A B

x /

x /

x /

/ x

/ x

/ x

x /

x /

x /

/ x

...

 

So only one path is active at one time. This would mean that more then 1 Gb of throughput to the STORAGE can only be achieved when more than one ESX server is active at a time - for example 2 servers - and both are configured for Round Robin multipathing? But not guaranteed, it is a matter of order of requests, in ideal scenario when Round Robin algorithm would be so lucy to select the exact oposite paths on ESX 01 and ESX 02:

 

A = path 1 (first cable to the switch and on to Storage)

B = path 2 (second cable to the switch and on to Storage)

x = traffic from ESX 01

y = traffic from ESX 02

 

traffic:

 

A B

x y

x y

x y

y x

y x

y x

x y

x y

x y

y x

...

 

So please, am I thinkig right here?

 

Are there any other methods to get more than 1 Gb throughput on an low-budget Gb iSCSI Software system configuration? Has anyone successfuly configured LACP (Link Aggregation Control Protocol http://en.wikipedia.org/wiki/Link_Aggregation_Control_Protocol#Link_Aggregation_Control_Protocol) on vSphere 4, or it is not supported anyway?

 

Thanks for help!

 

--

Kind regards, Marko.

binoche Expert 358 posts since
Mar 24, 2008
Currently Being Moderated
1. May 24, 2010 6:22 PM in response to: markokobal
Re: Multipathing for Software iSCSI - Load Balancing

do your storage support port binding?

 

how many physical paths you have found for each lun?

 

binoche, VMware VCP, Cisco CCNA

binoche, Vmware VCP, Cisco CCNA
mittim12 Champion User Moderators vExpert 5,518 posts since
Mar 24, 2006
Currently Being Moderated
2. May 24, 2010 8:35 PM in response to: markokobal
Multipathing for Software iSCSI - Load Balancing

Interesting blogpost on utilizing multiple vmkernel portgroups and round robin, http://vmetc.com/2009/08/12/vswitch-with-multiple-vkernel-portgroups-for-vsphere-iscsi-round-robin-mpio

 






If you found this or any other post helpful please consider the use of the Helpful/Correct buttons to award points

vanak Novice 12 posts since
Jul 29, 2009
Currently Being Moderated
3. Jun 11, 2010 2:42 AM in response to: markokobal
Re: Multipathing for Software iSCSI - Load Balancing

 

Hi

 

 

 

 

 

I'm actually doing some bench, and i got the same "problem" that you encounter. Round Robin ok, multipath ok, etc etc... But i can never get more than one 1 gb throughput, in spite of a bench with 4 luns, 4 target, 4 vmkernels, 4 ip with 4 differents subnets..

 

 

 

 

 

A feeback from a vmware guru about this point will be greatly appreciated

 

 

vnk.

 

 

binoche Expert 358 posts since
Mar 24, 2008
Currently Being Moderated
4. Jun 11, 2010 2:48 AM in response to: vanak
Re: Multipathing for Software iSCSI - Load Balancing

how many physical nics you have?

 

can you make sure each of 4 luns is using different nic, different controller storage iscsi port?

 

binoche, VMware VCP, Cisco CCNA

binoche, Vmware VCP, Cisco CCNA
vanak Novice 12 posts since
Jul 29, 2009
Currently Being Moderated
5. Jun 11, 2010 3:09 AM in response to: binoche
Re: Multipathing for Software iSCSI - Load Balancing

 

In this case, im using 4 physical nics, connected to a single vswitch, with 4 vmkernel (failover policy "MAC"). Im using a Netapp, with 4 differents nic. I created 3 luns to make a bandwith test, and make a dd on this 3 luns at the same time.

 

 

On monitoring the ESX, all vmkernel and all vmknic are used. On the netapp, all four interfaces are used to. But i never get > than 80MB/s.

 

 

It seems trafic is well spreading on all available nics, but seems we cant get more than the 1gb throughput. Is this an ESX limitation ???

 

 

binoche Expert 358 posts since
Mar 24, 2008
Currently Being Moderated
6. Jun 11, 2010 3:14 AM in response to: vanak
Re: Multipathing for Software iSCSI - Load Balancing

Hi,

 

vSphere has the feature "port binding" to use all nic bandwidth, please refer to http://www.vmware.com/pdf/vsphere4/r40/vsp_40_iscsi_san_cfg.pdf,

 

binoche, VMware VCP, Cisco CCNA

binoche, Vmware VCP, Cisco CCNA
vanak Novice 12 posts since
Jul 29, 2009
Currently Being Moderated
7. Jun 11, 2010 3:22 AM in response to: binoche
Re: Multipathing for Software iSCSI - Load Balancing

Well, i've also done the port binding

binoche Expert 358 posts since
Mar 24, 2008
Currently Being Moderated
8. Jun 11, 2010 3:26 AM in response to: vanak
Re: Multipathing for Software iSCSI - Load Balancing

did you see any iscsi failure messages in vmkernel log?

 

binoche, VMware VCP, Cisco CCNA

binoche, Vmware VCP, Cisco CCNA
Svedja Enthusiast 85 posts since
Jun 22, 2006
Currently Being Moderated
9. Jun 11, 2010 3:30 AM in response to: markokobal
Re: Multipathing for Software iSCSI - Load Balancing

What kind of connections setup is between your switch and the storage?

 

If you have ie etherchannel aggregate/"trunk" it will loadbalance on IP (default I think) and you will not get more that 1Gb/s  due to the fact that ESX uses one IP an and the storage one IP and the etherchannel will only send the traffic down one of the paths.

 

 

 

But if you have two single-IP connections between the switch and storage then the problem is probably not there.

vanak Novice 12 posts since
Jul 29, 2009
Currently Being Moderated
10. Jun 11, 2010 3:34 AM in response to: Svedja
Re: Multipathing for Software iSCSI - Load Balancing

 

I got no significative error in the vmkernel log.

 

 

And i tried for the network between sw & storage with a direct attached connection. First i let them connected on a switch, but i didn't use any kind of etherchannel.

 

 

I connected them directly to be sure there were no problem with the switch...

 

 

 

 

 

'But if you have two single-IP connections between the switch and storage

then the problem is probably not there."

 

 

 

 

 

I do not really understand, but 4 nics of the storage with 4 differents ip on differents subnet are actually directly connected to the ESX.

 

 

rogard Hot Shot 265 posts since
Mar 24, 2006
Currently Being Moderated
11. Jun 11, 2010 4:15 AM in response to: vanak
Re: Multipathing for Software iSCSI - Load Balancing

Which iscsi software are you using on your storage device?

binoche Expert 358 posts since
Mar 24, 2008
Currently Being Moderated
12. Jun 11, 2010 6:18 AM in response to: vanak
Re: Multipathing for Software iSCSI - Load Balancing

jumbo frame is helpful here? can you enable it and have a retest?

 

binoche, VMware VCP, Cisco CCNA

binoche, Vmware VCP, Cisco CCNA
vanak Novice 12 posts since
Jul 29, 2009
Currently Being Moderated
13. Jun 11, 2010 7:45 AM in response to: rogard
Re: Multipathing for Software iSCSI - Load Balancing

Hmm... My storage device is a netapp... So its the ISCSI from DataOnTap.

vanak Novice 12 posts since
Jul 29, 2009
Currently Being Moderated
14. Jun 11, 2010 7:45 AM in response to: binoche
Re: Multipathing for Software iSCSI - Load Balancing

Test with jumbo frames : same thing.

Bookmarked By (0)

Share This Page

Communities