Hi,
I would like to fully understand iSCSI Load Balancing with Software iSCSI and I would like to make the best of an low-budget Software iSCSI system. Se my configuration below:
On my storage I have enabled multi path to single LUN.
On both ESX servers I have enabled Multipathing with Round Robin Selection Policy on the iSCSI Paths (have done everything exactly as it is shown in "iSCSI SAN Configuration Guide" for ESX 4.0 u1. And It works, I would just like to check if I got this correct: so the Load Balancing function for the software iSCSI configuration can not provide more than 1 Gb throughput for single ESX server, no matter how many connections are used for multipathing, is this corret? Because as I have seen the traffic goes like this:
A = path 1 (first cable to the switch and on to Storage)
B = path 2 (second cable to the switch and on to Storage)
x = traffic from ESX 01
traffic:
A B
x /
x /
x /
/ x
/ x
/ x
x /
x /
x /
/ x
...
So only one path is active at one time. This would mean that more then 1 Gb of throughput to the STORAGE can only be achieved when more than one ESX server is active at a time - for example 2 servers - and both are configured for Round Robin multipathing? But not guaranteed, it is a matter of order of requests, in ideal scenario when Round Robin algorithm would be so lucy to select the exact oposite paths on ESX 01 and ESX 02:
A = path 1 (first cable to the switch and on to Storage)
B = path 2 (second cable to the switch and on to Storage)
x = traffic from ESX 01
y = traffic from ESX 02
traffic:
A B
x y
x y
x y
y x
y x
y x
x y
x y
x y
y x
...
So please, am I thinkig right here?
Are there any other methods to get more than 1 Gb throughput on an low-budget Gb iSCSI Software system configuration? Has anyone successfuly configured LACP (Link Aggregation Control Protocol http://en.wikipedia.org/wiki/Link_Aggregation_Control_Protocol#Link_Aggregation_Control_Protocol) on vSphere 4, or it is not supported anyway?
Thanks for help!
--
Kind regards, Marko.
do your storage support port binding?
how many physical paths you have found for each lun?
binoche, VMware VCP, Cisco CCNA
Interesting blogpost on utilizing multiple vmkernel portgroups and round robin, http://vmetc.com/2009/08/12/vswitch-with-multiple-vkernel-portgroups-for-vsphere-iscsi-round-robin-mpio
If you found this or any other post helpful please consider the use of the Helpful/Correct buttons to award points
Hi
I'm actually doing some bench, and i got the same "problem" that you encounter. Round Robin ok, multipath ok, etc etc... But i can never get more than one 1 gb throughput, in spite of a bench with 4 luns, 4 target, 4 vmkernels, 4 ip with 4 differents subnets..
A feeback from a vmware guru about this point will be greatly appreciated ![]()
vnk.
how many physical nics you have?
can you make sure each of 4 luns is using different nic, different controller storage iscsi port?
binoche, VMware VCP, Cisco CCNA
In this case, im using 4 physical nics, connected to a single vswitch, with 4 vmkernel (failover policy "MAC"). Im using a Netapp, with 4 differents nic. I created 3 luns to make a bandwith test, and make a dd on this 3 luns at the same time.
On monitoring the ESX, all vmkernel and all vmknic are used. On the netapp, all four interfaces are used to. But i never get > than 80MB/s.
It seems trafic is well spreading on all available nics, but seems we cant get more than the 1gb throughput. Is this an ESX limitation ???
Hi,
vSphere has the feature "port binding" to use all nic bandwidth, please refer to http://www.vmware.com/pdf/vsphere4/r40/vsp_40_iscsi_san_cfg.pdf,
binoche, VMware VCP, Cisco CCNA
Well, i've also done the port binding ![]()
did you see any iscsi failure messages in vmkernel log?
binoche, VMware VCP, Cisco CCNA
What kind of connections setup is between your switch and the storage?
If you have ie etherchannel aggregate/"trunk" it will loadbalance on IP (default I think) and you will not get more that 1Gb/s due to the fact that ESX uses one IP an and the storage one IP and the etherchannel will only send the traffic down one of the paths.
But if you have two single-IP connections between the switch and storage then the problem is probably not there.
I got no significative error in the vmkernel log.
And i tried for the network between sw & storage with a direct attached connection. First i let them connected on a switch, but i didn't use any kind of etherchannel.
I connected them directly to be sure there were no problem with the switch...
'But if you have two single-IP connections between the switch and storage
then the problem is probably not there."
I do not really understand, but 4 nics of the storage with 4 differents ip on differents subnet are actually directly connected to the ESX.
Which iscsi software are you using on your storage device?
jumbo frame is helpful here? can you enable it and have a retest?
binoche, VMware VCP, Cisco CCNA
Hmm... My storage device is a netapp... So its the ISCSI from DataOnTap.
Test with jumbo frames : same thing.