slickshoes
Enthusiast
Enthusiast

Single ESX Server, Single iSCSI SAN, using MPIO... only transmitting on 1 NIC

I have a single ESX 3.5 U3 server. Connected via iSCSI to an Enhance-Tech RS8 IP4. I assigned two NICs on the SAN with different IP's and set up the 2 IP's in send targets. I have set some of my luns to use 1 path and some to use the other. According to the ESX performance reports, traffic is balancing between the 2 on the receiving side, but not transmitting. The ESX server is only transmitting on 1 of the NIC's

Graph is attached. any suggestions on how to get it to transmit on the 2 nics?

0 Kudos
9 Replies
RParker
Immortal
Immortal

traffic is balancing between the 2 on the receiving side, but not transmitting. The ESX server is only transmitting on 1 of the NIC's

You are contradicting yourself. You acknowledge that receiving is load balanced, and not transmitting, yet you mention ESX is transmitting on 1 NIC. So if you know that, it's not an ESX problem since it can't control transmit as evidenced by the documentation you saw.

Plus are you using an HBA or Software? I believe THEY can only utilize 1 path. The work around is to create 2 NIC's inside the VM and use iSCSI initator inisde the guest and THEN you can load balance, if that's your objective.

Another thing to consider is threshold. How much traffic is being trasmitted / received? Maybe if only half the bandwidth is being used it WON'T load balance since there isn't enough to warrant traffic saturation.

In any case I don't perceive this as a problem, unless you can somehow stress test the iSCSI path and see if it fails over WHEN traffic is heavy or if even it's necessary to have multiple paths on iSCSI in the first place. Failover is one thing, load balance is another. For performance reasons you may not even have close to needed additional bandwidth.

slickshoes
Enthusiast
Enthusiast

I'm using the software initiator. The vSwitch has 2 NIC's and is routing based on IP Hash. Since there are 2 targets with 2 different IP's then the hash should be different so it should transmit via different nic's right?

I'm just trying to utilize both NIC's

0 Kudos
AndreTheGiant
Immortal
Immortal

A good solution could be use more LUNs (at least 2).

In this case each LUN will use a different NIC.

Or you have to enable the experimental (on ESX 3.5) round robin balancing policy in ESX multipath.

Andre

Andre | http://about.me/amauro | http://vinfrastructure.it/ | @Andrea_Mauro
0 Kudos
dominic7
Virtuoso
Virtuoso

The problem here is that there is only a single software initiator, and like a virtual nic for a VM, that initiator takes up a single virtual port on a vSwitch, which only gives it access to a single physical adapter. If you want to multipath iscsi you have a couple of options:

1. Get 2 hardware iSCSI HBAs and fiddle with esxcfg-mpath to set path swapping on number of commands of number of blocks headed across the hba

2. Use a software initator inside the VM, give the VM 2 virtual nics where you have 2 vSwitches each connected to a seperate physical nic to ensure that those nics in the VM don't end up on the same vSwitch. Then you can use the MPIO option using the software iniatiator inside the VM to multipath to your iscsi SAN.

0 Kudos
Chuck8773
Hot Shot
Hot Shot

Duncan wrote a great article that got me up and running quickly.

http://www.yellow-bricks.com/2009/03/18/iscsi-multipathing-with-esxcliexploring-the-next-version-of-...

The only hiccup I ran into, was that I set the other nic to standby instead of not used. After that it worked like a charm.

Charles Killmer, VCP

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".

Charles Killmer, VCP4 If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".
Chuck8773
Hot Shot
Hot Shot

Also, I needed to click on manage paths and set it to round robin instead of fixed.

Charles Killmer, VCP

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".

Charles Killmer, VCP4 If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".
0 Kudos
slickshoes
Enthusiast
Enthusiast

I have 5 LUN's. 2 of the LUNS are set to use path vmhba32:2:x and the other 3 are set to use path vmhba32:3:x. I am using a Fixed multipathing policy and when i say they are set to use a path I mean that path is preferred, See pic

Also, the NIC team policy is to route based on IP Hash, not virtual port ID.

0 Kudos
jbruelasdgo
Virtuoso
Virtuoso

when you use the NIC teaming policy of IP , you need to consider publishing multiple IP addresses (Targets ) in order to the policy to balance the load

Jose B Ruelas http://aservir.wordpress.com
0 Kudos
slickshoes
Enthusiast
Enthusiast

I do. I have 2 send targets configured. 10.10.10.22 and 10.10.10.23....

0 Kudos