VMware Cloud Community
joshp
Enthusiast
Enthusiast
Jump to solution

Balance disk traffic across multiple HBA's

I have 9 ESX servers. Each server has two HBA's connected to a Clariion CX340 SAN via two Brocade switches. Each ESX server sees four paths to each LUN; however, it appears only one path is ever used (vmhba1). This creates a load problem because all vmhba1 connections are using a single Brocade switch; the other Brocade switch used for vmhba2 never processes any traffic. What is the textbook way to balance load to the SAN using ESX? Should I set the preferred path on each ESX server to alternate using VC Client? If yes, this seems like a nightmare to manage with so many paths. See screenshot for example.

2986_2986.gif

VCP 3, 4 www.vstable.com
Reply
0 Kudos
1 Solution

Accepted Solutions
lamw
Community Manager
Community Manager
Jump to solution

I agree, many of the cool features with 3.5 have all been slated as experimental and should not be used for production.

There's a automated script that can be ran to load-balance between your storage paths:

http://www.yellow-bricks.com/2008/04/01/load-balancing-activeactive-sans/

Hopefully this helps

View solution in original post

Reply
0 Kudos
5 Replies
williamarrata
Expert
Expert
Jump to solution

This is what I've found out about ESX 3.5 Multipathing via Round-robin balancing for ESX 3.5. This is the best the ESX can do for now.

First of all, its "experimental" meaning VMware won't support it in a production environment.

You can enable it on a 3.5 host within the VIC. Select the host / configuration tab / storage adapters. Highlight an HBA and right-click a path and select manage paths. Under "Policy" hit the Change button. The selection Policy page opens. Select the Multipathing policy (Round-Robin) option.

Check out the doc here --> http://www.vmware.com/pdf/vi3_35_25_roundrobin.pdf

Excerpt:

To achieve better load balancing across paths, administrators can specify that the ESX Server host should switch paths under certain circumstances. Different settable options determine when the ESX Server host switches paths and what paths are chosen.

When to switch - Specify that the ESX Server host should attempt a path switch after a specified number of I/O blocks have been issued on a path or after a specified number of read or write commands have been issued on a path. If another path exists that meets the specified path policy for the target, the active path to the target is switched to the new path. The --custom-max-commands and --custom-max-blocks options specify when to switch.

Which target to use - Specify that the next path should be on the preferred target, the most recently used target, or any target. The --custom-target-policy option specifies which target to use.

Which HBA to use - Specify that the next path should be on the preferred HBA, the most recently used HBA, the HBA with the minimum outstanding I/O requests, or any HBA. The --custom-HBA-policy option specifies which HBA to use.

What the new experimental features in ESX 3.5 add is the ability to effectively aggregate multiple paths. In my view, with 4Gb FC this may be less useful than you may think, since it's tough enough for a disk subsystem to saturate that amount of bandwidth today on a pure streaming basis, let alone with normal workloads. I've not seen benchmarks yet that reliably informs that view, though.

What we may be looking for is MPIO (Multi Pathing I/O) which is NOT available yet for ESX. Maybe in Vi4.

Hope that helped. Smiley Happy

Hope that helped. 🙂
mcowger
Immortal
Immortal
Jump to solution

we've been using round robin for the past few months and its been great.

--Matt

--Matt VCDX #52 blog.cowger.us
Reply
0 Kudos
joshp
Enthusiast
Enthusiast
Jump to solution

I had considered using round robin but I do not have the option of using an unsupported configuration in my environment. How would one go about manually balancing the load across hba's? It seems the only way to do this in a supported configuration is to alternate the preferred hba on each LUN in VC client. How would one keep track of this manual configuration?

VCP 3, 4 www.vstable.com
Reply
0 Kudos
lamw
Community Manager
Community Manager
Jump to solution

I agree, many of the cool features with 3.5 have all been slated as experimental and should not be used for production.

There's a automated script that can be ran to load-balance between your storage paths:

http://www.yellow-bricks.com/2008/04/01/load-balancing-activeactive-sans/

Hopefully this helps

Reply
0 Kudos
williamarrata
Expert
Expert
Jump to solution

Or you can add a few more hba's and fether out your front end of your SAN so that it spreads the load. I don't see that much advantage in that but it builds in more redundancy and your load is not pushing thru 1 card. I wish I could give you better solutions but, this is the best VMWare puts out. You would think, the're owned by EMC, don't you think they would come up with a better i/o solution?

Hope that helped. Smiley Happy

Hope that helped. 🙂
Reply
0 Kudos