I have 9 ESX servers. Each server has two HBA's connected to a Clariion CX340 SAN via two Brocade switches. Each ESX server sees four paths to each LUN; however, it appears only one path is ever used (vmhba1). This creates a load problem because all vmhba1 connections are using a single Brocade switch; the other Brocade switch used for vmhba2 never processes any traffic. What is the textbook way to balance load to the SAN using ESX? Should I set the preferred path on each ESX server to alternate using VC Client? If yes, this seems like a nightmare to manage with so many paths. See screenshot for example.
I agree, many of the cool features with 3.5 have all been slated as experimental and should not be used for production.
There's a automated script that can be ran to load-balance between your storage paths:
http://www.yellow-bricks.com/2008/04/01/load-balancing-activeactive-sans/
Hopefully this helps
This is what I've found out about ESX 3.5 Multipathing via Round-robin balancing for ESX 3.5. This is the best the ESX can do for now.
First of all, its "experimental" meaning VMware won't support it in a production environment.
You can enable it on a 3.5 host within the VIC. Select the host / configuration tab / storage adapters. Highlight an HBA and right-click a path and select manage paths. Under "Policy" hit the Change button. The selection Policy page opens. Select the Multipathing policy (Round-Robin) option.
Check out the doc here --> http://www.vmware.com/pdf/vi3_35_25_roundrobin.pdf
Excerpt:
To achieve better load balancing across paths, administrators can specify that the ESX Server host should switch paths under certain circumstances. Different settable options determine when the ESX Server host switches paths and what paths are chosen.
When to switch - Specify that the ESX Server host should attempt a path switch after a specified number of I/O blocks have been issued on a path or after a specified number of read or write commands have been issued on a path. If another path exists that meets the specified path policy for the target, the active path to the target is switched to the new path. The --custom-max-commands and --custom-max-blocks options specify when to switch.
Which target to use - Specify that the next path should be on the preferred target, the most recently used target, or any target. The --custom-target-policy option specifies which target to use.
Which HBA to use - Specify that the next path should be on the preferred HBA, the most recently used HBA, the HBA with the minimum outstanding I/O requests, or any HBA. The --custom-HBA-policy option specifies which HBA to use.
What the new experimental features in ESX 3.5 add is the ability to effectively aggregate multiple paths. In my view, with 4Gb FC this may be less useful than you may think, since it's tough enough for a disk subsystem to saturate that amount of bandwidth today on a pure streaming basis, let alone with normal workloads. I've not seen benchmarks yet that reliably informs that view, though.
What we may be looking for is MPIO (Multi Pathing I/O) which is NOT available yet for ESX. Maybe in Vi4.
Hope that helped.
we've been using round robin for the past few months and its been great.
--Matt
I had considered using round robin but I do not have the option of using an unsupported configuration in my environment. How would one go about manually balancing the load across hba's? It seems the only way to do this in a supported configuration is to alternate the preferred hba on each LUN in VC client. How would one keep track of this manual configuration?
I agree, many of the cool features with 3.5 have all been slated as experimental and should not be used for production.
There's a automated script that can be ran to load-balance between your storage paths:
http://www.yellow-bricks.com/2008/04/01/load-balancing-activeactive-sans/
Hopefully this helps
Or you can add a few more hba's and fether out your front end of your SAN so that it spreads the load. I don't see that much advantage in that but it builds in more redundancy and your load is not pushing thru 1 card. I wish I could give you better solutions but, this is the best VMWare puts out. You would think, the're owned by EMC, don't you think they would come up with a better i/o solution?
Hope that helped.