XIV and ESX 3.5 Best practice questions

[jketron]

Does anyone have any advice, gotchas, links to some best practices using XIV storage and ESX 3.5?

Is there any reason to not use the upper end of the LUN size limit of 2TB?

What block Size are you or would you use?

Are you changing your HBA BIOS settings? to what?

[LucentOne]

One issue we ran into was not striping our IO across all of the interface modules/ports. With XIV, LUN queues/spindle sets are not the IO bottle neck because a given LUN is not tied to a spindle set like it is in conventional RAID. The limiting factor we ran into is how many IOPS can be driven down each port of the XIV. We made this mistake and performance was not steller under heavy load. We are in the process of fixing this error.

The key to remeber with XIV is that parallelism is core to the architecture and the entire disk sub-system must reflect this. vSphere 4 should show better performance with it's active-active multi-pathing (round robin).

I hope this helps.

[bobross]

lucentone, let us know how it turns out. we tested on XIV as well and even with multipathing across ports, perf was less than stellar due to the inherent nature of random reads, the workload that the XIV is not good at handling. We got around 3x better IOPS and response time as well on a bunch of VMs by using a smaller set of FC 15K spindles.

[LucentOne]

bobross: Are you on ESX 4? or did you use the multipathing in ESX 3.5?

[LucentOne]

Our performance has increased (more IOPS, less latency) now that we are (more) evenly distributing the workload across all ports/modules. I have not been able to test vSphere 4 with round robin multipathing enabled. I do have concerns about the LUN-path table in ESX filling up and causing issues and/or excessive overhead. Has anyone looked into this?

[Rumple]

Is there any reason to not use the upper end of the LUN size limit of 2TB?

I would use 8MB block size as it makes a difference when you have snapshots growing. Larger block size means they grow less, creating less reservations.

Also remember that vmware usually recommends 10-15 VM's per LUN (because of scvsi reservations), therefore a 2TB lun will need some pretty large vmdk files to stay inside that best practice.

I would regarless stay closer to 1.9TB vs 2TB...some people found that between esx 3.5 and esx 4 2TB was actually about 512bytes different...causing VM's to not load because of the vmfs volume being oversized.

If you have 2TB of space I would usually recommend something like 600GB LUNS (thats 12-40GB VM's with some space for snapshots,etc)

[bobross]

multipathing on 3.5. we also run 4.0 on our current array and are very happy with it.

[Twadeao]

How many targets did you put per zone and per Host?