My colleague and I have been tuning some of our vSphere installations that utilize a SAN and iSCSI for datastore storage.
All iSCSI connections are configured for jumbo frames.
Additional multipath tuning can be done at the round robin level to force hops between paths based on the number of IOPs, or the byte size, etc.
While discussing best practices and actual implementation of those practices, we came up with a question:
The assumption is that ESX communicates with the VMDK on the SAN over the iSCSI connection, utilizing jumbo frames to maximize through put and performance. If round robin is implemented, we can force hops between paths every 8800 bytes, essentially waiting until the packet is full ( ie., the jumbo frame ) before sending off to the SAN and then hopping to the other path. On the other hand, we could force hops based on the the number of IOPs ( ie., 1 ).
Increasing the byte size seemed to offer a bit more throughput, though more testing would need to be performed to fully conclude this.
But here is the question:
ESX likely utilizes the jumbo frames to communicate with the SAN and "manage" the VMDK. Obviously, you would want to send full frames through each path and realize the full potential. But what does the virtual server who owns the VMDK do? Who or how does its traffic get managed? In other words, is ESX managing the virtual server's reads/writes and still defaulting back to the full jumbo frame theory? If that is the case, what if your virtual server is such that its function is low byte/high IOPS preferred?
Hopefully this makes. The central question is how are reads and writes managed for virtual machines?
Thanks
Bill
