I followed all of the instructions from the great post below, but I seem to only be averaging 30mb/s when doing sequential reads. I'm not sure what else I can change, but I know each NIC is capable of going up to 60 mb/s so I'd imagine I should be able to go at least that fast. It's attached to an OpenFiler that can go that fast so I'm not sure where the bottleneck is at.
I've played around with the round robin IOPS settings and it doesn't make much of a difference. Looking at the averages for both my NIC's through the performance tab it seems each is averaging around 15 mb/s. Are there any other tests or settings that I can try to see where the bottleneck is?
How are you testing this?
Could you give some detail of the setup?
Please award points to any useful answer.
I just created a VM on the iscsi datastore and have been running tools like "HD Tach" and "Iometer" to generate bemchmarks to see how fast the sequential reads are. On a Linux guest I also just did a simple dd test and I've was consistently getting 30mb/s across the board. I just made some changes on the Openfiler side and bumped it up to 60mb/s! Not bad although it'd be nice i fit was a little more faster
Before I didn't have any additional settings in my configuration file, but I added the parameters below from a thread I saw on the openfiler forums and it seemed to bumped up the average speed.
HeaderDigest None
DataDigest None
MaxConnections 1
InitialR2T Yes
ImmediateData Yes
MaxRecvDataSegmentLength 262144
MaxXmitDataSegmentLength 262144
MaxBurstLength 262144
FirstBurstLength 262144
DefaultTime2Wait 10
DefaultTime2Retain 20
MaxOutstandingR2T 8
DataPDUInOrder Yes
DataSequenceInOrder Yes
ErrorRecoveryLevel 0
Using IOMeter with 32K sequential read or write you'll need a fair queue depth specified (outstanding IOs) - say 32, to get good throughput, and a large test space (4GB+) to reduce the impact of caching and get meaningful results.
Try to enable flow control on your switch, if it is supported, and leave all connections as auto/auto (speed/duplex).
Please award points to any useful answer.
When you say "openfiler can go that fast" have you confirmed this in some sort of testing, or is this based on figures from the guys that produced OpenFiler?
I ran Iometer on the Openfiler array and was able to get 400 mb/s sequential reads on the array I have setup on there. Running iPerf from a guest VM on the ESX machine to the array I was able to get 60 mb/s per NIC. I figured with two NIC's I should be able to achieve around 100 mb/s of sequential read throughput. Looks like maybe I need to tweak the ISCSI settings some more as when I added those parameters it seemed to double the performance.