I have found that enabling Jumbo frames resulted in decreased read and write performance with the following hardare:
Server:
IBM x3550 M3 (7944AC1) with 1 6c Intel Xeon cpu X5650 and 12gb of memory
dual port QLogic iSCSI hba QLA406x with bios 1.14
Switch:
Two HP 2810-24G
Storage:
IBM DS3300 dual controller with 512MB cache
Configuration:
Jumbo frames are enabled at the QLogic Card in the FAST!util setup
Jumbo frames are enable for the VLANS on the HP 2810-24 switches
Jumbo frames are enabled on the four host ports of the DS3300.
I tested with a 2 disk SAS RAID 1 array on the DS3300.
The test Virtual machine was windows XP with the C: drive on local SAS disks and it's E: drive an 8GB buslogic scsi drive on the iSCSI storage array added via VMware.
I use microsoft's SQLIO as a benchmarking tool and record the mb/s for each test.
Without JUMBO FRAMES
8KB Random Write | 64kb Random Write | 128kb Random write | 256kb Random write | 8KB SEQUENTIAL Write | 64KB SEQUENTIAL Write | 128KB SEQUENTIALWrite | 256KB SEQUENTIAL Write | 8KB RANDOM Read | 64KB RANDOM Read | 128KB RANDOM Read | 256KB RANDOM Read | 8KB SEQUENTIAL Read | 64KB SEQUENTIAL Read | 128KB SEQUENTIAL Read | 256KB SEQUENTIAL Read |
<!--
BODY,DIV,TABLE,THEAD,TBODY,TFOOT,TR,TH,TD,P { font-family:"Liberation Sans"; font-size:x-small }
-->
7.16 | 39.73 | 56.82 | 77.78 | 17.07 | 78.98 | 84.13 | 88.49 | 68.43 | 111.49 | 111.67 | 111.45 | 67.94 | 111.72 | 111.76 | 111.8 |
WITH JUMBO FRAMES:
<!--
BODY,DIV,TABLE,THEAD,TBODY,TFOOT,TR,TH,TD,P { font-family:"Liberation Sans"; font-size:x-small }
-->
<!--
BODY,DIV,TABLE,THEAD,TBODY,TFOOT,TR,TH,TD,P { font-family:"Liberation Sans"; font-size:x-small }
-->
8KB Random Write | 64kb Random Write | 128kb Random write | 256kb Random write | 8KB SEQUENTIAL Write | 64KB SEQUENTIAL Write | 128KB SEQUENTIALWrite | 256KB SEQUENTIAL Write | 8KB RANDOM Read | 64KB RANDOM Read | 128KB RANDOM Read | 256KB RANDOM Read | 8KB SEQUENTIAL Read | 64KB SEQUENTIAL Read | 128KB SEQUENTIAL Read | 256KB SEQUENTIAL Read |
<!--
BODY,DIV,TABLE,THEAD,TBODY,TFOOT,TR,TH,TD,P { font-family:"Liberation Sans"; font-size:x-small }
-->
6.73 | 38.38 | 56.81 | 68.83 | 17.66 | 73.41 | 76.11 | 76.31 | 66.66 | 91.35 | 92.89 | 90.14 | 66.27 | 91.33 | 92.89 | 90.14 |
Here is the test program:
<!--
BODY,DIV,TABLE,THEAD,TBODY,TFOOT,TR,TH,TD,P { font-family:"Liberation Sans"; font-size:x-small }
-->
Download SQLIO from Microsoft: |
|
|
run the installer |
|
it will put the program in C:\Program Files (x86)\SQLIO\ |
|
|
create a bat or cmd file called SQLIO.cmd ( any name will do) and paste this into it: |
|
sqlio -kW -s10 -frandom -o8 -b8 -LS -Fparam.txt timeout /T 60 > results.txt |
sqlio -kW -s360 -frandom -o8 -b64 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kW -s360 -frandom -o8 -b128 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kW -s360 -frandom -o8 -b256 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kW -s360 -fsequential -o8 -b8 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kW -s360 -fsequential -o8 -b64 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kW -s360 -fsequential -o8 -b128 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kW -s360 -fsequential -o8 -b256 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kR -s360 -frandom -o8 -b8 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kR -s360 -frandom -o8 -b64 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kR -s360 -frandom -o8 -b128 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kR -s360 -frandom -o8 -b256 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kR -s360 -fsequential -o8 -b8 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kR -s360 -fsequential -o8 -b64 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kR -s360 -fsequential -o8 -b128 -LS -Fparam.txt timeout /T 60 >> results.txt |
sqlio -kR -s360 -fsequential -o8 -b256 -LS -Fparam.txt >> results.txt |
I'm assuming that I must not have something configured properly, Does someone have similiar hardware have better results?
strange, are there vmkernel failure messages when jumbo frame enabled?
binoche, VMware VCP, Cisco CCNA
did you create the vswitches with jumbo frames enabled...I believe at once point you needed to do that manually at the command line but not sure if thats still true in the latest esx version.
I've had to enable the jumbo frames via the VMA on ESXi 4.0 so I believe you are correct., of course you'll need jumbo frames through out your entire setup.
The qlogic cards show up as storage adapters, not network adapters. There is no vswitch to create.