VMware Cloud Community
aritting
Contributor
Contributor

IBM DS3300 - QLogic QLA406x iSCSI hba, ESXi 4.0 Jumbo Frames Slower than 1500 mtu

I have found that enabling Jumbo frames resulted in decreased read and write performance with the following hardare:

Server:

IBM x3550 M3 (7944AC1) with 1 6c Intel Xeon cpu X5650 and 12gb of memory

dual port QLogic iSCSI hba QLA406x with bios 1.14

Switch:

Two HP 2810-24G

Storage:

IBM DS3300 dual controller with 512MB cache

Configuration:

Jumbo frames are enabled at the QLogic Card in the FAST!util setup

Jumbo frames are enable for the VLANS on the HP 2810-24 switches

Jumbo frames are enabled on the four host ports of the DS3300.

I tested with a 2 disk SAS RAID 1 array on the DS3300.

The test Virtual machine was windows XP with the C: drive on local SAS disks and it's E: drive an 8GB buslogic scsi drive on the iSCSI storage array added via VMware.

I use microsoft's SQLIO as a benchmarking tool and record the mb/s for each test.

Without JUMBO FRAMES

8KB Random Write

64kb Random Write

128kb Random write

256kb Random write

8KB SEQUENTIAL Write

64KB SEQUENTIAL Write

128KB SEQUENTIALWrite

256KB SEQUENTIAL Write

8KB RANDOM Read

64KB RANDOM Read

128KB RANDOM Read

256KB RANDOM Read

8KB SEQUENTIAL Read

64KB SEQUENTIAL Read

128KB SEQUENTIAL Read

256KB SEQUENTIAL Read

<!--

BODY,DIV,TABLE,THEAD,TBODY,TFOOT,TR,TH,TD,P { font-family:"Liberation Sans"; font-size:x-small }

-->

7.16

39.73

56.82

77.78

17.07

78.98

84.13

88.49

68.43

111.49

111.67

111.45

67.94

111.72

111.76

111.8

WITH JUMBO FRAMES:

<!--

BODY,DIV,TABLE,THEAD,TBODY,TFOOT,TR,TH,TD,P { font-family:"Liberation Sans"; font-size:x-small }

-->

<!--

BODY,DIV,TABLE,THEAD,TBODY,TFOOT,TR,TH,TD,P { font-family:"Liberation Sans"; font-size:x-small }

-->

8KB Random Write

64kb Random Write

128kb Random write

256kb Random write

8KB SEQUENTIAL Write

64KB SEQUENTIAL Write

128KB SEQUENTIALWrite

256KB SEQUENTIAL Write

8KB RANDOM Read

64KB RANDOM Read

128KB RANDOM Read

256KB RANDOM Read

8KB SEQUENTIAL Read

64KB SEQUENTIAL Read

128KB SEQUENTIAL Read

256KB SEQUENTIAL Read

<!--

BODY,DIV,TABLE,THEAD,TBODY,TFOOT,TR,TH,TD,P { font-family:"Liberation Sans"; font-size:x-small }

-->

6.73

38.38

56.81

68.83

17.66

73.41

76.11

76.31

66.66

91.35

92.89

90.14

66.27

91.33

92.89

90.14

Here is the test program:

<!--

BODY,DIV,TABLE,THEAD,TBODY,TFOOT,TR,TH,TD,P { font-family:"Liberation Sans"; font-size:x-small }

-->

Download SQLIO from Microsoft:

run the installer

it will put the program in C:\Program Files (x86)\SQLIO\

create a bat or cmd file called SQLIO.cmd ( any name will do) and paste this into it:

sqlio -kW -s10 -frandom -o8 -b8 -LS -Fparam.txt timeout /T 60 &gt; results.txt

sqlio -kW -s360 -frandom -o8 -b64 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kW -s360 -frandom -o8 -b128 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kW -s360 -frandom -o8 -b256 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kW -s360 -fsequential -o8 -b8 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kW -s360 -fsequential -o8 -b64 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kW -s360 -fsequential -o8 -b128 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kW -s360 -fsequential -o8 -b256 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kR -s360 -frandom -o8 -b8 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kR -s360 -frandom -o8 -b64 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kR -s360 -frandom -o8 -b128 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kR -s360 -frandom -o8 -b256 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kR -s360 -fsequential -o8 -b8 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kR -s360 -fsequential -o8 -b64 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kR -s360 -fsequential -o8 -b128 -LS -Fparam.txt timeout /T 60 &gt;&gt; results.txt

sqlio -kR -s360 -fsequential -o8 -b256 -LS -Fparam.txt &gt;&gt; results.txt

I'm assuming that I must not have something configured properly, Does someone have similiar hardware have better results?

0 Kudos
4 Replies
binoche
VMware Employee
VMware Employee

strange, are there vmkernel failure messages when jumbo frame enabled?

binoche, VMware VCP, Cisco CCNA

0 Kudos
Rumple
Virtuoso
Virtuoso

did you create the vswitches with jumbo frames enabled...I believe at once point you needed to do that manually at the command line but not sure if thats still true in the latest esx version.

0 Kudos
jonowebster
Contributor
Contributor

I've had to enable the jumbo frames via the VMA on ESXi 4.0 so I believe you are correct., of course you'll need jumbo frames through out your entire setup.

0 Kudos
aritting
Contributor
Contributor

The qlogic cards show up as storage adapters, not network adapters. There is no vswitch to create.

0 Kudos