christianZ
Champion
Champion

Open unofficial storage performance thread

Attention!

Since this thread is getting longer and longer, not to mention the load times, Christian and I decided to close this thread and start a new one.

The new thread is available here:

Oliver Reeh[/i]

[VMware Communities User Moderator|http://communities.vmware.com/docs/DOC-2444][/i]

My idea is to create an open thread with uniform tests whereby the results will be all inofficial and w/o any

warranty.

If anybody shouldn't be agreed with some results then he can make own tests and presents

his/her results too.

I hope this way to classify the different systems and give a "neutral" performance comparison.

Additionally I will mention that the performance is one of many aspects to choose the right system.

The others could be e.g.

\- support quality

\- system management integration

\- distribution

\- self made experiences

\- additional features

\- costs for storage system and infrastructure, etc.

There are examples of IOMETER Tests:

=====================================

\######## TEST NAME: Max Throughput-100%Read

size,% of size,% reads,% random,delay,burst,align,reply

32768,100,100,0,0,1,0,0

\######## TEST NAME: RealLife-60%Rand-65%Read

size,% of size,% reads,% random,delay,burst,align,reply

8192,100,65,60,0,1,0,0

\######## TEST NAME: Max Throughput-50%Read

size,% of size,% reads,% random,delay,burst,align,reply

32768,100,50,0,0,1,0,0

\######## TEST NAME: Random-8k-70%Read

size,% of size,% reads,% random,delay,burst,align,reply

8192,100,70,100,0,1,0,0

The global options are:

=====================================

Worker

Worker 1

Worker type

DISK

Default target settings for worker

Number of outstanding IOs,test connection rate,transactions per connection

64,ENABLED,500

Disk maximum size,starting sector

8000000,0

Run time = 5 min

For testing the disk C is configured and the test file (8000000 sectors) will be created by

first running - you need free space on the disk.

The cache size has direct influence on results. By systems with cache over 2GB the test

file should be increased.

LINK TO IOMETER:

Significant results are: Av. Response time, Av. IOS/sek, Av. MB/s

To mention are: what server (vm or physical), Processor number/type; What storage system, How many disks

Here the config file *.icf

\####################################### BEGIN of *.icf

Version 2004.07.30

'TEST SETUP ====================================================================

'Test Description

IO-Test

'Run Time

' hours minutes seconds

0 5 0

'Ramp Up Time (s)

0

'Default Disk Workers to Spawn

NUMBER_OF_CPUS

'Default Network Workers to Spawn

0

'Record Results

ALL

'Worker Cycling

' start step step type

1 5 LINEAR

'Disk Cycling

' start step step type

1 1 LINEAR

'Queue Depth Cycling

' start end step step type

8 128 2 EXPONENTIAL

'Test Type

NORMAL

'END test setup

'RESULTS DISPLAY ===============================================================

'Update Frequency,Update Type

4,WHOLE_TEST

'Bar chart 1 statistic

Total I/Os per Second

'Bar chart 2 statistic

Total MBs per Second

'Bar chart 3 statistic

Average I/O Response Time (ms)

'Bar chart 4 statistic

Maximum I/O Response Time (ms)

'Bar chart 5 statistic

% CPU Utilization (total)

'Bar chart 6 statistic

Total Error Count

'END results display

'ACCESS SPECIFICATIONS =========================================================

'Access specification name,default assignment

Max Throughput-100%Read,ALL

'size,% of size,% reads,% random,delay,burst,align,reply

32768,100,100,0,0,1,0,0

'Access specification name,default assignment

RealLife-60%Rand-65%Read,ALL

'size,% of size,% reads,% random,delay,burst,align,reply

8192,100,65,60,0,1,0,0

'Access specification name,default assignment

Max Throughput-50%Read,ALL

'size,% of size,% reads,% random,delay,burst,align,reply

32768,100,50,0,0,1,0,0

'Access specification name,default assignment

Random-8k-70%Read,ALL

'size,% of size,% reads,% random,delay,burst,align,reply

8192,100,70,100,0,1,0,0

'END access specifications

'MANAGER LIST ==================================================================

'Manager ID, manager name

1,PB-W2K3-04

'Manager network address

193.27.20.145

'Worker

Worker 1

'Worker type

DISK

'Default target settings for worker

'Number of outstanding IOs,test connection rate,transactions per connection

64,ENABLED,500

'Disk maximum size,starting sector

8000000,0

'End default target settings for worker

'Assigned access specs

'End assigned access specs

'Target assignments

'Target

C:

'Target type

DISK

'End target

'End target assignments

'End worker

'End manager

'END manager list

Version 2004.07.30

\####################################### ENDE of *.icf

TABLE SAMPLE

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE oF RESULTS

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: VM or PHYS.

CPU TYPE / NUMBER: VCPU / 1

HOST TYPE: Dell PE6850, 16GB RAM; 4x XEON 51xx, 2,66 GHz, DC

STORAGE TYPE / DISK NUMBER / RAID LEVEL: EQL PS3600 x 1 / 14+2 Disks / R50

##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

##################################################################################

Max Throughput-100%Read........__________..........__________.........__________

RealLife-60%Rand-65%Read......__________..........__________.........__________

Max Throughput-50%Read..........__________..........__________.........__________

Random-8k-70%Read.................__________..........__________.........__________

EXCEPTIONS: CPU Util.-XX%;

##################################################################################

I hope YOU JOIN IN !

Regards

Christian

A Google Spreadsheet version is here:

Message was edited by:

ken.cline@hp.com to remove ALL CAPS from thread title

Message was edited by:

RDPetruska

Added link to Atamido's Google Spreadsheet

Tags (1)
0 Kudos
457 Replies
christianZ
Champion
Champion

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: VM ON ESX 3.0.1

CPU TYPE / NUMBER: VCPU / 1

HOST TYPE: Dell PE6850, 16GB RAM; 4x XEON 7020, 2,66 GHz, DC

STORAGE TYPE / DISK NUMBER / RAID LEVEL: EQL PS3600 x 1 / 14+2 SAS10k / R50

SAN TYPE / HBAs : iSCSI, QLA4050 HBA

\##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

\##################################################################################

Max Throughput-100%Read........__17______..........___3551___.........___111____

RealLife-60%Rand-65%Read......___21_____..........___2550___.........____20____

Max Throughput-50%Read..........____10____..........___5803___.........___181____

Random-8k-70%Read.................____23____..........___2410___.........____19____

EXCEPTIONS: VCPU Util. 60-46-75-46 %;

0 Kudos
christianZ
Champion
Champion

HERE INTERESTING TEST WITH VM ON VMWARE SERVER 1.x[/b]

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: Win2003 Std. VM ON VMWare Server 1.0.1 / SLES10

CPU TYPE / NUMBER: VCPU / 1

HOST TYPE: Dell PE1950, 4GB RAM, 2x XEON 5130, 2,0 GHz, DC

STORAGE TYPE / DISK NUMBER / RAID LEVEL: EQL PS3600 x 1 / 14+2 SAS10k / R50

SAN TYPE / HBAs : iSCSI; iSCSI softw. initiator in SLES10, 1XGb NIC for iSCSI

\##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

\##################################################################################

Max Throughput-100%Read........____20____..........___3200___.........___105____

RealLife-60%Rand-65%Read......____36____..........___1630___.........____15____

Max Throughput-50%Read..........____30____..........___2170___.........____68____

Random-8k-70%Read.................____36____..........___1635___.........____13____

EXCEPTIONS: performance partial bumpy; by seq. tests "Connection Rate" set to off;

"Max. Response Time" much higher as VM on ESX; by 50%Read high level of CPU %Wait on host

0 Kudos
christianZ
Champion
Champion

VMWare Server's VM seems to be much slower as VM on ESX with iscsi HBA but not so bad compared to ESX iscsi software initiator - doesn't it ??

0 Kudos
eliot
Enthusiast
Enthusiast

To make life a little easier, here's the config file (from above):

http://www.mez.co.uk/OpenPerformanceTest.icf

0 Kudos
christianZ
Champion
Champion

Thanks for that.

0 Kudos
christianZ
Champion
Champion

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: Win2003 Std. PHYS.[/b]

CPU TYPE / NUMBER: CPU / 2

HOST TYPE: Dell PE2650, 4GB RAM, 2x XEON, 2,8 GHz

STORAGE TYPE / DISK NUMBER / RAID LEVEL: INFORTREND S12F-R1420 / 6+1 SAS15k / R5[/b]

SAN TYPE / HBAs : FC, QLA2460

\##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

\##################################################################################

Max Throughput-100%Read........____7_____..........___8350___.........___260____

RealLife-60%Rand-65%Read......____42____..........___1340___.........____11____

Max Throughput-50%Read..........____11____..........___5300___.........____165___

Random-8k-70%Read.................____40____..........___1450___.........____11____

EXCEPTIONS:

0 Kudos
happyhammer
Hot Shot
Hot Shot

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: VM ON ESX 3.0.1

CPU TYPE / NUMBER: VCPU / 1

HOST TYPE: Dell PE2950, 8GB RAM; 2x XEON 5355(Quad), 2,66 GHz,

STORAGE TYPE / DISK NUMBER / RAID LEVEL: EQL PS100e x 1 / 14+2 SATA / R50

SAN TYPE / HBAs : iSCSI, QLA4052 HBA Jumbo's and Flow Control

##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

##################################################################################

Max Throughput-100%Read........__25______..........___2321___.........___72____

RealLife-60%Rand-65%Read......___37_____..........___1303___.........____10____

Max Throughput-50%Read..........____14____..........___4030___.........___126____

Random-8k-70%Read.................____42____..........___1174___.........____9____

EXCEPTIONS: VCPU Util. 22-35-35-33 %;

0 Kudos
doubleH
Expert
Expert

i am looking at purchasing the same storage array and hba as happyhammer. although this thread is nice i really don't know what the numbers mean. would someone be able to explain what they mean and provide an average number to compare against?

If you found this or any other post helpful please consider the use of the Helpfull/Correct buttons to award points
0 Kudos
davidbarclay
Virtuoso
Virtuoso

Great idea mate. It's always nice to compare notes..just in case things aren't right! I hope a few more people get involved. I'll do both physical and virtual. Physical is running now. Will post when done.

Dave

0 Kudos
davidbarclay
Virtuoso
Virtuoso

BTW, another very important factor will be the block size of the VMFS partition. Might want to add that to your results template.

Dave

0 Kudos
davidbarclay
Virtuoso
Virtuoso

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS - PHYSICAL

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: Win2k3 Standard Physical

CPU TYPE / NUMBER: CPU / 2

HOST TYPE: HP BL460c Blade, 4GB RAM; 2x XEON 5150 (Dualcore), 2,66 GHz,

STORAGE TYPE / DISK NUMBER / RAID LEVEL: HP EVA4000 x 2 / 12 x 10k FC HDD on vRAID5

VMFS: 100GB LUN, 1MB Block Size

SAN TYPE / HBAs : 4GB FC, HP/QLogic QMH2462 Dual HBAs, Dual Brocade 4100 Switches

##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

##################################################################################

Max Throughput-100%Read......._____2.91____.......___19573.45___....___610.64____

RealLife-60%Rand-65%Read......_____13.96___.......___2766.94____....___21.59_____

Max Throughput-50%Read........_____3.21____.......___18270.35___....___567.12____

Random-8k-70%Read............._____11.92___.......___2881.94____....___22.43_____

EXCEPTIONS:

0 Kudos
davidbarclay
Virtuoso
Virtuoso

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS - VM on RDM

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: VM ON ESX 3.0.1

CPU TYPE / NUMBER: VCPU / 1

HOST TYPE: HP BL480c Blade, 16GB RAM; 2x XEON 5150 (Dualcore), 2,66 GHz,

STORAGE TYPE / DISK NUMBER / RAID LEVEL: HP EVA4000 x 2 / 12 x 10k FC HDD on vRAID5

VMFS: 100GB LUN, RDM

SAN TYPE / HBAs : 4GB FC, HP/QLogic QMH2462 Dual HBAs, Dual Brocade 4100 Switchesl

##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

##################################################################################

Max Throughput-100%Read.......____5.68_____.......___10104.08___....___315.75____

RealLife-60%Rand-65%Read......____18.37____.......___2520.07_____....__19.69______

Max Throughput-50%Read........____3.64_____.......___15316.05___....___478.63_____

Random-8k-70%Read.............____17.41____.......___2661.18____....___20.79_____

EXCEPTIONS:

0 Kudos
davidbarclay
Virtuoso
Virtuoso

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS - VM on 1MB Block Size VMFS

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: VM ON ESX 3.0.1

CPU TYPE / NUMBER: VCPU / 1

HOST TYPE: HP BL480c Blade, 16GB RAM; 2x XEON 5150 (Dualcore), 2,66 GHz,

STORAGE TYPE / DISK NUMBER / RAID LEVEL: HP EVA4000 x 2 / 12 x 10k FC HDD on vRAID5

VMFS: 100GB LUN, 1MB Block Size

SAN TYPE / HBAs : 4GB FC, HP/QLogic QMH2462 Dual HBAs, Dual Brocade 4100 Switchesl

##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

##################################################################################

Max Throughput-100%Read.......___5.25______.......__10476.40____....__327.39_____

RealLife-60%Rand-65%Read......___17.73_____.......__2642.48______...._20.50_______

Max Throughput-50%Read........___39.51______......._1311.69_____....__40.99______

Random-8k-70%Read.............___18.61_____.......__2464.30_____....__19.28______

EXCEPTIONS:

0 Kudos
davidbarclay
Virtuoso
Virtuoso

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS - VM on 2MB Block Size VMFS

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: VM ON ESX 3.0.1

CPU TYPE / NUMBER: VCPU / 1

HOST TYPE: HP BL480c Blade, 16GB RAM; 2x XEON 5150 (Dualcore), 2,66 GHz,

STORAGE TYPE / DISK NUMBER / RAID LEVEL: HP EVA4000 x 2 / 12 x 10k FC HDD on vRAID5

VMFS: 100GB LUN, 2MB Block Size

SAN TYPE / HBAs : 4GB FC, HP/QLogic QMH2462 Dual HBAs, Dual Brocade 4100 Switchesl

##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

##################################################################################

Max Throughput-100%Read.......__6.25_______.......__9548.12____....__294.21_____

RealLife-60%Rand-65%Read......__16.21______.......__2815.03____....__21.08______

Max Throughput-50%Read........__43.81_______......._1239.55_____...._38.68_______

Random-8k-70%Read.............__18.74_____.......___2430.08____....__19.02______

EXCEPTIONS:

0 Kudos
davidbarclay
Virtuoso
Virtuoso

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS - VM on 4MB Block Size VMFS

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: VM ON ESX 3.0.1

CPU TYPE / NUMBER: VCPU / 1

HOST TYPE: HP BL480c Blade, 16GB RAM; 2x XEON 5150 (Dualcore), 2,66 GHz,

STORAGE TYPE / DISK NUMBER / RAID LEVEL: HP EVA4000 x 2 / 12 x 10k FC HDD on vRAID5

VMFS: 100GB LUN, 4MB Block Size

SAN TYPE / HBAs : 4GB FC, HP/QLogic QMH2462 Dual HBAs, Dual Brocade 4100 Switchesl

##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

##################################################################################

Max Throughput-100%Read.......__5.39_______......._10220.13_____....__321.71_____

RealLife-60%Rand-65%Read......__19.25______......._2322.95_______...._18.20_______

Max Throughput-50%Read........__43.23_______.....__1249.08____....____39.03____

Random-8k-70%Read.............__18.27______......._2507.81______....__19.56______

EXCEPTIONS:

0 Kudos
davidbarclay
Virtuoso
Virtuoso

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS - VM on 8MB Block Size VMFS

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: VM ON ESX 3.0.1

CPU TYPE / NUMBER: VCPU / 1

HOST TYPE: HP BL480c Blade, 16GB RAM; 2x XEON 5150 (Dualcore), 2,66 GHz,

STORAGE TYPE / DISK NUMBER / RAID LEVEL: HP EVA4000 x 2 / 12 x 10k FC HDD on vRAID5

VMFS: 100GB LUN, 8MB Block Size

SAN TYPE / HBAs : 4GB FC, HP/QLogic QMH2462 Dual HBAs, Dual Brocade 4100 Switchesl

##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

##################################################################################

Max Throughput-100%Read.......__6.45_______......._9193.11_____....__288.85_____

RealLife-60%Rand-65%Read......__20.79______......._2082.77_______..._16.11_______

Max Throughput-50%Read........__39.86_______......_1261.83______...._39.51_______

Random-8k-70%Read.............__18.09______......._2510.84______...._19.39_______

EXCEPTIONS:

0 Kudos
davidbarclay
Virtuoso
Virtuoso

So, it would seem that;

1) VMFS block size seems to have little to no effect on performance.

2) RDM has minimal performance increases over VMFS (except in 100% sequential tests which just won't ever happen in the real world)

3) VMFS has minimal impact on performance, achieving approximately 98%+ of physical performance

Anyone care to comment with their experiences?

Dave

0 Kudos
christianZ
Champion
Champion

Dave,

thanks for that - great input.

0 Kudos
Gabrie1
Commander
Commander

How about vmdks running in snapshot mode?

Gabrie

http://www.GabesVirtualWorld.com
0 Kudos