VMware Cloud Community
christianZ
Champion
Champion

Open unofficial storage performance thread

Attention!

Since this thread is getting longer and longer, not to mention the load times, Christian and I decided to close this thread and start a new one.

The new thread is available here:

Oliver Reeh[/i]

[VMware Communities User Moderator|http://communities.vmware.com/docs/DOC-2444][/i]

My idea is to create an open thread with uniform tests whereby the results will be all inofficial and w/o any

warranty.

If anybody shouldn't be agreed with some results then he can make own tests and presents

his/her results too.

I hope this way to classify the different systems and give a "neutral" performance comparison.

Additionally I will mention that the performance is one of many aspects to choose the right system.

The others could be e.g.

\- support quality

\- system management integration

\- distribution

\- self made experiences

\- additional features

\- costs for storage system and infrastructure, etc.

There are examples of IOMETER Tests:

=====================================

\######## TEST NAME: Max Throughput-100%Read

size,% of size,% reads,% random,delay,burst,align,reply

32768,100,100,0,0,1,0,0

\######## TEST NAME: RealLife-60%Rand-65%Read

size,% of size,% reads,% random,delay,burst,align,reply

8192,100,65,60,0,1,0,0

\######## TEST NAME: Max Throughput-50%Read

size,% of size,% reads,% random,delay,burst,align,reply

32768,100,50,0,0,1,0,0

\######## TEST NAME: Random-8k-70%Read

size,% of size,% reads,% random,delay,burst,align,reply

8192,100,70,100,0,1,0,0

The global options are:

=====================================

Worker

Worker 1

Worker type

DISK

Default target settings for worker

Number of outstanding IOs,test connection rate,transactions per connection

64,ENABLED,500

Disk maximum size,starting sector

8000000,0

Run time = 5 min

For testing the disk C is configured and the test file (8000000 sectors) will be created by

first running - you need free space on the disk.

The cache size has direct influence on results. By systems with cache over 2GB the test

file should be increased.

LINK TO IOMETER:

Significant results are: Av. Response time, Av. IOS/sek, Av. MB/s

To mention are: what server (vm or physical), Processor number/type; What storage system, How many disks

Here the config file *.icf

\####################################### BEGIN of *.icf

Version 2004.07.30

'TEST SETUP ====================================================================

'Test Description

IO-Test

'Run Time

' hours minutes seconds

0 5 0

'Ramp Up Time (s)

0

'Default Disk Workers to Spawn

NUMBER_OF_CPUS

'Default Network Workers to Spawn

0

'Record Results

ALL

'Worker Cycling

' start step step type

1 5 LINEAR

'Disk Cycling

' start step step type

1 1 LINEAR

'Queue Depth Cycling

' start end step step type

8 128 2 EXPONENTIAL

'Test Type

NORMAL

'END test setup

'RESULTS DISPLAY ===============================================================

'Update Frequency,Update Type

4,WHOLE_TEST

'Bar chart 1 statistic

Total I/Os per Second

'Bar chart 2 statistic

Total MBs per Second

'Bar chart 3 statistic

Average I/O Response Time (ms)

'Bar chart 4 statistic

Maximum I/O Response Time (ms)

'Bar chart 5 statistic

% CPU Utilization (total)

'Bar chart 6 statistic

Total Error Count

'END results display

'ACCESS SPECIFICATIONS =========================================================

'Access specification name,default assignment

Max Throughput-100%Read,ALL

'size,% of size,% reads,% random,delay,burst,align,reply

32768,100,100,0,0,1,0,0

'Access specification name,default assignment

RealLife-60%Rand-65%Read,ALL

'size,% of size,% reads,% random,delay,burst,align,reply

8192,100,65,60,0,1,0,0

'Access specification name,default assignment

Max Throughput-50%Read,ALL

'size,% of size,% reads,% random,delay,burst,align,reply

32768,100,50,0,0,1,0,0

'Access specification name,default assignment

Random-8k-70%Read,ALL

'size,% of size,% reads,% random,delay,burst,align,reply

8192,100,70,100,0,1,0,0

'END access specifications

'MANAGER LIST ==================================================================

'Manager ID, manager name

1,PB-W2K3-04

'Manager network address

193.27.20.145

'Worker

Worker 1

'Worker type

DISK

'Default target settings for worker

'Number of outstanding IOs,test connection rate,transactions per connection

64,ENABLED,500

'Disk maximum size,starting sector

8000000,0

'End default target settings for worker

'Assigned access specs

'End assigned access specs

'Target assignments

'Target

C:

'Target type

DISK

'End target

'End target assignments

'End worker

'End manager

'END manager list

Version 2004.07.30

\####################################### ENDE of *.icf

TABLE SAMPLE

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE oF RESULTS

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: VM or PHYS.

CPU TYPE / NUMBER: VCPU / 1

HOST TYPE: Dell PE6850, 16GB RAM; 4x XEON 51xx, 2,66 GHz, DC

STORAGE TYPE / DISK NUMBER / RAID LEVEL: EQL PS3600 x 1 / 14+2 Disks / R50

##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

##################################################################################

Max Throughput-100%Read........__________..........__________.........__________

RealLife-60%Rand-65%Read......__________..........__________.........__________

Max Throughput-50%Read..........__________..........__________.........__________

Random-8k-70%Read.................__________..........__________.........__________

EXCEPTIONS: CPU Util.-XX%;

##################################################################################

I hope YOU JOIN IN !

Regards

Christian

A Google Spreadsheet version is here:

Message was edited by:

ken.cline@hp.com to remove ALL CAPS from thread title

Message was edited by:

RDPetruska

Added link to Atamido's Google Spreadsheet

Tags (1)
0 Kudos
457 Replies
christianZ
Champion
Champion

Well I think this all results here should give more direction for storage choice / throughput - I don't know any other (vendors) virtualisation product that works faster than VMWare and in addition we could saw here (after careful reading the postings) that in fc area the vm's results are only a bit worse that those from ph. servers (iscsi looks a bit more negativ).

And I believe when somebody posts here not truly results there will be always someone correcting them soon.

So I hope VMWare lets this THREAD RUNNING !

AND ONCE AGAIN THE HINT -

remember when your vcpu goes high the cpu clocks cycles (in vm) can change so that the results you can see in iometer are inexactly - one should always verify the results by e.g. VC or storage statistics (the block sizes from the tests are known).

0 Kudos
Atamido
Contributor
Contributor

Is there a quick comparison chart with all of this data? It's a bit hard to compare on the different pages and the skewed formatting.

0 Kudos
christianZ
Champion
Champion

Unfortunately none for now. Sorry.

0 Kudos
taylorb
Hot Shot
Hot Shot

Maybe Atamido could make one and share?

0 Kudos
Atamido
Contributor
Contributor

Sure I could, but then how would I get points for that? 😛

0 Kudos
pubare
Contributor
Contributor

Out of curiousity, where are you getting the "Av. IOs/sek-------Av. MB/sek" in the results? I'm not seeing those listed in the results.csv (using Iometer 2006.07.27). Are you computing them manually from other data?

I have some results I would like to share showing the differences between segment sizes on an IBM DS4300 (FastT 600) dual controller, but want to be sure and post them in the "accepted standard" format.

0 Kudos
christianZ
Champion
Champion

They all you can see in gui of iometer.

0 Kudos
Atamido
Contributor
Contributor

Maybe Atamido could make one and share?

Done.

http://files.commo.de/vmwaresan.html

(Better viewed in Firefox than IE)

A Google Spreadsheet version is here:

http://spreadsheets.google.com/pub?key=p2IFgyUF_v5Jn-7QobgY9Fw

I'd love some help to clean up the spreadsheet and add new entries. If you shoot me your email address, I'll add you as a collaborator.

0 Kudos
pubare
Contributor
Contributor

What I'm seeing is the "Total I/Os per second" and "Total MBs per Second" with no option for "Average" (which I was assuming the Av. stood for). Just to be safe, is the Av. referring to "Total"?

0 Kudos
christianZ
Champion
Champion

All those values there are average values.

0 Kudos
christianZ
Champion
Champion

Good job ! Many thanks for that.

Maybe one of forum moderators could put the two links into the first porsting - that would make the search easier! Thanks.

Regards

Christian

0 Kudos
christianZ
Champion
Champion

I will pm you on friday - today we have here a holiday.

0 Kudos
christianZ
Champion
Champion

My email-address is:

czimny@kdvz-frechen.de

Regards

Christian

0 Kudos
Ridiz
Enthusiast
Enthusiast

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS PHYS. WIN2003 / MS ISCSI INITIATOR / 1 X PS100E

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: Win2003 Std. PHYS.

CPU TYPE / NUMBER: CPU / 2

HOST TYPE: Proliant ML 530 G2, 4GB RAM, 2x XEON 2.4 GHz

STORAGE TYPE / DISK NUMBER / RAID LEVEL: EQL PS100E x 1 / 1x14+2x2 SATA / R50

SAN TYPE / HBAs : iSCSI; iSCSI softw. initiator in WIN2003, 1XGb NIC for iSCSI, Jumbos and Flow Control

##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

##################################################################################

Max Throughput-100%Read........____27____..........___2221___.........___69____

RealLife-60%Rand-65%Read......_____33____..........___1363___.........____11____

Max Throughput-50%Read..........____13____..........___4593___.........____144___

Random-8k-70%Read................._____33____..........___1323___.........____10____

EXCEPTIONS:

0 Kudos
Ridiz
Enthusiast
Enthusiast

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS PHYS. WIN2003 / MS ISCSI INITIATOR / 2 X PS100E

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: Win2003 Std. PHYS.

CPU TYPE / NUMBER: CPU / 2

HOST TYPE: Proliant ML 530 G2, 4GB RAM, 2x XEON 2.4 GHz

STORAGE TYPE / DISK NUMBER / RAID LEVEL: EQL PS100E x 2 / 2x14+2x2 SATA / R50

SAN TYPE / HBAs : iSCSI; iSCSI softw. initiator in WIN2003, 1XGb NIC for iSCSI, Jumbos and Flow Control

##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

##################################################################################

Max Throughput-100%Read........____18____..........___3117___.........___97____

RealLife-60%Rand-65%Read......_____22____..........___1939___.........____15____

Max Throughput-50%Read..........____13____..........___4390___.........____137___

Random-8k-70%Read................._____26____..........___1724___.........____14____

EXCEPTIONS:

0 Kudos
christianZ
Champion
Champion

Ridiz,

thanks for that. First time we can see here 2XPS100 with spanned volumes.

Regards

Christian

0 Kudos
Ridiz
Enthusiast
Enthusiast

Christian, I've been meaning to put those numbers up for a while. This thread has been extremely useful. I'll put up some numbers from ESX later today.

Rich

0 Kudos
christianZ
Champion
Champion

Ok, remember to verify your vm results e.g. over VC statistics (especially when the vcpu goes high).

Thanks.

0 Kudos
Ridiz
Enthusiast
Enthusiast

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS VM on ESX / EqualLogic 1 X PS100E

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: ESX 3.0.1

CPU TYPE / NUMBER: VCPU / 1

HOST TYPE: HP Proliant DL385 G2, 16GB RAM, 2x AMD 2.6 GHz

STORAGE TYPE / DISK NUMBER / RAID LEVEL: EQL PS100E x 1 / 1x14+2x2 SATA / R50

SAN TYPE / HBAs : iSCSI; ESX softw. initiator, 2XGb NIC for iSCSI

##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

##################################################################################

Max Throughput-100%Read........____16____..........___3485___.........__109____

RealLife-60%Rand-65%Read......_____46____..........___874___.........____6.8____

Max Throughput-50%Read..........____5____..........___1338___.........____42___

Random-8k-70%Read................._____57____..........___701___.........____6____

EXCEPTIONS: Done a few months ago w/ 3 hosts and about 30 VMs connected to storage

\##################################################################################

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TABLE OF RESULTS VM on ESX / EqualLogic 2 X PS100E

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

SERVER TYPE: ESX 3.0.1

CPU TYPE / NUMBER: VCPU / 1

HOST TYPE: HP Proliant DL385 G2, 16GB RAM, 2x AMD 2.6 GHz

STORAGE TYPE / DISK NUMBER / RAID LEVEL: EQL PS100E x 2 / 2x14+2x2 SATA / R50

SAN TYPE / HBAs : iSCSI; ESX softw. initiator, 2XGb NIC for iSCSI

##################################################################################

TEST NAME--


Av. Resp. Time ms--Av. IOs/sek---Av. MB/sek----

##################################################################################

Max Throughput-100%Read........____17____..........___3416___.........__107____

RealLife-60%Rand-65%Read......_____47____..........___852___.........____6.7____

Max Throughput-50%Read..........____5____..........___1526___.........____48___

Random-8k-70%Read................._____52____..........___770___.........____6____

EXCEPTIONS: These tests were done a couple months apart and the load on the group

has changed. This was run with 6 hosts and about 60 VMs connected to the storage.

\##################################################################################

Christian, I am not clear on what you mean by verify my number vs. VC. If I check VC

during the runs, I get 111, 8, 51 and 8 MB/sek peak respectively. If this isn't

what you meant, please let me know and I will rerun (I can only run the 2xPS100E now).

Thanks,

Rich

0 Kudos
christianZ
Champion
Champion

Yes VC statistics are mentioned.

When you made your test with ph. server

\- were the 60 vms still running,

\- was your volume spanned over 2 members with config it to R50 (or automatic)

\- to the tests with vms - what I noticed now/(ca. 30 vms running by me) is, that

when I set the volume to be spanned over 2 members I can't reach more ios than by volume to be configured only on one member and the capacity of this volume won't be distributed equal over the 2 members- that seems to be proportional to used space on each member - I missing here the more throughput, by first test I saw definitely much more ios (but there woren't any other vms on storage running then).

Can you see that phenomena too?

Have you tried not to spann the volumes and check the performance?

0 Kudos