Hello everybody,
the old thread seems to be sooooo looooong - therefore I decided (after a discussion with our moderator oreeh - thanks Oliver -) to start a new thread here.
Oliver will make a few links between the old and the new one and then he will close the old thread.
Thanks for joining in.
Reg
Christian
We saw similar numbers on our PS4000E until we ran the configuration script included with the multipathing extension module on each of the hosts. High latency and low IOPS across the board. Afterwards nearly half the latency and double the IOPS. Have the results at the office.
@mikeyb79
Thanks for the tip. I will try this very soon. I plan on installing an addtional 4 pNICs into each host this week. I guess this was perfect timing.
I will post the results when finished.
Here are the numbers we saw from our PS4000E using the standard testing:
Dell EqualLogic PS4000E (No MEM Installed)
Access Specification Name
IOps
MBps (Binary)
Average Response Time
Max Throughput-100%Read
3493.163964
109.161374
17.178869
RealLife-60%Rand-65%Read
932.360549
7.284067
50.225846
Max Throughput-50%Read
4878.569157
152.455286
12.162984
Random-8k-70%Read
839.927753
6.561936
56.769025
Dell EqualLogic PS4000E (MEM Installed)
Access Specification Name
IOps
MBps (Binary)
Average Response Time
Max Throughput-100%Read
6729.109541
210.284673
8.874006
RealLife-60%Rand-65%Read
970.692039
7.583532
47.552126
Max Throughput-50%Read
6419.513468
200.609796
9.181681
Random-8k-70%Read
837.034044
6.539328
56.821859
Do you have a post MEM results?
Wow, that formatting turned out horrible. Both non-MEM and MEM results are there, in the read results there were large improvements. Random is fairly even between then two, with slightly better performance on the Real Life bench on the MEM-enabled system.
Hi all,
I'm getting strange results with http://vmktree.org/iometer/ and IOMeter 1.1.0
While analyzing the .csv, it seems that the csv output isn't interpreted as expected, the CGI taking the 9th column "Write MBps (Decimal)" instead of the 12th "Average Response Time" for the latency.
As an easy way to see that, I'm getting 0ms for the 100% read test.
I'm suspecting too that the CPU load isn't either picked where it should be. Could someone check on its side ?
Here attached is one of my CSV with faulty interpretation.
attached CSV
Solved ! IOMeter 1.1.0 (devel) doesn't have the expected output format. Use the regular versions
a
Added the Dell MEM and upgraded the host ti ESXi 5.1
Test name | Latency | Avg iops | Avg MBps | cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 8.50 | 6896 | 215 | 3% |
RealLife-60%Rand-65%Read | 22.73 | 1949 | 15 | 28% |
Max Throughput-50%Read | 7.27 | 7952 | 248 | 7% |
Random-8k-70%Read | 24.91 | 1805 | 14 | 27% |
Previous results
Test name | Latency | Avg iops | Avg MBps | cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 31.32 | 1852 | 57 | 4% |
RealLife-60%Rand-65%Read | 21.40 | 2050 | 16 | 17% |
Max Throughput-50%Read | 21.02 | 2793 | 87 | 2% |
Random-8k-70%Read | 23.91 | 1838 | 14 | 18% |
****New test was on a different host.
Dear sender,
Thanks for your e-mail. I am currently on holidays with no e-mail access. I will be back the 8th of January
In urgent cases send me an sms to 00491743194824.
Feliz Navidad y prospero año nuevo!
Manuel
SERVER TYPE: VM Windows 2008 R2
Av Resp Time | Av IO/Sek | Av MB/sek | % CPU | |
Max Throughput-100%Read | 0,47 | 12456,25 | 389,45 | 2,75 |
RealLife-60%Rand-65%Read | 0,72 | 62790,45 | 486,74 | 9,37 |
Max Throughput-50%Read | 0,96 | 19924,08 | 622,28 | 4,23 |
Random-8k-70%Read | 0,79 | 62002,10 | 484,36 | 9,39 |
These results might show some kind of interests. They exercise the current 60 slots Engenio (now NetApp) base controllers Dell MD3660, NetApp E2660, SGI IS5000, IBM DSC3700 Base.
Here is a Dell MD3660F, dual controller, dual FC 8GB attachement, 2GB/controller.
1 Worker
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 2vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / SAS 300 15krpm / RAID5 4+1 (5 discs)/ALUA Round Robin / Cache block size 4k / Dynamic cache prefetch
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 2.62 | 22408 | 700 | 69% |
RealLife-60%Rand-65%Read | 23.81 | 2152 | 16 | 6% |
Max Throughput-50%Read | 4.39 | 13604 | 425 | 50% |
Random-8k-70%Read | 25.28 | 1966 | 15 | 6% |
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 2vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / SAS 300 15krpm / RAID5 4+1 (5 discs)/ALUA MRU/ Cache block size 4k/ Dynamic cache prefetch
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 2.63 | 22393 | 699 | 68% |
RealLife-60%Rand-65%Read | 24.38 | 2086 | 16 | 7% |
Max Throughput-50%Read | 4.25 | 14092 | 440 | 51% |
Random-8k-70%Read | 25.42 | 1931 | 15 | 7% |
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 4vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / SAS 300 15krpm / RAID5 4+1 (5 discs) /ALUA Round Robin / 1 Workers/ Cache block size 8k / Dynamic cache prefetch disabled
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 2.60 | 23163 | 723 | 1% |
RealLife-60%Rand-65%Read | 25.67 | 2034 | 15 | 1% |
Max Throughput-50%Read | 2.89 | 20747 | 648 | 2% |
Random-8k-70%Read | 27.04 | 1881 | 14 | 15% |
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 2vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / SAS 300 15krpm / RAID5 8+1 (9 discs)/ALUA Round Robin/ Cache block size 4k/ Dynamic cache prefetch
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 2.62 | 22407 | 700 | 69% |
RealLife-60%Rand-65%Read | 13.72 | 3544 | 27 | 8% |
Max Throughput-50%Read | 4.23 | 14176 | 443 | 51% |
Random-8k-70%Read | 14.20 | 3185 | 24 | 10% |
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 2vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / SAS 300 15krpm / RAID5 8+1 (9 discs)/ALUA MRU/ Cache block size 4k/ Dynamic cache prefetch
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 2.62 | 22401 | 700 | 70% |
RealLife-60%Rand-65%Read | 14.13 | 3430 | 26 | 6% |
Max Throughput-50%Read | 4.29 | 13959 | 436 | 52% |
Random-8k-70%Read | 14.09 | 3215 | 25 | 9% |
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 2vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / SAS 300 15krpm / RAID6 DDP (11 discs)/ALUA Round Robin/ Cache block size 4k/ Dynamic cache prefetch
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 2.62 | 22426 | 700 | 69% |
RealLife-60%Rand-65%Read | 14.17 | 3743 | 29 | 7% |
Max Throughput-50%Read | 4.05 | 14779 | 461 | 53% |
Random-8k-70%Read | 14.39 | 3466 | 27 | 8% |
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 2vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / NLSAS 2TB 7200rpm / RAID6 8+2 (10 discs)/ALUA Round Robin/ Cache block size 4k/ Dynamic cache prefetch
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 2.64 | 22249 | 695 | 11% |
RealLife-60%Rand-65%Read | 29.23 | 1505 | 11 | 33% |
Max Throughput-50%Read | 4.28 | 14010 | 437 | 7% |
Random-8k-70%Read | 28.48 | 1420 | 11 | 37% |
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 4vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / SAS 300GB 15krpm / RAID10 11+11 (22 discs)/ALUA Round Robin/ Cache block size 4k/ Dynamic cache prefetch / 2x8Gb FC attachements
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 2.70 | 21991 | 687 | 12% |
RealLife-60%Rand-65%Read | 8.17 | 6688 | 52 | 8% |
Max Throughput-50%Read | 2.99 | 20107 | 628 | 12% |
Random-8k-70%Read | 8.02 | 6583 | 51 | 7% |
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 4vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / NLSAS 2TB 7k2rpm / RAID10 11+11 (22 discs)/ALUA Round Robin/ Cache block size 4k/ Dynamic cache prefetch/ 2x8Gb FC attachements
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 2.71 | 21991 | 687 | 13% |
RealLife-60%Rand-65%Read | 15.45 | 3450 | 26 | 7% |
Max Throughput-50%Read | 2.91 | 20784 | 649 | 12% |
Random-8k-70%Read | 14.76 | 3511 | 27 | 7% |
2 Workers
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 4vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / SAS 300 15krpm / RAID5 8+1 (9 discs) + RAID5 4+1 (5 discs) /ALUA Round Robin / 2 Workers/ Cache block size 4k/ Dynamic cache prefetch
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 2.94 | 40411 | 1262 | 25% |
RealLife-60%Rand-65%Read | 18.38 | 5405 | 42 | 10% |
Max Throughput-50%Read | 5.60 | 21272 | 664 | 27% |
Random-8k-70%Read | 19.57 | 4915 | 38 | 9% |
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 4vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / SAS 300 15krpm / RAID5 8+1 (9 discs) + RAID5 4+1 (5 discs) /ALUA Round Robin / 2 Workers/ Cache block size 8k/ Dynamic cache prefetch
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 2.84 | 41844 | 1307 | 26% |
RealLife-60%Rand-65%Read | 19.59 | 5088 | 39 | 10% |
Max Throughput-50%Read | 4.23 | 28051 | 876 | 27% |
Random-8k-70%Read | 19.89 | 4794 | 37 | 9% |
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 4vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / SAS 300 15krpm / RAID5 8+1 (9 discs) + RAID5 4+1 (5 discs) /ALUA Round Robin / 2 Workers/ Cache block size 8k / Dynamic cache prefetch disabled
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 2.85 | 41649 | 1301 | 26% |
RealLife-60%Rand-65%Read | 19.30 | 5153 | 40 | 10% |
Max Throughput-50%Read | 3.96 | 29539 | 923 | 27% |
Random-8k-70%Read | 19.91 | 4821 | 37 | 9% |
Cache block size 8k / Dynamic cache prefetch disabled clearly shows a better usage of the limited cache of the controller for this bench
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 4vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / SAS 300 15krpm / RAID5 14+1 (15 discs) + RAID5 14+1 (15 discs) /ALUA Round Robin / 2 Workers/ Cache block size 8k / Dynamic cache prefetch disabled
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 0.89 | 43114 | 1347 | 30% |
RealLife-60%Rand-65%Read | 11.09 | 9015 | 70 | 15% |
Max Throughput-50%Read | 3.73 | 30744 | 960 | 28% |
Random-8k-70%Read | 10.70 | 8656 | 67 | 12% |
3 Workers
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 4vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610 x 3
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / NLSAS 2TB 7k2rpm / RAID6 8+2 (10 discs) : 3 volumes /ALUA Round Robin / 3 Workers/ Cache block size 8k / Dynamic cache prefetch disabled / Only 2 FC8 attachements
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 2.76 | 46675 | 1458 | 16% |
RealLife-60%Rand-65%Read | 35.44 | 3909 | 30 | 9% |
Max Throughput-50%Read | 4.28 | 33055 | 1046 | 13% |
Random-8k-70%Read | 36.11 | 3807 | 29.6 | 10% |
6 Workers
SERVER TYPE: VM Windows 2008 R2 64bits / ESX 4.1U3 / 4vCPU
CPU TYPE / NUMBER: X5680 / 2
HOST TYPE: M610 x 3
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Dell DS3660F / SAS 300 15krpm / RAID5 4+1 (5 discs) : 6 volumes /ALUA Round Robin / 6 Workers/ Cache block size 8k / Dynamic cache prefetch disabled / Only 2 FC8 attachements
TEST NAME | Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 7.08 | 50364 | 1573 | 32% |
RealLife-60%Rand-65%Read | 25.87 | 12120 | 94.8 | 15% |
Max Throughput-50%Read | 9.45 | 38939 | 1216 | 15% |
Random-8k-70%Read | 26.57 | 11742 | 91 | 15% |
FC bandwidth limitation on 100% Read
Bis zum 07.01.13 bin ich nicht im Büro. Bitte wenden Sie sich in dieser Zeit an support@mdm.de. I'm out of office until January 7th 2013, please contact support@mdm.de instead.
Hello all,
Here are some results from our Netapp C-Mode system currently being put through it's paces. So far I am pretty happy with the performance, but there is still some tweaking to do before we go into production with it.
First up, results with 2x SAS 600GB 15k shelves (38 data disks):
SERVER TYPE: Win2008 R2, 1GB mem CPU TYPE / NUMBER: 1 vCPU E5540, 2.5GHz HOST TYPE: HP DL380G6, 2x E5540, 2.5GHz STORAGE TYPE / DISK NUMBER / RAID LEVEL: Netapp 3240, ONTAP 8.1.2 C-Mode/ 38X 600GB SAS 15K, RAID-DP 10G NFS through Nexus 5010 | ||||
Test name | Latency | Avg iops | Avg MBps | cpu load |
Max Throughput-100%Read | 5.30 | 11347 | 354 | 0% |
RealLife-60%Rand-65%Read | 3.70 | 11658 | 91 | 70% |
Max Throughput-50%Read | 5.10 | 11476 | 358 | 0% |
Random-8k-70%Read | 3.33 | 12590 | 98 | 73% |
Next, results using SATA drives + SSD in a Flash Pool (SSD cache in front of the SATA drives). Only 12 SATA drives in this aggregate, but will likely increase that to the full 24 drive shelf, so that should help the Real Life test result.
SERVER TYPE: Win2008 R2, 1GB mem CPU TYPE / NUMBER: 1 vCPU E5540, 2.5GHz HOST TYPE: HP DL380G6, 2x E5540, 2.5GHz STORAGE TYPE / DISK NUMBER / RAID LEVEL: Netapp 3240, ONTAP 8.1.2 C-Mode/ 12X 3TB Sata 7.2 + 6x 100GB SSD (Flash Pool), RAID-DP 10G NFS through Nexus 5010 | ||||
Test name | Latency | Avg iops | Avg MBps | cpu load |
Max Throughput-100%Read | 4.66 | 12890 | 402 | 42% |
RealLife-60%Rand-65%Read | 7.53 | 5279 | 41 | 64% |
Max Throughput-50%Read | 4.85 | 12066 | 377 | 42% |
Random-8k-70%Read | 2.83 | 17612 | 137 | 66% |
Thanks,
Tim
Sehr geehrte Damen und Herren,
ich befinde mich im Urlaub.
Bei drigenden Fragen wenden sie sich bitte an
Herr Becker Tel.: 06027409030.
Mit freundlichen Grüßen
Sven Scheller
Server Type: | Win 2008R2 on ESX 5.1 | |||
Host Type: | Dell R320 | |||
Storage Type: | LOCAL PERC H310 RAID 10 4xSAS 600GB 15k | |||
Avg Resp. Time ms | Avg IOs/sec | Avg MB/sec | % cpu load | |
Max Throughput-100%Read | 13 | 4482 | 140 | 26 |
RealLife-60%Rand-65%Read | 64 | 891 | 7 | 18 |
Max Throughput-50%Read | 67 | 861 | 27 | 18 |
Random-8k-70%Read | 63 | 907 | 7 | 18 |
These values are very low for a local storage test, right? Especially the 50% test? What could be the problem? Thanks!
This is nice..
Test name | Latency | Avg iops | Avg MBps | cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 8.70 | 6655 | 207 | 6% |
RealLife-60%Rand-65%Read | 14.74 | 3191 | 24 | 50% |
Max Throughput-50%Read | 9.60 | 6027 | 188 | 53% |
Random-8k-70%Read | 16.06 | 2901 | 22 | 58% |
Dell mem is officially unsupported on Essentials Plus license. Going to switch over to Round Robin. Logging this for comparison.
Test name | Latency | Avg iops | Avg MBps | cpu load |
---|---|---|---|---|
Max Throughput-100%Read | 8.53 | 6704 | 209 | 10% |
RealLife-60%Rand-65%Read | 12.73 | 3535 | 27 | 18% |
Max Throughput-50%Read | 8.70 | 6606 | 206 | 29% |
Random-8k-70%Read | 12.91 | 3543 | 27 | 17% |
Switched from Dell Multipath Module to Round Robin because of Enterprise licensing requirement for 3rd party multipathing. Improvements across the board (see above!) But one test either way probably doesn't count as evidence.
jb
Seems to me you didnt get that large of an increase but any improvement helps. Try setting the round robin policy limits to IOPs, and set the limit to 1. Re-run your tests and see the difference
This is what I run from PowerCLI to make the changes to the entire cluster and all the LUN's:
$Cluster = Read-host "Enter Cluster Name"
foreach ($vmhost in get-cluster $Cluster | get-VMHost){
Connect-VIServer -Server $vmhost.Name -User root -Password <yourpasswd>
$esxcli = Get-EsxCli -VMHost $vmhost.Name
$esxcli.storage.nmp.satp.set($false,"VMW_PSP_RR","VMW_SATP_SYMM")
foreach ($lunpath in $esxcli.storage.nmp.device.list() | where {$_.device -like "naa.*"}){
$esxcli.storage.nmp.device.set($null,$lunpath.Device, "VMW_PSP_RR")
$esxcli.storage.nmp.psp.roundrobin.deviceconfig.set($null,$lunpath.Device,1,"iops",$null)
}
}