I'm assuming there's some optimisation/configuration options I should be changing in VMware to optimise for local SSD datastores, as I'm getting horrible SSD write performance when using VMWare compared to just raw Ubuntu on the same hardware.
Host:
HP Workstation
Xeon CPU W3550 @ 3.07GHz
16GB ECC RAM
Local SSD (no RAID, straight AHCI)
Remote NFS Datastore (Not being used in this test, just FYI)
Test 1:
Host: VMWare 6.5.0 (Build 4887370) installed on USB.
VM: Ubuntu Server 17.04. VMWare Paravirtual controller. 4 CPU. 4GB RAM.
u1@vmh1:~$ sudo hdparm -t /dev/mapper/vmh1--vg-root
/dev/mapper/vmh1--vg-root:
Timing buffered disk reads: 698 MB in 3.01 seconds = 232.02 MB/sec
u1@vmh1:~$ sudo hdparm -t /dev/mapper/vmh1--vg-root
/dev/mapper/vmh1--vg-root:
Timing buffered disk reads: 696 MB in 3.00 seconds = 231.78 MB/sec
u1@vmh1:~$ sudo hdparm -T /dev/mapper/vmh1--vg-root
/dev/mapper/vmh1--vg-root:
Timing cached reads: ^[[A14270 MB in 2.00 seconds = 7139.60 MB/sec
u1@vmh1:~$ sudo hdparm -T /dev/mapper/vmh1--vg-root
/dev/mapper/vmh1--vg-root:
Timing cached reads: 14334 MB in 2.00 seconds = 7171.26 MB/sec
u1@vmh1:~# dd if=/dev/zero of=/u1/testfile bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 365.22 s, 2.9 MB/s
u1@vmh1:~# dd if=/dev/zero of=/u1/testfile bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 509.536 s, 2.1 MB/s
During this process, esxtop was showing high latency values:
DEVICE PATH/WORLD/PARTITION NPH NWD NPN DQLEN WQLEN ACTV QUED %USD LOAD CMDS/s READS/s WRITES/s MBREAD/s MBWRTN/s RESV/s CONS/s DAVG/cmd KAVG/cmd GAVG/cmd QAVG/cmd
mpx.vmhba32:C0:T0:L0 - 1 2 6 1 - 0 0 0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
t10.ATA_____INTEL_SSDSA2BW160G3H____ - 1 16 1 31 - 2 0 6 0.06 7.62 0.00 4.76 0.00 1.92 0.00 0.00 284.81 99.20 384.01 5314.41
{NFS}NAS_V1_NFS_01 - - - - - - 0 - - - 7.62 0.48 7.15 0.00 0.03 - - - - 0.43 -
Test 2:
Host: Ubuntu Server 17.04 installed on SSD.
u1@tinh1:~# hdparm -t /dev/mapper/tinh1--vg-u1
/dev/mapper/tinh1--vg-u1:
Timing buffered disk reads: 668 MB in 3.00 seconds = 232.57 MB/sec
u1@tinh1:~# hdparm -t /dev/mapper/tinh1--vg-u1
/dev/mapper/tinh1--vg-u1:
Timing buffered disk reads: 670 MB in 3.00 seconds = 233.29 MB/sec
u1@tinh1:~# hdparm -T /dev/mapper/tinh1--vg-u1
/dev/mapper/tinh1--vg-u1:
Timing cached reads: 16254 MB in 2.00 seconds = 8133.97 MB/sec
u1@tinh1:~# hdparm -T /dev/mapper/tinh1--vg-u1
/dev/mapper/tinh1--vg-u1:
Timing cached reads: 15564 MB in 2.00 seconds = 7787.91 MB/sec
u1@tinh1:~# dd if=/dev/zero of=/u1/testfile bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 5.17446 s, 208 MB/s
u1@tinh1:~# dd if=/dev/zero of=/u1/testfile bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 5.27765 s, 203 MB/s
Any ideas?
