Slower VMDK speed than RDM + recommendation

[adriankoooo]

I have two questions. Is there any reasonable reason why WD-RED HDD in RDM (139MB/s) is much faster than my WD-RED in VMDK (98MB/s)? Not true for my SSD, which is enough fast under VMDK.

madrian@ubuntu:~# #SSD (VMDK)

madrian@ubuntu:~$ sync; dd if=/dev/zero of=tempfile bs=1M count=1024; sync

1024+0 records in

1024+0 records out

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.90094 s, 370 MB/s

madrian@ubuntu:~$#VMDK

madrian@ubuntu:~$ cd /media/hdd

madrian@ubuntu:/media/hdd$ sync; dd if=/dev/zero of=tempfile bs=1M count=1024; sync

1024+0 records in

1024+0 records out

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.958 s, 98.0 MB/s

madrian@ubuntu:/media/hdd$ #RDM

madrian@ubuntu:/media$ cd backup/

madrian@ubuntu:/media/backup$ sync; dd if=/dev/zero of=tempfile bs=1M count=1024; sync

1024+0 records in

1024+0 records out

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.70613 s, 139 MB/s

madrian@ubuntu:/media/backup$

I get same values with fio.

My other question is: what do you recommend for me: I have 2-3 Linux OS installed on my Esxi server/primary hdd is an ssd. On my second WD RED I have not very important data. Currently on this HDD I am using VMDK and today I wanted shrink it's size. As I read it's not possible, only by Converter. Meh, I'll skip this. I am now thinking in RDM because it's easier to maitance I think. In this case I would shrink the main partition and create a new one for other OS. What do you think? Or maybe I am going to start again, format the drive and start with a smaller VMDK? Another possibility is to use (multiple) VMDK for different OSes and use thin provisioning to not use full disk? I can't decide which is the best way for the future (expand/shrink disk freely).

[adriankoooo]

up

[cyberpaul]

Hi, I have no answer for question 1. I have never tried this, but in a way it makes sense. VMFS means an additional layer between the guest and the device, so in a straightforward test like this, one would expect slightly lower performance. VMware claims that VMFS is optimized for real-life situations and its benefits should outweight losses.

As for question 2, I would always use VMFS and create thin provisioned virtual disks only as big as I need. Personally, I do not use partition tables nor the shrink operation. From my experience, I find this layout the easiest to manage with linux VMs:

• small (300 MB) virtual disk as /boot partition, partition table in place, no LVM (sda1 mounted to /boot)
• decent size (10 - 100 GB) virtual disk as root, no partition table, LVM PV created directly on sdb
• optionally more virtual disks for other data, no partition table, LVM PV directly on block device

There is a great advantage to this - you can expand sdb (and other data disks) without reboot - just perform scsi rescan, pvresize, lvextend and FS grow. Or even easier, add a new virtual disk, create a new PV and join it with current volume group.

If you ever need to shrink, you can add a new virtual disk with the appropriate size, evacuate old virtual disk with pvmove and vgreduce and then remove it. You'll need to power off your VM though. This is the only way to "shrink" without using the converter as VMware does not support virtual disk shrinking.

Stay away from RDM unless you really need it...