Creating a read-only vmdk to share between multiple VM's

[sirus569]

Hi,

For the past year, we have been upgrading our physial server to Virtual Machines. We have been using VMWare ESXi 5.0 for about 9 months now and it has been great so far.

Our use case is that we have about 10 VMs on a VM Host to build our product. Each VM has to have a "tools" drive - a 200 GB drive that has the different 3rd party tools we use: ex, all the different compilers or OS specific toolchains needed in our build process.

Previously, we had physical machines that mounted a NFS folder that stored all the compilers. We had terrible speeds - it would take about 3 hours to build something that should take 45 mins(which is how long it takes now).

Unfortunately, we are running out of disk space on our NAS since we have a 200 GB drive replicated 10 times. That is about 2 TBs of wasted space.

I was wondering if there was a way to have a single VMDK shared between the VMs. It doesn't have to be 1 VMDK for all 10 VMs. Even if it was a 1-2 mapping, that would be a great improvment over the 1-1 mapping we have now. It only has to serve up the tools for reading and executing the files, the VMs using it are not going to write to this VMDK.

Our current setup is as such: 2 Dell T-410 VM Hosts connected to a QNAP TS-879. The Dell hosts are connected to the QNAP's over a iSCSI adapter. 

A VM has: 1 20 GB boot vmdk, 1 200 GB "tools" vmdk and 1 20 GB "storage" vmdk; the storage vmdk is on a SSD disk.

I have read a lot about possibly creating a read-only vmdk, a cluster of VMs or having a VM serve up the vmdk as a NFS mount [since we have already tried with NFS mount points and experienced a lot of time related pain, I am not too keen on going back to this method]. We want to preserver the performance we are getting with the 1-1 mapping of tools drive to VMs with the new approach.

What are my options here? Is it possible to create a single VMDK to share? Is NFS the only approach other than having 10, 200 GB VMDKs? Any suggestion/help is greatly appreciated.

I hope I have been able to outline our use case clearly. As a heads up, even though I have been working with VMWare for the past 9 months, I would still say I am fairly new to VMs and the IT side of things. So please bear with me if I ask a lot of follow up questions.

Thank you,

Abhi

[sketchy00]

Since I administer an IT environment for a software Development Company, I think I can contribute here.  We too have a 3rdpartylibs directory for similar needs, that is also handled withing source code control (subversion).  Your best bet is going to be to have this simply as an NFS export served up on a dedicated VM (aliased of course so that you can interchange it easily) that would be mountable by these other systems.  You may choose to have some stuff in local sandboxes in each VM, but essentially, it will all come from /3rdpartylibs.  If a file system isn't capable of being clustered, I wouldn't recommend trying to make it do something that it is not capable of doing.

I would suggest that you look at your underlying configuration to make sure that your storage and network setup are adequate (vswitches, portgroups, teaming, etc.), as well as the underlying infrastructure that is serving up that storage.  That can make a signifigant difference, for better or worse.  Since it sounds like you are using just one host, remember that "network" traffic to other VM's on that same host, while still running up and down the protocol stack, are just running through the system bus, so you should see good performance.  Also, it is worth evaluating the specs of your VMs.  Linux VM's can be a bit misleading in some respects.  They are almost too efficient with their RAM, and will use everything it's got for file I/O.  Spec it under 2GB for a lot of file copying, or svn checkouts, etc., and performance is going to suffer.  Of course, if you are relatively new, underprovisioning is not usually the problem (overprovisioning typically is).

It might be helpful if you gave some performance numbers so that one can tell if you are suffering from abnormal performance or not.

[weinstein5]

Welcome to the Community - Why not just create a single VM that has the 200 GB tools dirve and share it out either via NFS or CIFS share as you when it was all physical - 

[happytohelp]

Anyone considered to symlink the  VMDKs extents from the other machine and share them like this?

The reference file needs to be copied as the references to the content will be different.
We dont have to tell the host what we do when we are only reading.....

I think the symlink on the file content level could be the thinest extra layer to serve the content.

Much thinner than any network stack!

Finally the symlink exists also on NTFS...

[sirus569]

We ended up implementing the NFS share at least on Linux and it has worked out great so far. On Windows, we have abysmal performance with a NFS share so we just have smaller individual VMDKs that are updated through a script.

VMware actually has a way to share VMDKs between virtual machines. Here is how I can do it (tried on ESXi 5.0)

1. Create a new VMDK but attach it to a NEW iSCSI adapter (if you have more than 1 VMDK on your VM). So this will show up as scsi(1:0) instead of scsi(0:1) assuming scsi(0:0) is your base disk.

2. Select the newly attached iscsi adapter in the VM's properties box and select SCSI bus sharing to be either physical or virtual per your needs.

Hope this helps someone.

[Stormarov]

We have tried this approach as well.  It works well and is easy to set up.

However there are some caveats

You cannot make this shared vmdk non-persistent.  Also while you can set the vmdk as independent you still will not be able to create snapshots as VMware will not snapshot any system with an iscsi device set up in either virtual or physical bus sharing.  If you have a backup system that uses snapshotting as a method to perform backups in you environment it will fail to back up these systems configure to use bus sharing.

I wish VMware would correct this so that these devices can be both non-persistent (or strictly read only) and able to be snapshotted if they only contain independent drives on the iscsi adaptor set up for bus sharing.

The workaround for this is to place an agent on the OS that resides on these systems and use agentful backups.  However this usually uses a different license than the agentless backups which are more expensive and more to manage.