This script allows users to create linked cloned virtual machine(s) from a master (or golden) virtual machine. Clones may be stored on any type of datastore (LOCAL, SAN, NFS) that is presented to the ESX host. The script is very capable in deploying a large VDI environment in a relatively short amount of time.
Compatiable with:
VMware ESX 3.5 Update 3
VMware ESX 3.5 Update 2
VMware ESX 3.5 Update 1
VMware ESX 3.5
VMware ESX Server 3.0.3
VMware ESX Server 3.0.2 Update 1
VMware ESX Server 3.0.2
VMware ESX Server 3.0.1
VMware ESX Server 3.0.0
There are currently 3 supported use cases:
1)Default
Generate linked clones located in the same datastore that houses the master (golden) virtual machine. Writes will be directed to their respective linked virtual machine delta file with reads originating from the master virtual machine.
2)
Distributed Write(s) I/O
Generate linked clones evenly across available datastores of choice. Reads will originate from the specified master virtual machine while writes are directed towards their respective virtual machine delta files that have been dispersed across the specified datastores.
3)
Distributed Read(s) & Write(s) I/O
Duplicate the master virtual machine on available datastores of choice and evenly generate linked clones off of each duplicated master virtual machine. Reads and writes will be isolated onto the datastore of the linked clone(s) in question. This process is similar to running multiple instances of the default behavior of the script.
Note: This linked clones script is virtual machine OS independent (i.e. it is not restricted to just Windows OS’s for VDI environment). For example, one can utilize this script to their advantage in development environments where new VMs need to be (mass) cloned quickly for testing.
DISCLAIMER - When deleting linked clones, our process requires that the admin utilizes the “my-vmware-cmd” toolset that has been developed. Under no circumstances should the “delete from disk” feature be used on a linked clone from within VIClient. This is to ensure that the master image of which clones are linked from is not terminated.
The following requirements will be validated by the script prior to execution:
- Master VM is offline
- Master VM does not contain any Snapshots or RDMs
- Valid eth0 interface
- Single VMDK disk (remove any instances in the .vmx file if you have others)
Example Execution of all 3 Use cases:
Storage Environment:
[root@olga lamw]# vmware-vim-cmd hostsvc/summary/fsvolume
Name Type SubType Path Device Capacity Mode
dlgCore-FC-LUN200.Templates VmfsVolume VMFS /vmfs/volumes/48e1cd81-81223593-6cd8-00215acaa2b2 vmhba1:0:200:1 133949292544 readWrite
dlgCore-FC-LUN202.VMstorage2 VmfsVolume VMFS /vmfs/volumes/48e1a4a6-3b55e69b-02a9-001f29c948e2 vmhba1:0:202:1 162940321792 readWrite
olga-local-SAS.Storage VmfsVolume VMFS /vmfs/volumes/49167992-ac31f0a5-3f9f-001f29c7a04c vmhba0:0:0:5 274072600576 readWrite
dlgCore-FC-LUN203.VMstorage3 VmfsVolume VMFS /vmfs/volumes/48e16ed0-420158d5-5601-00215acaa2b2 vmhba1:0:203:1 164014063616 readWrite
dlgCore-FC-LUN201.VMstorage1 VmfsVolume VMFS /vmfs/volumes/48e1b819-3443974c-03c4-001f29c948e2 vmhba1:0:201:1 161866579968 readWrite
[root@olga lamw]# vdf -h
Filesystem Size Used Avail Use% Mounted on
/dev/cciss/c0d0p2 7.9G 1.2G 6.3G 16% /
/dev/cciss/c0d0p1 244M 27M 204M 12% /boot
/dev/cciss/c0d0p7 2.0G 33M 1.9G 2% /home
none 391M 0 391M 0% /dev/shm
/dev/cciss/c0d0p8 2.0G 33M 1.9G 2% /tmp
/dev/cciss/c0d0p6 4.0G 289M 3.5G 8% /var
/vmfs/devices 273G 0 273G 0% /vmfs/devices
/vmfs/volumes/48e16ed0-420158d5-5601-00215acaa2b2
152G 108G 44G 70% /vmfs/volumes/dlgCore-FC-LUN203.VMstorage3
/vmfs/volumes/48e1a4a6-3b55e69b-02a9-001f29c948e2
151G 105G 46G 69% /vmfs/volumes/dlgCore-FC-LUN202.VMstorage2
/vmfs/volumes/48e1b819-3443974c-03c4-001f29c948e2
150G 126G 24G 84% /vmfs/volumes/dlgCore-FC-LUN201.VMstorage1
/vmfs/volumes/48e1cd81-81223593-6cd8-00215acaa2b2
124G 71G 53G 57% /vmfs/volumes/dlgCore-FC-LUN200.Templates
/vmfs/volumes/49167992-ac31f0a5-3f9f-001f29c7a04c
255G 1.3G 253G 0% /vmfs/volumes/olga-local-SAS.Storage
Default Case
- Master Virtual Machine: /vmfs/volumes/olga-local-SAS.Storage/UCSB-GAUCHOS/UCSB-GAUCHOS.vmx stored on LOCAL STORAGE
- Storage Medium: LOCAL STORAGE
- 32 Linked Clones on the same datastore as the Master VM
1. Confirmation after initial execution of script:
######################################################
#
# UCSB ResNet Linked Clones Tool for ESX 3.x
# Author: william2003[at]gmail[dot]com
# duonglt[at]engr[dot]ucsb[dot]edu
# Created: 06/26/2008
#
######################################################
Requested parameters:
Master Virtual Machine Image: /vmfs/volumes/olga-local-SAS.Storage/UCSB-GAUCHOS/UCSB-GAUCHOS.vmx
Requested Number of Linked Clones: 32 VMs
Initial Master Memory: 2048 MB
Initial Master VMDK Size: 10 GB
Expected Storage Consumption: 84 GB
Please verify that there is enough storage space for the requested configuration before running
Would you like to continue with these parameters y/n?
y
2. Linked Clones construction progress:
Generating Linked Clones ...
59% Complete! - Linked Clones Created: 19/32
3. Completed Linked Clones output:
Generating Linked Clones ...
100% Complete! - Linked Clones Created: 32/32
Waiting for Virtual Machine(s) to obtain their MAC addresses...
Linked clones VM MAC addresses stored at:
lcs_created_on-2008-12-09-210343/UCSB-GAUCHOS
Start time: Tue Dec 9 21:00:06 PST 2008
End time: Tue Dec 9 21:03:40 PST 2008
Duration : 3.57 Minutes
Maps before Default case execution:
Maps after Default execution:
Red denotes the Master VM
Green denotes the datastore
Orange denotes the last Linked Clone created
Distributed Write(s) I/O Case
- Master Virtual Machine: /vmfs/volumes/dlgCore-FC-LUN200.Templates/UCSB-RESNET/UCSB-RESNET.vmx stored on FC SAN
- Storage Mediums: LOCAL STORAGE and FC SAN
- 64 Linked Clones distributed across 2 datastores
1. Datstore selection:
#FREE SPACE DATASTORE(S)
1) 44G /vmfs/volumes/dlgCore-FC-LUN203.VMstorage3
2) 44G /vmfs/volumes/dlgCore-FC-LUN202.VMstorage2
3) 24G /vmfs/volumes/dlgCore-FC-LUN201.VMstorage1
4) 53G /vmfs/volumes/dlgCore-FC-LUN200.Templates
5) 253G /vmfs/volumes/olga-local-SAS.Storage
6) Quit
Please select datastore(s) to be used: (e.g. 1,2,3):
4,5
2. Confirmation after initial execution of script:
######################################################
#
# UCSB ResNet Linked Clones Tool for ESX 3.x
# Author: william2003[at]gmail[dot]com
# duonglt[at]engr[dot]ucsb[dot]edu
# Created: 06/26/2008
#
######################################################
Requested parameters:
Master Virtual Machine Image: /vmfs/volumes/dlgCore-FC-LUN200.Templates/UCSB-RESNET/UCSB-RESNET.vmx
Requested Number of Linked Clones: 64 VMs
Initial Master Memory: 2048 MB
Initial Master VMDK Size: 10 GB
Expected Storage Consumption: 167 GB
Please verify that there is enough storage space for the requested configuration before running
Distributed Write I/O Scheme selected. Linked clones will be evenly distributed across the following datastore(s):
/vmfs/volumes/dlgCore-FC-LUN200.Templates
/vmfs/volumes/olga-local-SAS.Storage
Would you like to continue with these parameters y/n?
y
3. Completed Linked Clones output:
Generating Linked Clones ...
100% Complete! - Linked Clones Created: 64/64
Waiting for Virtual Machine(s) to obtain their MAC Addresses...
Linked clones VM MAC addresses stored at:
lcs_created_on-2008-12-09-211724/UCSB-RESNET
Start time: Tue Dec 9 21:12:43 PST 2008
End time: Tue Dec 9 21:17:19 PST 2008
Duration : 4.60 Minutes
Maps before Default execution:
Maps after Default execution:
Red denotes the Master VM
Green denotes the two datastores
Orange denotes the last Linked Clones created
Distributed Read(s) & Write(s) I/O Case
- Master Virtual Machine: /vmfs/volumes/dlgCore-FC-LUN200.Templates/UCSB-ENGINEERING/UCSB-ENGINEERING.vmx stored on FC SAN
- Storage medium: FC SAN
- 128 Linked Clones distributed across 2 datastores
1. Datstore selection:
#FREE SPACE DATASTORE(S)
1) 44G /vmfs/volumes/dlgCore-FC-LUN203.VMstorage3
2) 43G /vmfs/volumes/dlgCore-FC-LUN202.VMstorage2
3) 24G /vmfs/volumes/dlgCore-FC-LUN201.VMstorage1
4) 53G /vmfs/volumes/dlgCore-FC-LUN200.Templates
5) 253G /vmfs/volumes/olga-local-SAS.Storage
6) Quit
Please select datastore(s) to be used: (e.g. 1,2,3):
1,2
2. Confirmation after initial execution of script:
######################################################
#
# UCSB ResNet Linked Clones Tool for ESX 3.x
# Author: william2003[at]gmail[dot]com
# duonglt[at]engr[dot]ucsb[dot]edu
# Created: 06/26/2008
#
######################################################
Requested parameters:
Master Virtual Machine Image: /vmfs/volumes/dlgCore-FC-LUN200.Templates/UCSB-ENGINEERING/UCSB-ENGINEERING.vmx
Naming convention for Linked-Cloned Master Virtual Machine: MASTER-LinkedClones-{1-2}
Naming convention for Linked-Cloned Virtual Machines: LinkedClones-{1-128}
Looking to create: 2 Master VMs
Requested Number of Linked Clones: 128 VMs
Initial Master Memory: 2048 MB
Initial Master VMDK Size: 10 GB
Expected Storage Consumption: 353 GB
Please verify that there is enough storage space for the requested configuration before running
Distributed Read & Write I/O Scheme selected. Operation will duplicate master image and evenly generate
linked clones from these dupulicated images across the specified datastore(s):
/vmfs/volumes/dlgCore-FC-LUN203.VMstorage3
/vmfs/volumes/dlgCore-FC-LUN202.VMstorage2
Distribution of Linked Clones per datastore: 64
MASTER-LinkedClones-1 will be stored on:
/vmfs/volumes/dlgCore-FC-LUN203.VMstorage3
The following VMs will be linked to "MASTER-LinkedClones-1" under this datastore:
LinkedClones-{1-64}
MASTER-LinkedClones-2 will be stored on:
/vmfs/volumes/dlgCore-FC-LUN202.VMstorage2
The following VMs will be linked to "MASTER-LinkedClones-2" under this datastore:
LinkedClones-{65-128}
Would you like to continue with these parameters y/n?
y
3. Creating Master image on each Datastore to distribute read(s)/write(s) Linked Clones
Creating MASTER-LinkedClones-1 on: /vmfs/volumes/dlgCore-FC-LUN203.VMstorage3
Creating MASTER-LinkedClones-2 on: /vmfs/volumes/dlgCore-FC-LUN202.VMstorage2
Initiating vmkfstools copy ...
Initiating vmkfstools copy ...
0% Complete! - Linked Clones Created: 0/128
Registering newly created MASTER-LinkedClones-1 ...
Registering newly created MASTER-LinkedClones-2 ...
2% Complete! - Linked Clones Created: 3/128
4. Completed Linked Clones output:
100% Complete! - Linked Clones Created: 128/128
Waiting for Virtual Machine(s) to obtain their MAC addresses...
Linked clones VM MAC addresses stored at:
lcs_created_on-2008-12-09-214627/MASTER-LinkedClones-1
lcs_created_on-2008-12-09-214627/MASTER-LinkedClones-2
Start time: Tue Dec 9 21:36:49 PST 2008
End time: Tue Dec 9 21:46:22 PST 2008
Duration : 9.55 Minutes
Maps before Default execution:
Maps after Default execution:
Red denotes the Master VM
Green denotes the two datastores
Purple denotes the 2 replicated Master VM in each of the datastores
Orange denotes the last Linked Clones created
Known Issues:
1. When running "
ghetto-esx-linked-clones.sh" on VMware ESX 3.0.0, the
"FULL_PATH_TO_MASTER_VMX_FILE" must be in the UUID format and not the human readable symlink, this was fixed in VMware ESX 3.0.1 and possibly updated in one of the 3.0.0 patchset.
Solution, use the following:
./ghetto-esx-linked-clones.sh /vmfs/volumes/4857f047-4e4ec6bf-a8b8-001b78361a3c/VM-MasterImage/VM-MasterImage.vmx LinkedClone- 1 200
Instead of
./ghetto-esx-linked-clones.sh vmfs/volumes/himalaya-local-SATA.Storage/VM-MasterImage/VM-MasterImage.vmx LinkedClone- 1 200
