This script allows users to create linked cloned virtual machine(s) from a master (or golden) virtual machine. Clones may be stored on any type of datastore (LOCAL, SAN, NFS) that is presented to the ESXi host. The script is very capable in deploying a large VDI environment in a relatively short amount of time.
Compatiable with:
VMware ESXi 4.0 Update 1
VMware ESXi 4.0
VMware ESXi 3.5 Update 5
VMware ESXi 3.5 Update 4
VMware ESXi 3.5 Update 3
VMware ESXi 3.5 Update 2
VMware ESXi 3.5 Update 1
VMware ESXi 3.5
Note: This linked clones script is virtual machine OS independent (i.e. it is not restricted to just Windows OS’s for VDI environment). For example, one can utilize this script to their advantage in development environments where new VMs need to be (mass) cloned quickly for testing.
The only supported case is the "Default case" in ghetto-esx-linked-clones.sh.
~ # ./ghetto-esxi-linked-clones.sh ###################################################### # # UCSB ResNet Linked Clones Tool for ESXi # Author: william2003[at]gmail[dot]com # duonglt[at]engr[dot]ucsb[dot]edu # Created: 09/30/2008 # ###################################################### Usage: ghetto-esxi-linked-clones.sh [FULL_PATH_TO_MASTER_VMX_FILE] [VM_NAME] [START_#] [END_#] i.e. ./ghetto-esxi-linked-clones.sh /vmfs/volumes/4857f047-4e4ec6bf-a8b8-001b78361a3c/LabMaster/LabMaster.vmx LabClient- 1 200 Output: LabClient-{1-200} ~ #
*Note:* Limited testing has been done with ESXi 3.5u3+ and EXi 4.0+
Forgot to mention, you can also find more scripts/resources located at:
Thanks for sharing this script I tested it and its worked great job!
but after creating a snapshot or more of linked clone it's not possible to delete the snapshot.
I got this error message:
an attempt was made to write to a read-only file
actually I checked the snapshot file properties and change it to 0755 but keep the same error.
can you help me please?
What issues are you having? Once you've created your pristine image and generated the Linked Clones off of that base image, you can use the following documentation for further configurations to make each LC unique: http://communities.vmware.com/docs/DOC-9201 , this is the implementation that my colleague Tuan and I have deployed into production at UCSB.
=========================================================================
--William
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
this is the situation
I have created 10 Linked clones of a golden image, I can start all of them and it's working, also I can take 1 or more snapshots from each linked clones but i can not remove the snapshots.
I got this error message:
an attempt was made to write to a read-only file.
Are you creating the new snapshots using the VI Client or through the console? What version of ESXi are you running? In either case, we've not heard or run into this issue before, if I get some time will try to test but I'll be pretty busy with some new projects.
=========================================================================
--William
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
yes through VI Client , I don't no how through console.
I'm using ESXi 3i, 3.5.0, 123629
also we have ESX server 3.5 same problem.
I had the same problem and it turned out that the clone was in a template mode.
http://hash-depot.blogspot.com/2009/05/vmwares-esxi-snapshot-removal-problem.html
Check advanced settings for templateVM and if its set to "true" change it to "false". Worked for me.
This is really good stuff. Thought I'd share a wee gotcha. If you dowhload the script using the obvious wget from the esxi commandline it turns up with ^M at the end of each line and the shell script falls over. Took me a while to work out what was going on.
Mike
just changed the script a bit, thought i might share back
vmdks_count=`grep ^scsi "$" | grep fileName | awk -F "\"" '{print $2}' | wc -l` vmdks=`grep ^scsi "$" | grep fileName | awk -F "\"" '{print $2}'`
what changed : I changed the grep scsi to ^scsi
why : I had a floppy mounted vmscsi*.flp that contains the drivers to install xp. This blocked me from doing linked clones even after i detached the floppy, because it was still in the config. Grepping on ^scsi makes more sense because this is the pattern you are looking for
macfile() {
#grab mac addresses of newly created VMs (file to populate dhcp static config)
for i in `cat ${LC_CREATED_VMS}`;
do
TMP_LIST=$/vm_list.$$
VM_P=`echo ${i##*/}`
VM_NAME=`echo ${VM_P%.vmx*}`
VM_MAC=`grep ethernet0.generatedAddress "$" | awk '{print $3}' | sed 's/\"//g' | head -1 | sed 's/://g'` while [ "$" == "" ] do sleep 1 VM_MAC=`grep ethernet0.generatedAddress "$" | awk '{print $3}' | sed 's/\"//g' | head -1 | sed 's/://g'`
echo $(grep ethernet $i)
done
echo "$ $" >> $ done LCS_OUTPUT="lcs_created_on-`date +%F-%H%M%S`" echo -e "Linked clones VM MAC addresses stored at:" cat $ | sed 's/[[:digit:]]/ &/1' | sort -k2n | sed 's/ //1' > "$" echo -e "\t$"
fi
}
#mac generation is done after the vm powers up. if you want to generate this file you will need to power on the vms first
#macfile()
what changed : Made a function to generate the file with static macs
why : Here in my version the mac addresses where only generated after the machine was powered on. This meant that the script kept looking and waiting for the mac address to be written to the vmx file. Since I don't need these mac addresses i just disabled it.
Hope it helps
Hi William. Do you plan on updating this script to support ESXi 4?
Thanks
The script actually should work but theres a section of the code that checks for the vmware -v version, you should be able to modify that and get it to work. I haven't had the time to update the older linked clones script but I know for a fact that ghetto-esx-linked-clones.sh does work and just needs the version modified but I've never tested the ESXi version.
If you're using a licensed version of ESX(i) 4.0, here is an updated version using the vSphere SDK for Perl to create the Linked Clones: http://engineering.ucsb.edu/~duonglt/vmware/vGhettoLinkedClone.html
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Thanks for your reply William! Yes, I've seen your vGhettoLinkedClone script for ESXi 4, but as I've asked in another post it look like I need vCenter to use this script. Is this correct or can I use it without vCenter?
Thanks again.
As stated per the requirements, you need vCenter.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hi,
Sorry for the stupid comment, but though i've been playin with VMware for quite a while, I only started using ESXi recently. I have several hosts running ESXi 4.0 U1 and was wondering where / how should I run this script. From what I could figure out, I need to run them on the host itself? If so, does anyone has a pointer to how to get access to the console (especially remotely). I found some links in google (using busybox and editing inetd.conf) but was just curious if this was the recommended way...
The scripts do look amazing though. I don't have vCenter, so im especially interested in the linked clones.
thanks in advance!
Diego
Thanks for this script !
With this new version, no need for vCenter : Great !
You may add in your script that we have to boot the VMs
to grab the new mac adresses. Otherwise we must wait for long !
@ garci66 :
Use of this script and access to the console is unsupported by VMware.
But it's work fine !
You can use http://vm-help.com/esx40i/ESXi_enable_SSH.php for example.
This is on ESXi 4.1.0 Build 260247
Sorry for the question, but when I try to power on a linked clone to get a MAC address, I'm getting an error stating that ESXi "cannot find the virtual disk" of my original machine. "Verify the path is valid and try again."
I've checked the VMX file of both the original and the linked clone, and both are pointing at the same location. I used the machine readable ID of the volume when running the script, rather than the human readable version.
Kind regards,
Eric
Hi Eric,
not sure what's the issue in your case, but the following was my issue:
make sure the disk name and VM names have no spaces on them, as well as the path to the disk and vmx and vmdk file.
also, you might want to look at the vmdk file (the small text file, not the binary) of the cloned machine.
The cloned machine's vmx file should point to a new vmdk. It can't point to the original's vmdk. The new vmdk should then have a reference to the original vmdk... and should mention some infor on the cloning.
Hi there,
I made sure there are no spaces in the machine/disk names, or in the paths, and re-ran the script. For some reason there does not appear to be a vmdk file for the linked clone.
The clone's vmx files shows this, which theoretically should be pointing at the master image's vmdk:
...
ide0:0.present = "TRUE"
ide0:0.fileName = "vm-elite-ws.vmdk"
...
scsi0:0.fileName = "/vmfs/volumes/4e39edee-86980f8d-3c0a-60eb699ba059/vm-Elite-WS/"
I've tried specifying the full path and file name in both ide.0:0.fileName and scsi0:0.fileName, but no joy.
Interestingly when I look at the settings of the linked clone in VSphere Client, the linked clone's hard drive shows: [datastore2] vm-elite-ws-2/vm-elite-ws.vmdk. One of the location settings is not being read, maybe?
The linked clone I'm trying to use is called "vm-elite-ws-2" (just as an FYI).
Regards,
Eric
Oh.. .. if its a VM with IDE disks, the original script won't work. It was meant for SCSI vmdks.
this is my version which I modified for IDE disks.
/vmfs/volumes/4e374605-74772abb/scripts # cat ghetto-esxi-linked-clones-ide.sh #!/bin/ash # Author: william2003[at]gmail[dot]com # duonglt[at]engr[dot]ucsb[dot]edu # Date: 09/30/2008 # # Custom Shell script to clone Virtual Machines for Labs at UCSB ResNet for ESXi, script will take number of agruments based on a golden image along with # designated virtual machine lab name and a range of VMs to be created. ####################################################################################################################################################### ESXI_VMWARE_VIM_CMD=/bin/vim-cmd #DEVEL_MODE=1 printUCSB() { echo "######################################################" echo "#" echo "# UCSB ResNet Linked Clones Tool for ESXi" echo "# Author: william2003[at]gmail[dot]com" echo -e "# \t duonglt[at]engr[dot]ucsb[dot]edu" echo "# Created: 09/30/2008" echo "#" echo "######################################################" } validateUserInput() { #sanity check to make sure you're executing on an ESX 3.x host if [ ! -f ${ESXI_VMWARE_VIM_CMD} ]; then echo "This script is meant to be executed on VMware ESXi, please try again ...." exit 1 fi if [ "${DEVEL_MODE}" -eq 1 ]; then echo "ESX Version valid (3.x+)"; fi if ! echo ${GOLDEN_VM} | egrep -i '[0-9A-Za-z]+.vmx$' > /dev/null && [[ ! -f "${GOLDEN_VM}" ]]; then echo "Error: Golden VM Input is not valid" exit 1 fi if [ "${DEVEL_MODE}" -eq 1 ]; then echo -e "\n############# SANITY CHECK START #############\n\nGolden VM .vmx file exists"; fi #sanity check to verify Golden VM is offline before duplicating ${ESXI_VMWARE_VIM_CMD} vmsvc/get.runtime ${GOLDEN_VM_VMID} | grep -i "powerState" | grep -i "poweredOff" > /dev/null 2>&1 if [ ! $? -eq 0 ]; then echo "Master VM status is currently online, not registered or does not exist, please try again..." exit 1 fi if [ "${DEVEL_MODE}" -eq 1 ]; then echo "Golden VM is offline"; fi local mastervm_dir=$(dirname "${GOLDEN_VM}") if ls "${mastervm_dir}" | grep -iE '(delta|-rdm.vmdk|-rdmp.vmdk)' > /dev/null 2>&1; then echo "Master VM contains either a Snapshot or Raw Device Mapping, please ensure those are gone and please try again..." exit 1 fi if [ "${DEVEL_MODE}" -eq 1 ]; then echo "Snapshots and RDMs were not found"; fi if ! grep -i "ethernet0.present = \"true\"" "${GOLDEN_VM}" > /dev/null 2>&1; then echo "Master VM does not contain valid eth0 vnic, script requires eth0 to be present and valid, please try again..." exit 1 fi if [ "${DEVEL_MODE}" -eq 1 ]; then echo "eth0 found and is valid"; fi vmdks_count=`grep -i ide0 "${GOLDEN_VM}" | grep -i fileName | awk -F "\"" '{print $2}' | wc -l` vmdks=`grep -i ide0 "${GOLDEN_VM}" | grep -i fileName | awk -F "\"" '{print $2}'` if [ "${vmdks_count}" -gt 1 ]; then echo "Found more than 1 VMDK associated with the Master VM, script only supports a single VMDK, please unattach the others and try again..."; exit 1; fi if [ "${DEVEL_MODE}" -eq 1 ]; then echo "Single VMDK disk found"; fi if ! echo ${START_COUNT} | egrep '^[0-9]+$' > /dev/null; then echo "Error: START value is not valid" exit 1 fi if [ "${DEVEL_MODE}" -eq 1 ]; then echo "START parameter is valid"; fi if ! echo ${END_COUNT} | egrep '^[0-9]+$' > /dev/null; then echo "Error: END value is not valid" exit 1 fi if [ "${DEVEL_MODE}" -eq 1 ]; then echo "END parameter is valid"; fi #sanity check to verify your range is positive if [ "${START_COUNT}" -gt "${END_COUNT}" ]; then echo "Your Start Count can not be greater or equal to your End Count, please try again..." exit 1 fi if [ "${DEVEL_MODE}" -eq 1 ]; then echo "START and END range is valid"; fi #end of sanity check if [ "${DEVEL_MODE}" -eq 1 ]; then echo -e "\n########### SANITY CHECK COMPLETE ############";exit; fi } #sanity check on the # of args if [ $# != 4 ]; then printUCSB echo -e "\nUsage: `basename $0` [FULL_PATH_TO_MASTER_VMX_FILE] [VM_NAME] [START_#] [END_#]" echo -e "\ti.e." echo -e "\t\t$0 /vmfs/volumes/4857f047-4e4ec6bf-a8b8-001b78361a3c/LabMaster/LabMaster.vmx LabClient- 1 200" echo -e "\tOutput:" echo -e "\t\tLabClient-{1-200}" exit 1 fi #DO NOT TOUCH INTERNAL VARIABLES #set variables GOLDEN_VM=$1 VM_NAMING_CONVENTION=$2 START_COUNT=$3 END_COUNT=$4 GOLDEN_VM_PATH=`echo ${GOLDEN_VM%%.vmx*}` GOLDEN_VM_NAME=`grep -i "displayName" ${GOLDEN_VM} | awk '{print $3}' | sed 's/"//g'` GOLDEN_VM_VMID=`${ESXI_VMWARE_VIM_CMD} vmsvc/getallvms | grep -i ${GOLDEN_VM_NAME} | awk '{print $1}'` STORAGE_PATH=`echo ${GOLDEN_VM%/*/*}` validateUserInput #print out user configuration - requires user input to verify the configs before duplication while true; do echo -e "Requested parameters:" echo -e "\tMaster Virtual Machine Image: $GOLDEN_VM" echo -e "\tLinked Clones output: $VM_NAMING_CONVENTION{$START_COUNT-$END_COUNT}" echo echo "Would you like to continue with these configuration y/n?" read userConfirm case $userConfirm in yes|YES|y|Y) echo "Cloning will proceed for $VM_NAMING_CONVENTION{$START_COUNT-$END_COUNT}" echo break;; no|NO|n|N) echo "Requested parameters canceled, application exiting" exit;; esac done #start duplication COUNT=$START_COUNT MAX=$END_COUNT START_TIME=`date` S_TIME=`date +%s` TOTAL_VM_CREATE=$(( ${END_COUNT} - ${START_COUNT} + 1 )) LC_EXECUTION_DIR=/tmp/esxi_linked_clones_run.$$ mkdir -p ${LC_EXECUTION_DIR} LC_CREATED_VMS=${LC_EXECUTION_DIR}/newly_created_vms.$$ touch ${LC_CREATED_VMS} WATCH_FILE=${LC_CREATED_VMS} EXPECTED_LINES=${TOTAL_VM_CREATE} while sleep 5; do REAL_LINES=$(wc -l < "${WATCH_FILE}") REAL_LINES=`echo ${REAL_LINES} | sed 's/^[ \t]*//;s/[ \t]*$//'` P_RATIO=$(( (${REAL_LINES} * 100 ) / ${EXPECTED_LINES} )) P_RATIO=${P_RATIO%%.*} clear echo -en "\r${P_RATIO}% Complete! - Linked Clones Created: ${REAL_LINES}/${EXPECTED_LINES}" if [ ${REAL_LINES} -ge ${EXPECTED_LINES} ]; then break fi done & while [ "$COUNT" -le "$MAX" ]; do FINAL_VM_NAME="${VM_NAMING_CONVENTION}${COUNT}" mkdir -p ${STORAGE_PATH}/$FINAL_VM_NAME cp ${GOLDEN_VM_PATH}.vmx ${STORAGE_PATH}/$FINAL_VM_NAME/$FINAL_VM_NAME.vmx VMDK_PATH=`grep -i ide0:0.fileName ${GOLDEN_VM_PATH}.vmx | awk '{print $3}' | sed 's/"//g'` sed -i 's/displayName = "'${GOLDEN_VM_NAME}'"/displayName = "'${FINAL_VM_NAME}'"/' ${STORAGE_PATH}/$FINAL_VM_NAME/$FINAL_VM_NAME.vmx sed -i '/ide0:0.fileName/d' ${STORAGE_PATH}/$FINAL_VM_NAME/$FINAL_VM_NAME.vmx echo "ide0:0.fileName = \"${STORAGE_PATH}/${GOLDEN_VM_NAME}/${VMDK_PATH}\"" >> ${STORAGE_PATH}/$FINAL_VM_NAME/$FINAL_VM_NAME.vmx sed -i 's/nvram = "'${GOLDEN_VM_NAME}.nvram'"/nvram = "'${FINAL_VM_NAME}.nvram'"/' ${STORAGE_PATH}/$FINAL_VM_NAME/$FINAL_VM_NAME.vmx sed -i 's/extendedConfigFile = "'${GOLDEN_VM_NAME}.vmxf'"/extendedConfigFile = "'${FINAL_VM_NAME}.vmxf'"/' ${STORAGE_PATH}/$FINAL_VM_NAME/$FINAL_VM_NAME.vmx sed -i '/ethernet0.generatedAddress/d' ${STORAGE_PATH}/$FINAL_VM_NAME/$FINAL_VM_NAME.vmx > /dev/null 2>&1 sed -i '/ethernet0.addressType/d' ${STORAGE_PATH}/$FINAL_VM_NAME/$FINAL_VM_NAME.vmx > /dev/null 2>&1 sed -i '/uuid.location/d' ${STORAGE_PATH}/$FINAL_VM_NAME/$FINAL_VM_NAME.vmx > /dev/null 2>&1 sed -i '/uuid.bios/d' ${STORAGE_PATH}/$FINAL_VM_NAME/$FINAL_VM_NAME.vmx > /dev/null 2>&1 sed -i '/sched.swap.derivedName/d' ${STORAGE_PATH}/$FINAL_VM_NAME/$FINAL_VM_NAME.vmx > /dev/null 2>&1 ${ESXI_VMWARE_VIM_CMD} solo/registervm ${STORAGE_PATH}/$FINAL_VM_NAME/$FINAL_VM_NAME.vmx > /dev/null 2>&1 FINAL_VM_VMID=`${ESXI_VMWARE_VIM_CMD} vmsvc/getallvms | grep -i ${FINAL_VM_NAME} | awk '{print $1}'` ${ESXI_VMWARE_VIM_CMD} vmsvc/snapshot.create ${FINAL_VM_VMID} Cloned ${FINAL_VM_NAME}_Cloned_from_${GOLDEN_VM_NAME} > /dev/null 2>&1 #output to file to later use echo "${STORAGE_PATH}/$FINAL_VM_NAME/$FINAL_VM_NAME.vmx" >> "${LC_CREATED_VMS}" COUNT=$(( $COUNT + 1 )) done sleep 10 echo -e "\n\nWaiting for Virtual Machine(s) to obtain MAC addresses...\n" END_TIME=`date` E_TIME=`date +%s` #grab mac addresses of newly created VMs (file to populate dhcp static config) if [ -f ${LC_CREATED_VMS} ]; then for i in `cat ${LC_CREATED_VMS}`; do TMP_LIST=${LC_EXECUTION_DIR}/vm_list.$$ VM_P=`echo ${i##*/}` VM_NAME=`echo ${VM_P%.vmx*}` VM_MAC=`grep -i ethernet0.generatedAddress "${i}" | awk '{print $3}' | sed 's/\"//g' | head -1 | sed 's/://g'` while [ "${VM_MAC}" == "" ] do sleep 1 VM_MAC=`grep -i ethernet0.generatedAddress "${i}" | awk '{print $3}' | sed 's/\"//g' | head -1 | sed 's/://g'` done echo "${VM_NAME} ${VM_MAC}" >> ${TMP_LIST} done LCS_OUTPUT="lcs_created_on-`date +%F-%H%M%S`" echo -e "Linked clones VM MAC addresses stored at:" cat ${TMP_LIST} | sed 's/[[:digit:]]/ &/1' | sort -k2n | sed 's/ //1' > "${LCS_OUTPUT}" echo -e "\t${LCS_OUTPUT}" fi echo echo "Start time: ${START_TIME}" echo "End time: ${END_TIME}" DURATION=`echo $((E_TIME - S_TIME))` #calculate overall completion time if [ ${DURATION} -le 60 ]; then echo "Duration : ${DURATION} Seconds" else echo "Duration : `awk 'BEGIN{ printf "%.2f\n", '${DURATION}'/60}'` Minutes" fi echo rm -rf ${LC_EXECUTION_DIR}
Sorry for the long post.. but that code should fix it. I don't remember exactly which changes I did to them though...
not sure what happens if you have both IDE and SCSI hard disks...
Actually, here is the "diff" of the original script and the IDE version:
Lamw - awesome bunch of scripts! used the .sh and it worked really well.
Just one question - does this perl script do linked clones for VMs with multiple disks? Can you get the shell script version to do linked clones for VMs with multiple disks?