LF Script to clone vm's based on a schedule to a set of DR datastores.

[Barren]

Hi All

The short story is that due to ongoing SAN issues (and solution value engineering) I am faced with an issue where my VM storage is currently considered 'at risk'. To attempt to alleviate this we obviously carry out backups but have found our current solution is restricted by large volumes and other issues which means a very complex solution is in place should we need to DR - and it is less than satisfactory.

As a 'quick fix' I've been tasked with cloning the environment on a regular schedule to another SAN which has been configured to be a replica of the live system. I've manually completed a clone of every machine to prove the concept but this took days and isn't really sustainable.

I've looked at some of the clone scripts within the community but my knowledge is very limited when it comes to the more complex operations required and the associated scripting therein.

My first question I suppose is 'Is there a way to 'dump' the powercli logic of the manual commands from which I can create a script? (presumably not as they're not directly linked .. )

My second question therefore is a very broad 'Can anyone suggest any useful tools/utilities/scripts which allow cloning on schedules please?'

Is this the sort of thing that VMware support could be engaged in or is this 3rd party only?

Should I just keep trying to extend existing scripts (i'm clearly a noob at this and am being pressured to deliver a solution in a time frame my current skill set doesn't support - i'm sure you've all been there ).

I'm ok with the simple clone vma to vmb but have struggled with the clone vma volume 1 on datastore 1 to vmb volume 1b on datastore 1b, vma volume 2 to vmb volume 2b, 3 -3b and similar for some of the more complex machines, coupled with deleting targets first and/or archiving/compressing so we don't have a clone failure at the worst time etc.

Any help would be appreciated at this time thanks

[LucD]

You're probably already aware that you could use the New-VM cmdlet with the VM parameter to clone an existing VM ?

To define the target datastore of the clone you use the Datastore parameter.

Do you have any more details on the actual cloning process ?

Is all from datastore1 to be cloned to datastore2 ?

Are there multiple source datastores involved ?

If you could give a bit more of the practical details, we could try to help you develop a script for your task.

[Barren]

Hi LucD

I was trying to to get too involved and fill the page with text in the first instance

I've looked into the New-VM cmdlet and it worked well for very simlpe vm's but i quickly realised i was out of my depth with multiple datastores.

The general layout is one that has grown over many years with multiple storage extensions. This basically means I have around 30 source guests based across 15 datastores, with some having volumes on multiple datatstores.

So VM1 may have OS volume on Datastore1\VM1 with file store volume on Datastore 2\VM1

VM2 may have OS on Datastore1\VM2 but then have other storage on a number of other datastores so Datastore2\VM2, Datastore3\VM2. Within that it may have multiple vmdk's VM2_1, VM2_2 etc . Within the UI for example using the clone operation I have to go into the 'Advanced' section of the Datastore and specify the exact clone replica for each Hard disk listed for the virtual machine.The destination target 'should' be the same nomenclature but with a 'CLONE ' tag in front of it.

Presumably there is some way to dump all of this config to a file to pipe into a later operation?

Maybe I need to break this down into clear steps for each vm and see how far i can get ..

I will continue to rtfm

thanks

[LucD]

Breaking a larger script down into smaller managable parts is a good plan.

Yes, you can extract the VM information and dump it to a CSV file for example.

To get you started, and to show how easy it is, the following will dump the VM and datastore info to a CSV

Get-VM | Get-HardDisk | 
Select @{N="VM";E={$_.Parent.Name}},Name,
@{N="Datastore";E={$_.Filename.Split(']')[0].TrimStart('[')}} |
Export-Csv "C:\report.csv" -NoTypeInformation -UseCulture

[aerodevil]

Sorry for not seeing this earlier.  I believe this funciton I wrote will do almost exactly what you need.

http://www.vtesseract.com/post/16447807254/clone-list-powercli-function

http://communities.vmware.com/docs/DOC-18155

Essentially you can generate whatever list you want and call Clone-List.  You can specify your target datastore and it will either cold or live clone the vm and append the name with prefix Clone-.  The function is set up so that you can change the prefix if you'd like.  I've used it for both live and cold cloning and it runs like a champ.  You will also want to verify if you want it to run -RunAsync or not depending on your environment.

Note:  When doing cold clones I found that the first 12 run w/out issue but the 13th and later may timeout.  This also depends on your environment I believe.

Possible usage if you wanted to evacuate a single datastore.

$TargetDS = Get-Datastore "Target"
$vms = Get-Datastore "SourceDS" | Get-VM
$vms | Clone-List -Datastore $TargetDS

Unless I am mistaken (and i haven't tested) initiating the clone will clone all disks over to the new target datastore.  One thought is to initiate the clone and that migrate the individual disks to other datastores as required.

I apologize if I missed a requirement here.

[Barren]

Thanks LucD.

I think I was attempting to get to a solution too quickly and needed to step back and work at some component parts. The information you have given clearly provides and excellent basis for importing relevant detail into the relevant parameters. I will continue to break it down into more workable chunks and see what i'm missing

Thanks again.

[Barren]

Thanks Aerodevil.

I had indeed come across this script from yourself but stalled when i started thinking about how many datastore targets each of the more complex machines had and indeed how i was going to incorporate that into something larger. Additionally I have some machines with volumes that won't hot clone and some that will (I should probably just have 2 scripts rather than trying to do too much )

I also think i was trying to overcomplicate the script (for me at least) by trying to find/modify/write something that did everything as i have found an example that did lots of what i wanted but i'll admit it didn't make a huge amount of sense when trying to apply it to my environment and found myself going round in circles.

Will try this out with the simple vm's and see where i get stuck

Thanks again

[Barren]

So going through this i think what i'm trying to achieve is something logically like the following:

1,Get all the relevant information for each VM i wish to clone, including all datastore info., and write this to a file.

2, Pass the above file into a clone script which

a, checks if the clone already exists and either deletes the target or continues

b, modifies the target name to be clone-<existing name> and the target datastore(s) to be "CLONE <existing name>"

c, processes the cloning operation

d, logs the operations to a file

3, create a schedule for these operations

I think i'd confused myself by also 'wanting'

1, email notification (although i've found seevral scripts which are easily modified to achieve this)

2, to limit the maximum number of clone operations (again have found something possibly modifiable that checks for running taks and creates a limit)

3, If the target already existed to be able to archive it in some way so that i wasn't left without a clone at any point (this was the point i went {wibble})

Does this seem a sensible approach? am i missing something glaringly obvious please?

(I do so love in at the deepend due to critical failures projects lol )

Thanks to all for all the help so far i really is very much appreciated.

[aerodevil]

I totally understand.  As blasphemous as this may start to sound, have you considered looking into vCenter Orchestrator for developing your workflow?  I believe you may find it able to do some of your workflow easily and can also help incorporate some of the PowerCLI stuff you may still need.

Definitely take it in steps. I have some scripts that can help with some of these items.

My initial thought was that the clone-list function could allow you to quickly get the clones and then you could use other scripts to "organize" everything later.  This could possibly make it easier for you, especially if you have a potential failure scenario.  The other nice thing about the function is that you could only send a small number at a time, thus making it easier to "thread".  It's all about the list of VMs you select and pass to it.  In fact on some of my larger clone jobs I've broken it up into a couple of lists and run them that way.

[Barren]

Thanks again.

Indeed I had looked at Orchestrator but tbh wasn't sure of the cost so delayed somewhat <EDIT Just seen it is part of vCenter so will investigate that option too .> I have been approved a minimal budget to achieve a solution due to time constraints and so obviously looked to the community for help and advice accordingly - though clearly I was 'encouraged' to provide an in-house solution. At a personal level I am interested in achieving the goals stated its just not my (or indeed anyones atm) day job hence my in at the deep end /noob comments

I just need to pull it all together and fill in my vast knowledge gaps.

Thanks again for the support it does make me feel like it can be achieved (and everyone else makes it sound so easy lol)

[aerodevil]

Well rest assured about Orchestrator and your budget.  It's FREE with vCenter server!!  <Edit:  Saw your edit but had to post that line!>

It can definitely be achieved though may take some time.

[LucD]

Your scenario looks realistic.

Email noticification (Send-EmailMessage) and limiting the number of parallel clone tasks is not too difficult.

For the archiving part you would need to know how to tackle the archiving.

Is the VM copied to another datastore (Move-VM) ?

Or should it be stored outside of the datastores (a Converter job) ?