VMware Cloud Community
dbutch1976
Hot Shot
Hot Shot
Jump to solution

Deploying, customizing, and modifying VMs from a csv file

Hello,

With a lot of help from Eco91 I have produced the script below.  It deploys a series of VMs at the same time, waits for them to boot and perfom an OS Customization, then changes basic networking from within the OS and also changes the Network Description:

$esxName = "esxqac4s01.MYDOMAIN.local"
$template = "TEST_W2K8R2ENTSP1"
$datastore = "NFS_VM_Storage01"
$newVmList = @(
    @{"Name" = "TESTSVR1"; "IP" = "10.244.186.136"; "Netmask" = "255.255.255.0"; "Gateway" = "10.244.186.1"; "DNS" = @("10.244.37.25", "10.244.37.26"); "NetworkName" = "VLAN186_QA"; }, 
    @{"Name" = "TESTSRV2"; "IP" = "10.244.186.137"; "Netmask" = "255.255.255.0"; "Gateway" = "10.244.186.1"; "DNS" = @("10.244.37.25", "10.244.37.26"); "NetworkName" = "VLAN186_QA"; }, 
    @{"Name" = "TESTSRV3"; "IP" = "10.244.186.138"; "Netmask" = "255.255.255.0"; "Gateway" = "10.244.186.1"; "DNS" = @("10.244.37.25", "10.244.37.26"); "NetworkName" = "VLAN186_QA"; }, 
    @{"Name" = "TESTSRV4"; "IP" = "10.244.186.139"; "Netmask" = "255.255.255.0"; "Gateway" = "10.244.186.1"; "DNS" = @("10.244.37.25", "10.244.37.26"); "NetworkName" = "VLAN186_QA"; }
)
$custSpec = "TEST_W2K8R2ENTSP1"
$location = "_Tobedeleted"
$taskTab = @{}
Connect-VIServer -Server vvcqa01yyz.MYDOMAIN.local -User "MYDOMAIN\MYACCOUNT" -Password MYPASSWORD
# Create all the VMs specified in $newVmList
foreach($VM in $newVmList) {
     $taskTab[(New-VM -Name $VM.Name -VMHost (Get-VMHost -Name $esxName) -Template $template -Datastore $datastore -OSCustomizationSpec $custSpec -Location $location -RunAsync).Id]=$VM.Name
}

# Start each VM that is completed
$runningTasks = $taskTab.Count
while($runningTasks -gt 0){
Get-Task | % {
  if($taskTab.ContainsKey($_.Id) -and $_.State -eq "Success"){
   Get-VM $taskTab[$_.Id] | Start-VM
   $taskTab.Remove($_.Id)
   $runningTasks--
  }
  elseif($taskTab.ContainsKey($_.Id) -and $_.State -eq "Error"){
   $taskTab.Remove($_.Id)
   $runningTasks--
  }
}
Start-Sleep -Seconds 5
}


# START HERE 
# Wait for OS Customization 
Start-Sleep -Seconds 300
# Customize network
foreach($VM in $newVmList) { 
    Get-NetworkAdapter -VM $VM.Name | Set-NetworkAdapter -NetworkName $VM.NetworkName -Confirm:$false 
    Get-VM -Name $VM.Name | Get-VMGuestNetworkInterface -Guestuser "Administrator" -GuestPassword "R!m2009" | ? { $_.name -eq "Local Area Connection 4" } | Set-VMGuestNetworkInterface -Guestuser "Administrator" -GuestPassword "R!m2009" -IPPolicy static -IP $VM.IP -Netmask $VM.Netmask -Gateway $VM.Gateway -DNS $VM.DNS 
}

I would now like to replace the $VMlist variable and instead put th eVM information from a .csv file.  Any suggestions?

1 Solution

Accepted Solutions
EKardinal
Enthusiast
Enthusiast
Jump to solution

Hi Smiley Happy

assuming this is your csv file

    "name","ip","netmask","gateway","dns","networkName"
    "TESTSRV1","10.244.186.136","255.255.255.0","10.244.186.1","10.244.37.25;10.244.37.26","VLAN186_QA"

you have one problem. You can't save any arrays into a csv and make powershell recognize them natively.

Instead, use a custom delimiter (not the same as csv uses!) and split them - at import - into an array. Look at the semicolon between the two dns servers.

    $newVmList = Import-Csv test.csv | Select-Object -Property "name", "ip", "netmask", "gateway", @{Name = "dns"; Expression = {$_.dns -split ";"} }, "networkName"

The resulting object looks like this

    name       : TESTSRV1
    ip         : 10.244.186.136
    netmask    : 255.255.255.0
    gateway    : 10.244.186.1
    dns        : {10.244.37.25, 10.244.37.26} <-- Array
    networkName: VLAN186_QA

Regards

Emanuel

View solution in original post

Reply
0 Kudos
26 Replies
EKardinal
Enthusiast
Enthusiast
Jump to solution

Hi Smiley Happy

assuming this is your csv file

    "name","ip","netmask","gateway","dns","networkName"
    "TESTSRV1","10.244.186.136","255.255.255.0","10.244.186.1","10.244.37.25;10.244.37.26","VLAN186_QA"

you have one problem. You can't save any arrays into a csv and make powershell recognize them natively.

Instead, use a custom delimiter (not the same as csv uses!) and split them - at import - into an array. Look at the semicolon between the two dns servers.

    $newVmList = Import-Csv test.csv | Select-Object -Property "name", "ip", "netmask", "gateway", @{Name = "dns"; Expression = {$_.dns -split ";"} }, "networkName"

The resulting object looks like this

    name       : TESTSRV1
    ip         : 10.244.186.136
    netmask    : 255.255.255.0
    gateway    : 10.244.186.1
    dns        : {10.244.37.25, 10.244.37.26} <-- Array
    networkName: VLAN186_QA

Regards

Emanuel

Reply
0 Kudos
dbutch1976
Hot Shot
Hot Shot
Jump to solution


Thanks again for your fast, accurate reply!

Reply
0 Kudos
markdjones82
Expert
Expert
Jump to solution

Just wondering why you decided to do the network setting after deploying the VM, any specific reason? If not,  you can actually pass those paramters to the OScustomization spec so you wouldn't need to wait for the VM's to power on.  Might be faster your deploys

    Get-OSCustomizationSpec $custspec | Get-OSCustomizationNicMapping | Set-OSCustomizationNicMapping -IpMode UseStaticIp -IpAddress $ipaddr -SubnetMask $subnet -DefaultGateway $gateway -Dns $pdns,$sdns

http://www.twitter.com/markdjones82 | http://nutzandbolts.wordpress.com
Reply
0 Kudos
dbutch1976
Hot Shot
Hot Shot
Jump to solution

Hi Mark,

We have a large number of VLANs here, so in order to get around creating several different OScustomization's I have a single OScustomization which uses DHCP.  This allows the server to boot, apply the OScustomization (join the domain etc), then once this is complete the second part of the script changes the VLAN and sets a static IP address which networking configs (gateway) which are specific to the final VLAN.

Thanks,

Duncan.

Reply
0 Kudos
markdjones82
Expert
Expert
Jump to solution

Ah, we have same thing lots of VLAN's but we just set the network label before powering on so it is on the proper vlan while it customizes and then joins the domain.  It does it all on the fly  So, when your VM's boot they are on a DHCP Vlan and then you change the ip later?

EDIT: the set OS customization spec will modify the vlan information in the cust spec. Here is how I do it just for reference if you are interested. Basically doing the same thing, just different methods Smiley Happy

$vcenter = Read-Host "Enter Vcenter Name"
$csvimport = Read-Host "Enter CSV filename (fullname with extension)"
$username = read-host "Enter your domain admin username for customization"
$pass = Read-Host -AsSecureString "Enter your password"
$adminpass = Read-Host -AsSecureString "Enter local admin password"


#convert back to string
$pass = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($pass)
$pass = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($pass)
$adminpass = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($adminpass)
$adminpass = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($adminpass)


Connect-VIServer $vcenter

$vmlist = Import-CSV $csvimport

#Define Tasks Array
$tasks = @()

foreach ($item in $vmlist) {

   
   
$basevm = $item.template
   
$datastore = $item.datastore
   
$cluster = $item.cluster
   
$custspec = $item.custspec
   
$vmname = $item.vmname
   
$ipaddr = $item.ipaddress
   
$subnet = $item.subnet
   
$gateway = $item.gateway
   
$pdns = $item.pdns
   
$sdns = $item.sdns
   
$vlan = $item.vlan
   
$hd = [int]$item.ddrive * 1024 * 1024
   
$vmname = $item.vmname
   
$cpu = $item.cpu
   
$mem = [int]$item.mem * 1024
   
$vds= $item.vsm

 
#set customization spec
  New-OSCustomizationSpec -OSCustomizationSpec $custspec -Name $vmname -type NonPersistent | Out-Null
 
Set-OSCustomizationSpec -OSCustomizationSpec $vmname -AdminPassword $adminpass -DomainUsername $username -DomainPassword $pass -Confirm:$false | Out-Null
 
Get-OSCustomizationSpec $vmname | Get-OSCustomizationNicMapping | Set-OSCustomizationNicMapping -IpMode UseStaticIp -IpAddress $ipaddr -SubnetMask $subnet -DefaultGateway $gateway -Dns $pdns,$sdns | Out-Null
 
$OsCustomization = Get-OSCustomizationSpec $vmname
     
 
#Clone the templates and add to tasks list.  It will spin all up at the same time
  $tasks += (New-VM -Name $vmname -OSCustomizationSpec $OScustomization -template $basevm -Datastore $datastore -ResourcePool $cluster -RunAsync)

}

Write-Host "Waiting for Copies to Finish"

#Wait for all taksks in tasks list before proceeding
Wait-Task -task $tasks

#Set network Label and add harddrives by looping through list and doing one at a time
foreach ($item in $vmlist) {
   
$vmname = $item.vmname
   
$vlan = $item.vlan
   
$hd = [int]$item.ddrive * 1024 * 1024
   
$cpu = $item.cpu
   
$mem = [int]$item.mem * 1024
   
$vds= $item.vsm

       
   
#Set network label
    $vdportgroup = Get-VDPortgroup -Name $vlan -VDSwitch $vds
   
Get-VM -Name $vmname | Get-NetworkAdapter -Name "Network adapter 1" | Set-NetworkAdapter -Portgroup $vdportgroup -Confirm:$false
   
   
#set vm
    Set-VM -VM $vmname -NumCpu $cpu -MemoryMB $mem -Confirm:$false
   
   
#Create D drive windows
    if ($hd){
   
New-HardDisk -VM $vmname -CapacityKB $hd -StorageFormat Thin -Confirm:$false
    }
       
}

#Loop through list and power all VM's on simultaneously
foreach ($item in $vmlist){
$vmname = $item.vmname
#Start VM
    Start-VM -VM $vmname -Confirm:$false -RunAsync
}

Disconnect-VIServer $vcenter -confirm:$false
http://www.twitter.com/markdjones82 | http://nutzandbolts.wordpress.com
Mattyfonz
Contributor
Contributor
Jump to solution

Hi All,

Great work on the script here, it's exactly what I'm looking for, I just need to tweak it a little for my purposes Smiley Happy

I do have a couple questions though, When using the above script (I am attempting to combinine dbutch1976 with Eco91's work so I can use a csv file with the variables):

If I only specify the vmname, IP, subnet, gateway, networkname and location (see below re: location)  in the csv, will the DNS ip's get picked up from the OSCustomization file I already have created? I suppose another way of putting it is, will any settings I don't list in the csv be covered by the customization file and template (admin password, licensing etc)?

Also, markdjones82 mentioned you can pass the network details into the customization file via 

Get-OSCustomizationSpec $custspec | Get-OSCustomizationNicMapping | Set-OSCustomizationNicMapping -IpMode UseStaticIp -IpAddress $ipaddr -SubnetMask $subnet -DefaultGateway $gateway -Dns $pdns,$sdns

Would I put this in the newVmlist loop so

-OSCustomizationSpec (Get-OSCustomizationSpec $custspec | Get-OSCustomizationNicMapping | Set-OSCustomizationNicMapping -IpMode UseStaticIp -IpAddress $ip -SubnetMask $subnet -DefaultGateway $gateway)

And last question:

For the location, I would like to have some vms in different locations, can I do this by having a base location specified and have a sub folder location as a variable in the csv that can be combined with the base location in the newvmlist loop?

How would I combine the two variables to create the vm folder path  and will it create it if it does not exist?

Again thanks for all the hard work that's been done, Myself and I'm sure many others appreciate it!

Reply
0 Kudos
markdjones82
Expert
Expert
Jump to solution

You should be able to do what you put, but it would modify the existing customization space every time with the IP you specify.  That is why I create a new one in memory every time.  I haven't tested doing it this way but it looks good.  As far as will it keep the DNS records you put in there I am not sure, I would test a deploy and see if it keeps them.  I would think it would though.  Let me know the results i'm interested!

-OSCustomizationSpec (Get-OSCustomizationSpec $custspec | Get-OSCustomizationNicMapping | Set-OSCustomizationNicMapping -IpMode UseStaticIp -IpAddress $ip -SubnetMask $subnet -DefaultGateway $gateway)

Also, if you customized it the way i'm doing it as opposed to after powering on you wouldn't need this line in their script:

Get-VM -Name $VM.Name | Get-VMGuestNetworkInterface -Guestuser "Administrator" -GuestPassword "R!m2009" | ? { $_.name -eq "Local Area Connection 4" } | Set-VMGuestNetworkInterface -Guestuser "Administrator" -GuestPassword "R!m2009" -IPPolicy static -IP $VM.IP -Netmask $VM.Netmask -Gateway $VM.Gateway -DNS $VM.DNS

http://www.twitter.com/markdjones82 | http://nutzandbolts.wordpress.com
Reply
0 Kudos
Mattyfonz
Contributor
Contributor
Jump to solution

No luck unfortunately. This is what I have:

$ClusterName = "ESXCLUSTER01"
$template = "WIN2k8"
$datastorecluster = "DATACLUSTER01"
$newVmList = Import-Csv vmlist.csv | Select-Object -Property "name", "ip", "subnet", "gateway", "pdns", "sdns", "networkname", "vmlocation"
$custSpec = "WIN2k8"
$baselocation = "Test-01"
$taskTab = @{}
Connect-VIServer -Server VCENTER1.domain.local
# Create all the VMs specified in $newVmList
foreach($VM in $newVmList) {
     $taskTab[(New-VM -Name $VM.Name -VMHost (Get-Cluster -Name $ClusterName) -Template $template -Datastore $datastorecluster -OSCustomizationSpec (Get-OSCustomizationSpec $custspec | Get-OSCustomizationNicMapping | Set-OSCustomizationNicMapping -IpMode UseStaticIp -IpAddress $ip -SubnetMask $subnet -DefaultGateway $gateway -Dns $pdns,$sdns) -NetworkName $networkname -Location (get-folder $baselocation | new-folder $vmlocation) -RunAsync).Id]=$VM.Name
   }

# Start each VM that is completed
$runningTasks = $taskTab.Count
while($runningTasks -gt 0){
Get-Task | % {
  if($taskTab.ContainsKey($_.Id) -and $_.State -eq "Success"){
   Get-VM $taskTab[$_.Id] | Start-VM
   $taskTab.Remove($_.Id)
   $runningTasks--
  }
  elseif($taskTab.ContainsKey($_.Id) -and $_.State -eq "Error"){
   $taskTab.Remove($_.Id)
   $runningTasks--
  }
}
Start-Sleep -Seconds 5
}

It errors out in with the -OSCustomizationSpec parameter, more specifically it states that I need to specify DNS names when using the UseStaticIP flag which rules out me keeping the dns entries in the OSCustomization file. I tweaked it to include a primary and secondary dns (pdns, sdns) but I still get the error.

the -location parameter was something extra I added to create each vm in a specific subfolder under a base folder location. I'm assuming it will work..

Reply
0 Kudos
markdjones82
Expert
Expert
Jump to solution

What is the error that you are getting?  Can you post it?

http://www.twitter.com/markdjones82 | http://nutzandbolts.wordpress.com
Reply
0 Kudos
Mattyfonz
Contributor
Contributor
Jump to solution

Capture.JPG

I tried with one dns entry with no luck either.

Reply
0 Kudos
markdjones82
Expert
Expert
Jump to solution

I just realized you aren't referencing the variables correctly. You need to put $VM.  in front of those variables as that is what you are looping through and referencing that object.header

YOu need to do this:

$taskTab[(New-VM -Name $VM.Name -VMHost (Get-Cluster -Name $ClusterName) -Template $template -Datastore $datastorecluster -OSCustomizationSpec (Get-OSCustomizationSpec $custspec | Get-OSCustomizationNicMapping | Set-OSCustomizationNicMapping -IpMode UseStaticIp -IpAddress $VM.ip -SubnetMask $subnet -DefaultGateway $VM.gateway -Dns $VM.pdns,$VM.sdns) -NetworkName $VM.networkname -Location (get-folder $baselocation | new-folder $VM.vmlocation) -RunAsync).Id]=$VM.Name

http://www.twitter.com/markdjones82 | http://nutzandbolts.wordpress.com
Reply
0 Kudos
Mattyfonz
Contributor
Contributor
Jump to solution

Heh, woops Smiley Happy

That error has been rectified thanks for pointing that out, last bit is to get the folder creation working, At the moment it seems to be trying to create the sub-folder twice and bombs out when it tries to create it a second time. the error in vcenter is the folder already exists.

Capture.JPG

I did have another requirement pop up today, it seems as though some of the vms will have 2 nics. is it possible to put a statement in the get-oscustomizationnicmapping where if the nic count is more than one set the ip config for the second adapter using the variables ip2, subnet2, gateway2 etc?


Reply
0 Kudos
markdjones82
Expert
Expert
Jump to solution

I am not sure on the folder issue what happens if you take out the runasync option?  I'm just grasping at straws, maybe one of the better powercli experts can chime in.


As far as VM's having 2 NIC's you might have to ahve different customization specs for that.  I have never tried to apply a customizaton spec with 2 NICS to a single NIC VM.

I also would probably do my  os customization modification outside of the new-vm command to manipulate it and I'm not sure if you can return 2 commands in parenthesis

if ((get-vm $vm.name | Get-NetworkAdapter).Count -eq "1") {

Get-OSCustomizationSpec test |Get-OSCustomizationNicMapping | Get-OSCustomizationNicMapping | Set-OSCustomizationNicMapping -IpMode UseStaticIp -IpAddress $VM.ip -SubnetMask $subnet -DefaultGateway $VM.gateway -Dns $VM.pdns,$VM.sdns) -NetworkName $VM.networkname

}

else{

Get-OSCustomizationSpec test |Get-OSCustomizationNicMapping | Where {$_.Position -eq "1"} | Get-OSCustomizationNicMapping | Set-OSCustomizationNicMapping -IpMode UseStaticIp -IpAddress $VM.ip -SubnetMask $subnet -DefaultGateway $VM.gateway -Dns $VM.pdns,$VM.sdns) -NetworkName $VM.networkname

Get-OSCustomizationSpec test |Get-OSCustomizationNicMapping | Where {$_.Position -eq "2"} | Get-OSCustomizationNicMapping | Set-OSCustomizationNicMapping -IpMode UseStaticIp -IpAddress $VM.ip2 -SubnetMask $subnet -DefaultGateway $VM.gateway2 -Dns $VM.pdns,$VM.sdns) -NetworkName $VM.networkname2

}
http://www.twitter.com/markdjones82 | http://nutzandbolts.wordpress.com
Reply
0 Kudos
Mattyfonz
Contributor
Contributor
Jump to solution

Hmm the 2 Nics does make it quite a bit harder, I think ill break it up and have a separate template and customization file for the servers that require 2 Nics. As for the New folder issue, I still get it even if I remove the async flag. It's a strange one, as the top level folder exists but comes back as created successfully but the sub folder doesn't but comes back as already exists..

Reply
0 Kudos
markdjones82
Expert
Expert
Jump to solution

Yeah that is probably best.  So on the new folder command does it create it once and then error out later or you get the error immediately?  You are sure the folder isn't already there?  Is there a folder with the same name somewhere else in the structure?  That might cause an issue possibly.

http://www.twitter.com/markdjones82 | http://nutzandbolts.wordpress.com
Reply
0 Kudos
Mattyfonz
Contributor
Contributor
Jump to solution

Ok, I worked out why I was getting the folder error, I had two sessions connected (changing the vcenter from shortname to fqdn in the script) however I still get a folder error when it tries to create one that is already there (eg: creating multiple vms in one folder will trigger it). it will continue but it will throw an error, Can I put some form of folder exist check in there?

I've been doing some searching on the forums and changed how I use the OsCustomizationSpec, now I clone a "baseline" Customization template to a new non persistent one that is customized for the vm it is building. This will stop the original customization file being changed.

However, I now get the below error
Capture.JPG

Here is the code:

$ClusterName = "CLUSTER1"
$template = "WIN2K8"
$datastorecluster = "DATACLUSTER1"
$newVmList = Import-Csv vmlist.csv | Select-Object -Property "name", "ip", "subnet", "gateway", "pdns", "sdns", "networkname", "vmlocation"
$custSpec = "WIN2K8SPEC"
$baselocation = "Test-01"
$taskTab = @{}
Connect-VIServer -Server VCENTER01.DOMAIN.LOCAL

# Create all the VMs specified in $newVmList
foreach($VM in $newVmList) {
     $taskTab[(New-VM -Name $VM.name -ResourcePool $ClusterName -Template $template -Datastore $datastorecluster -OSCustomizationSpec (New-OSCustomizationSpec -Spec $custspec -Name $vm.name -Type NonPersistent | Get-OSCustomizationNicMapping | Set-OSCustomizationNicMapping -IpMode UseStaticIp -IpAddress $vm.ip -SubnetMask $vm.subnet -DefaultGateway $vm.gateway -Dns $vm.pdns,$vm.sdns) -NetworkName $vm.networkname -Location (get-folder $baselocation | new-folder $vm.vmlocation) -RunAsync).Id]=$VM.name
  }

# Start each VM that is completed
$runningTasks = $taskTab.Count
while($runningTasks -gt 0){
Get-Task | % {
  if($taskTab.ContainsKey($_.Id) -and $_.State -eq "Success"){
   Get-VM $taskTab[$_.Id] | Start-VM
   $taskTab.Remove($_.Id)
   $runningTasks--
  }
  elseif($taskTab.ContainsKey($_.Id) -and $_.State -eq "Error"){
   $taskTab.Remove($_.Id)
   $runningTasks--
  }
}
Start-Sleep -Seconds 5
}

Reply
0 Kudos
markdjones82
Expert
Expert
Jump to solution

If you read my deploy script that I posted on first page i also do the cloning to create a non persistent :smileysilly:

Does your customization spec have DHCP set or static IP?  I believe it has to have a dummy static IP setting to work.

You can do an IF statement to check for a folder, but you are doing so much into the one single new-vm command i dont' know if it possible to do an IF statement AND the new folder command and have it returned in the parenthesis.

I personally like to split out my code a little more as trying to do everything inside of parenthesis seems to get complicated and hard to debug.

http://www.twitter.com/markdjones82 | http://nutzandbolts.wordpress.com
Reply
0 Kudos
Mattyfonz
Contributor
Contributor
Jump to solution

So you did Smiley Happy  Should have read more carefully Smiley Wink

My base customization is set as Static for testing purposes. I was just thinking that I'm cramming too much into one line, I will try and split the folder creation out to a separate line above the new-VM so it is already created when the new-vm command is executed.

Reply
0 Kudos
markdjones82
Expert
Expert
Jump to solution

Here is your if statement you can place within the loop:

If (Get-Folder $vm.vmlocation -ErrorAction SilentlyContinue){

     Write-host "Folder Already Exists!"

}

else{

     Get-Folder $baselocation | new-folder $vm.vmlocation

}

http://www.twitter.com/markdjones82 | http://nutzandbolts.wordpress.com
Reply
0 Kudos