sidbryd
Enthusiast
Enthusiast

How to increase speed of script?

Jump to solution

I have created a script that will connect to multiple VC's and pull information on all VM's. I have attached a copy. At the moment it is taking 12-15 hours to run across and environment of 1500 VM's.

In a small sandbox environment it runs in 35 seconds (25 VM's). I rebuilt the script using the Get-View Viewtype Virtual Machine as I thought this would help, but it actually ended up running in 48 seconds in the same sandbox environments. Any tips would be really useful.

The script is to pull a list of VM's from all the hosts, then generate a html page with some javascipt attached which is searchable. We have something like 2000 VM's and it can be hard to locate them sometimes.

Script attached.

Regards

Sid

0 Kudos
1 Solution

Accepted Solutions
mattboren
Expert
Expert

Alright, then -- older environment.  I changed the piece that was using the Storage info from v4.0 of the API to use VirtualDisk items instead:

## for environments older that v4.0 (older API)
Get-View -ViewType ClusterComputeResource -Property Name | %{
   
## store this cluster in a variable for later use
    $viewThisCluster = $_
   
Get-View -ViewType HostSystem -SearchRoot $viewThisCluster.MoRef -Property Name | %{
       
## store this host in a variable for later use
        $viewThisHost = $_
       
## get the .Net View object for all VMs (not templates) and return a bit of info for each; add info to the Report array
        Get-View -ViewType VirtualMachine -SearchRoot $viewThisHost.MoRef -Property Name, Guest.HostName, Config.Hardware.NumCPU, Config.Hardware.MemoryMB, Config.Hardware.Device, Guest.IPAddress, Runtime.PowerState, Config.DatastoreUrl -Filter @{"Config.Template" = "False"} | %{
           
$Report += Select -InputObject $_ Name,
                @{N
="DNS";E={$_.Guest.Hostname}},
                @{N
="NumCPU"; E={$_.Config.Hardware.NumCPU}},
                @{N
="MemoryMB"; E={$_.Config.Hardware.MemoryMB}},
                @{N
="DiskGB";E={[Math]::Round((($_.Config.Hardware.Device | ?{$_ -is [VMware.Vim.VirtualDisk]} | Measure-Object -Property CapacityInKB -Sum).Sum / 1mb),2)}},
                @{N
="IP Address";E={@($_.Guest.IPAddress)[0]}},
                @{N
="State";E={$_.Runtime.PowerState}},
                @{N
="VM VC";E={$VISRV}},
                @{N
="Cluster";E={$viewThisCluster.Name}},
                @{N
="ESX Host";E={$viewThisHost.Name}},
                @{N
="Datastore";E={@($_.Config.DatastoreUrl | %{$_.Name})}},
                @{N
="VM URL";E={"<a href=https://$VISRV/ui/vmDirect.do?view=" + ($URLVM = ($SdkUrlB64 = [System.Convert]::ToBase64String(([System.Text.Encoding]::ASCII.GetBytes("wsUrl=http://localhost/sdk&vmId=VirtualMachine|" + $_.MoRef.Value + "&ui=33")))) + "_") + ">$($_.Name)</a>"}}
        }
## end foreach-object
    } ## end Get-View for hosts
} ## end Get-View for Clusters

This has the potential to be less accurate than using the new API's Storage info, as this way is reporting the provisioned size of the virtual disks, not their committed size on the datastores.  Say, for example, the virtual disks are Thin...

But, how does this version do for you if you use this within your overall script?

View solution in original post

0 Kudos
7 Replies
mattboren
Expert
Expert

Hello, sidbryd-

Ooph, 12-15 hours.  Yes, that is rough.  Well, I gave it a bit of work, and updated the main part that I saw to be the issue.  Lines 57 - 68 in the original script are where I focused, and I have attached the optimized script in a txt file.

Here is a snippet with the updated goodness that replaces the items on lines 57 - 68:

Get-View -ViewType ClusterComputeResource -Property Name | %{
   
## store this cluster in a variable for later use
    $viewThisCluster = $_
   
Get-View -ViewType HostSystem -SearchRoot $viewThisCluster.MoRef -Property Name | %{
       
## store this host in a variable for later use
        $viewThisHost = $_
       
## get the .Net View object for all VMs (not templates) and return a bit of info for each; add info to the Report array
        Get-View -ViewType VirtualMachine -SearchRoot $viewThisHost.MoRef -Property Name, Guest.HostName, Config.Hardware.NumCPU, Config.Hardware.MemoryMB, Summary.Storage.Committed, Guest.IPAddress, Runtime.PowerState, Config.DatastoreUrl -Filter @{"Config.Template" = "False"} | %{
           
$Report += Select -InputObject $_ Name,
                @{N
="DNS";E={$_.Guest.Hostname}},
                @{N
="NumCPU"; E={$_.Config.Hardware.NumCPU}},
                @{N
="MemoryMB"; E={$_.Config.Hardware.MemoryMB}},
                @{N
="DiskGB";E={[Math]::Round(($_.Summary.Storage.Committed / 1GB),2)}},
                @{N
="IP Address";E={$_.Guest.IPAddress[0]}},
                @{N
="State";E={$_.Runtime.PowerState}},
                @{N
="VM VC";E={$VISRV}},
                @{N
="Cluster";E={$viewThisCluster.Name}},
                @{N
="ESX Host";E={$viewThisHost.Name}},
                @{N
="Datastore";E={@($_.Config.DatastoreUrl | %{$_.Name})}},
                @{N
="VM URL";E={"<a href=https://$VISRV/ui/vmDirect.do?view=" + ($URLVM = ($SdkUrlB64 = [System.Convert]::ToBase64String(([System.Text.Encoding]::ASCII.GetBytes("wsUrl=http://localhost/sdk&vmId=VirtualMachine|" + $_.MoRef.Value + "&ui=33")))) + "_") + ">$($_.Name)</a>"}}
        }
## end foreach-object
    } ## end Get-View for hosts
} ## end Get-View for Clusters


The big improvements came from doing away with the Get-VM call, and not having to make a call to each of Get-Cluster, Get-VMHost, and Get-Datastore for _each_ VM.

As for time to run -- I tested the original section (roughly lines 57 - 68) and the updated section in an environment with about 360 VMs.  The updated section took about fifty (50) seconds.  After about a half an hour (30 min), I stopped waiting for the original section.  It is still running, and I will check in the morning to see how long it took, just for comparison's sake (and may post back with that info).

So, you just need to test the whole script now.  Again, I only dealt with the aforementioned section, so the rest (which did not look like it would be terribly slow) should work as it did when you posted the script.

I'll be interested to hear how this updated version does for you.

sidbryd
Enthusiast
Enthusiast

I inserted your updated code so it looked like this:

$Report = @()

Foreach ($VISRV in $VISRVs) {
    if ($VISRV.contains($ProdDom))
        {Connect-VIServer -Server $VISRV -user $UserProd -pass $PassProd}
    else
        {Connect-VIServer -Server $VISRV -user $UserLabs -pass $PassLabs}
               
        Get-View -ViewType ClusterComputeResource -Property Name | %{
            ## store this cluster in a variable for later use
            $viewThisCluster = $_
            Get-View -ViewType HostSystem -SearchRoot $viewThisCluster.MoRef -Property Name | %{
                ## store this host in a variable for later use
                $viewThisHost = $_
                ## get the .Net View object for all VMs (not templates) and return a bit of info for each; add info to the Report array
                Get-View -ViewType VirtualMachine -SearchRoot $viewThisHost.MoRef -Property Name, Guest.HostName, Config.Hardware.NumCPU, Config.Hardware.MemoryMB, Summary.Storage.Committed, Guest.IPAddress, Runtime.PowerState, Config.DatastoreUrl -Filter @{"Config.Template" = "False"} | %{
                    $Report += Select -InputObject $_ Name,
                        @{N="DNS";E={$_.Guest.Hostname}},
                        @{N="NumCPU"; E={$_.Config.Hardware.NumCPU}},
                        @{N="MemoryMB"; E={$_.Config.Hardware.MemoryMB}},
                        @{N="DiskGB";E={[Math]::Round(($_.Summary.Storage.Committed / 1GB),2)}},
                        @{N="IP Address";E={$_.Guest.IPAddress[0]}},
                        @{N="State";E={$_.Runtime.PowerState}},
                        @{N="VM VC";E={$VISRV}},
                        @{N="Cluster";E={$viewThisCluster.Name}},
                        @{N="ESX Host";E={$viewThisHost.Name}},
                        @{N="Datastore";E={@($_.Config.DatastoreUrl | %{$_.Name})}},
                        @{N="VM URL";E={"<a href=https://$VISRV/ui/vmDirect.do?view=" + ($URLVM = ($SdkUrlB64 = [System.Convert]::ToBase64String(([System.Text.Encoding]::ASCII.GetBytes("wsUrl=http://localhost/sdk&vmId=VirtualMachine|" + $_.MoRef.Value + "&ui=33")))) + "_") + ">$($_.Name)</a>"}}
                } ## end foreach-object
            } ## end Get-View for hosts
        } ## end Get-View for Clusters

    Disconnect-VIServer -Server $VISRV -Confirm:$false
}

($Report  | Convertto-Html -head $head -body $body -title $title -PostContent $postcont) -replace '&lt;', "<" -replace '&gt;', ">" -replace '<table>', $tableout  | Out-File "$fileloc\$file"

This then gave me the following error:

Get-View : 6/04/2011 3:10:37 PM    Get-View
At D:\Scripts\Get\Testing\TestVMReport3.ps1:72 char:19
+                 Get-View <<<<  -ViewType VirtualMachine -SearchRoot $viewThis
Host.MoRef -Property Name, Guest.HostName, Config.Hardware.NumCPU, Config.Hardw
are.MemoryMB, Summary.Storage.Committed, Guest.IPAddress, Runtime.PowerState, C
onfig.DatastoreUrl -Filter @{"Config.Template" = "False"} | %{
    + CategoryInfo          : NotSpecified: (:) [Get-View], InvalidProperty
    + FullyQualifiedErrorId : Client20_MoServiceImpl_GetNetInteropEntityView_V
   iError,VMware.VimAutomation.ViCore.Cmdlets.Commands.DotNetInterop.GetVIVie
  w

Get-View : 6/04/2011 3:10:38 PM    Get-View
At D:\Scripts\Get\Testing\TestVMReport3.ps1:72 char:19
+                 Get-View <<<<  -ViewType VirtualMachine -SearchRoot $viewThis
Host.MoRef -Property Name, Guest.HostName, Config.Hardware.NumCPU, Config.Hardw
are.MemoryMB, Summary.Storage.Committed, Guest.IPAddress, Runtime.PowerState, C
onfig.DatastoreUrl -Filter @{"Config.Template" = "False"} | %{
    + CategoryInfo          : NotSpecified: (:) [Get-View], InvalidProperty
    + FullyQualifiedErrorId : Client20_MoServiceImpl_GetNetInteropEntityView_V
   iError,VMware.VimAutomation.ViCore.Cmdlets.Commands.DotNetInterop.GetVIVie
  w

0 Kudos
sidbryd
Enthusiast
Enthusiast

And thank you for the awesomely quick feedback!

0 Kudos
mattboren
Expert
Expert

Hello -- Are you connecting to ESX/ESXi hosts directly to run this, or to vCenter server(s)?  And, what version(s) of VMware items are they?

I ask because from that error message (particularly, "InvalidProperty"), it appears that the query might be running against a server (ESX/ESXi or vCenter) that does not have at least version 4.0 of the API.  That is, it seems that there might be some 3.5 or older hosts/Virtual Center servers in play.

In particular, there is one property in use in the code that I provided that was new starting in v4.0 of the API:  Summary.Storage.Committed (reference at http://www.vmware.com/support/developer/vc-sdk/visdk41pubs/ApiReference/vim.vm.Summary.StorageSummar...).

In order to test/narrow it down, you could try the following two lines by themselves from PowerShell:

(Get-View -ViewType VirtualMachine -Property Name).Count

and

(Get-View -ViewType VirtualMachine -Property Name, Summary.Storage.Committed).Count

If the first succeeds and the second fails, that would indicate the problem being in trying to use the property "Summary.Storage.Committed".  There are other ways to get that disk/storage info for the report that you want to make, but finding the cause of the error is the next step.

One other thing that comes to mind is your version of PowerCLI -- with what version are you running the script?  You can get that info via:

Get-PowerCLIVersion

...and, you are welcome.  Let's get this working.

0 Kudos
sidbryd
Enthusiast
Enthusiast

Thanks for getting back to me, please see the information requested below:

PowerCLI Version
----------------
   VMware vSphere PowerCLI 4.1 U1 build 332441

And yes I am running this against an ESX 3.5 environment.

As you thought first command works and second returns an error:

[vSphere PowerCLI] D:\Scripts\Get\Testing> (Get-View -ViewType VirtualMachine -P
roperty Name).Count
18

[vSphere PowerCLI] D:\Scripts\Get\Testing> (Get-View -ViewType VirtualMachine -P
roperty Name, Summary.Storage.Committed).Count
Get-View : 7/04/2011 1:59:43 PM    Get-View
At line:1 char:10
+ (Get-View <<<<  -ViewType VirtualMachine -Property Name, Summary.Storage.Comm
itted).Count
    + CategoryInfo          : NotSpecified: (:) [Get-View], InvalidProperty
    + FullyQualifiedErrorId : Client20_MoServiceImpl_GetNetInteropEntityView_V
   iError,VMware.VimAutomation.ViCore.Cmdlets.Commands.DotNetInterop.GetVIVie
  w

Thanks again for your time.

Regards

Sid

0 Kudos
mattboren
Expert
Expert

Alright, then -- older environment.  I changed the piece that was using the Storage info from v4.0 of the API to use VirtualDisk items instead:

## for environments older that v4.0 (older API)
Get-View -ViewType ClusterComputeResource -Property Name | %{
   
## store this cluster in a variable for later use
    $viewThisCluster = $_
   
Get-View -ViewType HostSystem -SearchRoot $viewThisCluster.MoRef -Property Name | %{
       
## store this host in a variable for later use
        $viewThisHost = $_
       
## get the .Net View object for all VMs (not templates) and return a bit of info for each; add info to the Report array
        Get-View -ViewType VirtualMachine -SearchRoot $viewThisHost.MoRef -Property Name, Guest.HostName, Config.Hardware.NumCPU, Config.Hardware.MemoryMB, Config.Hardware.Device, Guest.IPAddress, Runtime.PowerState, Config.DatastoreUrl -Filter @{"Config.Template" = "False"} | %{
           
$Report += Select -InputObject $_ Name,
                @{N
="DNS";E={$_.Guest.Hostname}},
                @{N
="NumCPU"; E={$_.Config.Hardware.NumCPU}},
                @{N
="MemoryMB"; E={$_.Config.Hardware.MemoryMB}},
                @{N
="DiskGB";E={[Math]::Round((($_.Config.Hardware.Device | ?{$_ -is [VMware.Vim.VirtualDisk]} | Measure-Object -Property CapacityInKB -Sum).Sum / 1mb),2)}},
                @{N
="IP Address";E={@($_.Guest.IPAddress)[0]}},
                @{N
="State";E={$_.Runtime.PowerState}},
                @{N
="VM VC";E={$VISRV}},
                @{N
="Cluster";E={$viewThisCluster.Name}},
                @{N
="ESX Host";E={$viewThisHost.Name}},
                @{N
="Datastore";E={@($_.Config.DatastoreUrl | %{$_.Name})}},
                @{N
="VM URL";E={"<a href=https://$VISRV/ui/vmDirect.do?view=" + ($URLVM = ($SdkUrlB64 = [System.Convert]::ToBase64String(([System.Text.Encoding]::ASCII.GetBytes("wsUrl=http://localhost/sdk&vmId=VirtualMachine|" + $_.MoRef.Value + "&ui=33")))) + "_") + ">$($_.Name)</a>"}}
        }
## end foreach-object
    } ## end Get-View for hosts
} ## end Get-View for Clusters

This has the potential to be less accurate than using the new API's Storage info, as this way is reporting the provisioned size of the virtual disks, not their committed size on the datastores.  Say, for example, the virtual disks are Thin...

But, how does this version do for you if you use this within your overall script?

View solution in original post

0 Kudos
sidbryd
Enthusiast
Enthusiast

Absolutely brilliant, runs in under 5 minutes! Thanks again for your help.

Regards

Sid

0 Kudos