TheVMinator
Expert
Expert

Find total space consumed on a datastore for a thick provisioned VM

Jump to solution

I'm attempting to find the complete amount of space that will be used for a VM on a datastore. The datastore has a property called "provisioned space"  This property includes swap file space and other items that are NOT included in the provisionedspaceGB number for each individual VM.  As a result, the sum of provisionedspaceGB for all VMS is NOT equal to the provisioned space value for the whole datastore.

How can I find the total space for an individual thick provisioned VM including all files for that VM that will contributed to the "provisioned space" property of the datastore?

Thanks!

0 Kudos
1 Solution

Accepted Solutions
LucD
Leadership
Leadership

You could do something like this.

It will give the total provisioned per datastore.

Note that VMProvisionedGB is the value that comes with the VirtualMachine object, which you mentioned earlier is not 100% correct.

You could/should use the ProvisionedGB property instead on the Measure-Object cmdlet.

$skipDS = @('ISO')

$spec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec

$spec.details = New-Object VMware.Vim.FileQueryFlags

$spec.details.fileType = $true

$spec.details.fileSize = $true

$spec.sortFoldersFirst = $true

$qDiskFile = New-Object VMware.Vim.VmDiskFileQuery

$qDiskFile.Details = New-Object VMware.Vim.VmDiskFileQueryFlags

$qDiskFile.Details.CapacityKb = $true

$qDiskFile.Details.DiskExtents = $true

$qDiskFile.Details.DiskType = $true

$qDiskFile.Details.Thin = $true

$spec.Query += $qDiskFile

$spec.Query += New-Object VMware.Vim.FileQuery

$space = @()

foreach($vm in (Get-VM -Name vm1,vm2,vm3)){

    foreach($ds in (Get-View -Id $vm.ExtensionData.Datastore | where{$skipDS -notcontains $_.Name})){

        $dsBrowser = Get-View -Id $ds.Browser

        $spec.MatchPattern = "*$($vm.Name)*"

        $result = @($dsBrowser.SearchDatastore("[$($ds.Name)] ",$spec))

        $spec.MatchPattern = $null

        $result += @($dsBrowser.SearchDatastore("[$($ds.Name)] $($vm.Name)",$spec))

        $provisioned = $result.File | %{

            if($_.CapacityKB -ne $null){

                $_.CapacityKB*1KB

            }

            else{

                $_.FileSize

            }} | Measure-Object -Sum | select -ExpandProperty Sum

        $used = $result.File | Measure-Object -Property FileSize -Sum | select -ExpandProperty Sum

  

        $space += New-Object PSObject -Property ([ordered]@{

            VM = $vm.Name

            Datastore = $ds.Name

            UsedGB = [Math]::Round($used/1GB,2)

            VMUsedGB = [Math]::Round($vm.UsedSpaceGB,2)

            ProvisionedGB = [Math]::Round($provisioned/1GB,2)

            VMProvisionedGB = [Math]::Round($vm.ProvisionedSpaceGB,2)

        })

    }

}

$space | Group-Object -Property Datastore | %{

    New-Object PSObject -Property @{

        Datastore = $_.Name

        TotalProvisionedGB = $_.Group | Measure-Object -Property VMProvisionedGB -Sum | Select -ExpandProperty Sum

    }

}


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

View solution in original post

0 Kudos
7 Replies
LucD
Leadership
Leadership

Try something like this.

Note this only handles Thick VMDK, for Thin provisioned disks, the script needs additional code.

$vmName = 'TestVM'

$vm = Get-VM -Name $vmName

$spec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec

$spec.MatchPattern = "*$($vmName)*"

$spec.details = New-Object VMware.Vim.FileQueryFlags

$spec.details.fileType = $true

$spec.details.fileSize = $true

$spec.sortFoldersFirst = $true

foreach($ds in Get-View -Id $vm.ExtensionData.Datastore){

    $dsBrowser = Get-View -Id $ds.Browser

    $result = $dsBrowser.SearchDatastoreSubFolders("[" + $ds.Name + "] ",$spec)

    $size = $result.File | Measure-Object -Property FileSize -Sum | select -ExpandProperty Sum

    New-Object PSObject -Property @{

        VM = $vmName

        Datastore = $ds.Name

        Size = $size

        SizeGB = [math]::Round($size/1GB,1)

    }

}


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

TheVMinator
Expert
Expert

OK thanks again.  Getting very warm but still having some issues.  I'm getting a list of VMs. These VMs are scattered across different datastores.  They are about to be migrated to a new storage platform.  They are right now thin provisioned.  After the storage vmotion operations they will be thick provisioned.  I'm attempting to find how much provisioned space the VMs will take on the destination datastore.  It seems the report is giving me used space rather than provisioned space.  For example if I do this:

    $vmlist = get-vm

    foreach($vm in $vmlist){

    $spec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec

    $spec.MatchPattern = "*$($vm.name)*"

    $spec.details = New-Object VMware.Vim.FileQueryFlags

    $spec.details.fileType = $true

    $spec.details.fileSize = $true

    $spec.sortFoldersFirst = $true

    foreach($ds in Get-View -Id $vm.ExtensionData.Datastore){

      $dsBrowser = Get-View -Id $ds.Browser

      $result = $dsBrowser.SearchDatastoreSubFolders("[" + $ds.Name + "] ",$spec)

      $size = $result.File | Measure-Object -Property FileSize -Sum | select -ExpandProperty Sum

      New-Object PSObject -Property @{

        VM = $vm.name

        Datastore = $ds.Name

        Size = $size

        SizeGB = [math]::Round($size/1GB,1)

      }

    }

  }

then the output for each VM from this report is still way less than $vm.provisionedspaceGB and looks about the same as $vm.usedspaceGB

The problem is that $vm.provisionedspaceGB still leaves out something.  The total of $vm.provisionedspaceGB for all VMs once they are all migrated to the new storage system is still less than what I see for "provisionedspaceGB"  on the datatastore's summary tab.  I need to take $vm.provisionedspaceGB and add whatever space it doesn't account for for each vm based on files on the datastore that might not be included in this metric.

can I somehow add $vm.provisionedspaceGB and whatever files aren't included in it and have a new property that includes everything?

Any ideas?

0 Kudos
LucD
Leadership
Leadership

Try the following version.

It will look at all the files that belong to a VM, and will display the result per datastore.
If a VM is located over multiple datastores, the report will contain multiple rows for that VM, one row for each datastore.

As a check I have added the ProvisionedGB and UsedGB properties that come with the VM.

There are some discrepancies between these values sometimes.

Note that $skipDS allows to specify datastores that should be excluded, like for example an ISO repository.

$skipDS = @('ISO')

$spec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec

$spec.details = New-Object VMware.Vim.FileQueryFlags

$spec.details.fileType = $true

$spec.details.fileSize = $true

$spec.sortFoldersFirst = $true

$qDiskFile = New-Object VMware.Vim.VmDiskFileQuery

$qDiskFile.Details = New-Object VMware.Vim.VmDiskFileQueryFlags

$qDiskFile.Details.CapacityKb = $true

$qDiskFile.Details.DiskExtents = $true

$qDiskFile.Details.DiskType = $true

$qDiskFile.Details.Thin = $true

$spec.Query += $qDiskFile

$spec.Query += New-Object VMware.Vim.FileQuery

foreach($vm in (Get-VM -Name VM1,VM2,VM3)){

    foreach($ds in (Get-View -Id $vm.ExtensionData.Datastore | where{$skipDS -notcontains $_.Name})){

        $dsBrowser = Get-View -Id $ds.Browser

        $spec.MatchPattern = "*$($vm.Name)*"

        $result = @($dsBrowser.SearchDatastore("[$($ds.Name)] ",$spec))

        $spec.MatchPattern = $null

        $result += @($dsBrowser.SearchDatastore("[$($ds.Name)] $($vm.Name)",$spec))

        $provisioned = $result.File | %{

            if($_.CapacityKB -ne $null){

                $_.CapacityKB*1KB

            }

            else{

                $_.FileSize

            }} | Measure-Object -Sum | select -ExpandProperty Sum

        $used = $result.File | Measure-Object -Property FileSize -Sum | select -ExpandProperty Sum

   

        New-Object PSObject -Property ([ordered]@{

            VM = $vm.Name

            Datastore = $ds.Name

            UsedGB = [Math]::Round($used/1GB,2)

            VMUsedGB = [Math]::Round($vm.UsedSpaceGB,2)

            ProvisionedGB = [Math]::Round($provisioned/1GB,2)

            VMProvisionedGB = [Math]::Round($vm.ProvisionedSpaceGB,2)

        })

    }

}


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

TheVMinator
Expert
Expert

Thanks - This is really good - I just have one more item I need to try to get but am unable.  How can I total up vmprovisionedGB for all the VMs that go into the foreach loop?

0 Kudos
LucD
Leadership
Leadership

You could do something like this.

It will give the total provisioned per datastore.

Note that VMProvisionedGB is the value that comes with the VirtualMachine object, which you mentioned earlier is not 100% correct.

You could/should use the ProvisionedGB property instead on the Measure-Object cmdlet.

$skipDS = @('ISO')

$spec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec

$spec.details = New-Object VMware.Vim.FileQueryFlags

$spec.details.fileType = $true

$spec.details.fileSize = $true

$spec.sortFoldersFirst = $true

$qDiskFile = New-Object VMware.Vim.VmDiskFileQuery

$qDiskFile.Details = New-Object VMware.Vim.VmDiskFileQueryFlags

$qDiskFile.Details.CapacityKb = $true

$qDiskFile.Details.DiskExtents = $true

$qDiskFile.Details.DiskType = $true

$qDiskFile.Details.Thin = $true

$spec.Query += $qDiskFile

$spec.Query += New-Object VMware.Vim.FileQuery

$space = @()

foreach($vm in (Get-VM -Name vm1,vm2,vm3)){

    foreach($ds in (Get-View -Id $vm.ExtensionData.Datastore | where{$skipDS -notcontains $_.Name})){

        $dsBrowser = Get-View -Id $ds.Browser

        $spec.MatchPattern = "*$($vm.Name)*"

        $result = @($dsBrowser.SearchDatastore("[$($ds.Name)] ",$spec))

        $spec.MatchPattern = $null

        $result += @($dsBrowser.SearchDatastore("[$($ds.Name)] $($vm.Name)",$spec))

        $provisioned = $result.File | %{

            if($_.CapacityKB -ne $null){

                $_.CapacityKB*1KB

            }

            else{

                $_.FileSize

            }} | Measure-Object -Sum | select -ExpandProperty Sum

        $used = $result.File | Measure-Object -Property FileSize -Sum | select -ExpandProperty Sum

  

        $space += New-Object PSObject -Property ([ordered]@{

            VM = $vm.Name

            Datastore = $ds.Name

            UsedGB = [Math]::Round($used/1GB,2)

            VMUsedGB = [Math]::Round($vm.UsedSpaceGB,2)

            ProvisionedGB = [Math]::Round($provisioned/1GB,2)

            VMProvisionedGB = [Math]::Round($vm.ProvisionedSpaceGB,2)

        })

    }

}

$space | Group-Object -Property Datastore | %{

    New-Object PSObject -Property @{

        Datastore = $_.Name

        TotalProvisionedGB = $_.Group | Measure-Object -Property VMProvisionedGB -Sum | Select -ExpandProperty Sum

    }

}


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

View solution in original post

0 Kudos
kbinger_tti
Contributor
Contributor
Anyone have a bit of extra code to get this nicely exported to a csv?
0 Kudos
kbinger_tti
Contributor
Contributor

Figured it out for anyone else who may want it:

$skipDS = @('ISO')
$spec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec
$spec.details = New-Object VMware.Vim.FileQueryFlags
$spec.details.fileType = $true
$spec.details.fileSize = $true
$spec.sortFoldersFirst = $true
$qDiskFile = New-Object VMware.Vim.VmDiskFileQuery
$qDiskFile.Details = New-Object VMware.Vim.VmDiskFileQueryFlags
$qDiskFile.Details.CapacityKb = $true
$qDiskFile.Details.DiskExtents = $true
$qDiskFile.Details.DiskType = $true
$qDiskFile.Details.Thin = $true
$spec.Query += $qDiskFile
$spec.Query += New-Object VMware.Vim.FileQuery
 
$report = foreach($vm in (Get-VM -Name vm1, vm2, vm3)){
    foreach($ds in (Get-View -Id $vm.ExtensionData.Datastore | where{$skipDS -notcontains $_.Name})){
        $dsBrowser = Get-View -Id $ds.Browser
        $spec.MatchPattern = "*$($vm.Name)*"
        $result = @($dsBrowser.SearchDatastore("[$($ds.Name)] ",$spec))
        $spec.MatchPattern = $null
        $result += @($dsBrowser.SearchDatastore("[$($ds.Name)] $($vm.Name)",$spec))
        $provisioned = $result.File | %{
            if($_.CapacityKB -ne $null){
                $_.CapacityKB*1KB
            }
            else{
                $_.FileSize
            }} | Measure-Object -Sum | select -ExpandProperty Sum
        $used = $result.File | Measure-Object -Property FileSize -Sum | select -ExpandProperty Sum
  
        New-Object PSObject -Property ([ordered]@{
            VM = $vm.Name
            Datastore = $ds.Name
            UsedGB = [Math]::Round($used/1GB,2)
            VMUsedGB = [Math]::Round($vm.UsedSpaceGB,2)
            ProvisionedGB = [Math]::Round($provisioned/1GB,2)
            VMProvisionedGB = [Math]::Round($vm.ProvisionedSpaceGB,2)
})
    }
$report | export-csv c:\scripts\output\vmSizePerDatastore.csv -notypeinformation
$skipDS = @('ISO')
$spec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec
$spec.details = New-Object VMware.Vim.FileQueryFlags
$spec.details.fileType = $true
$spec.details.fileSize = $true
$spec.sortFoldersFirst = $true
$qDiskFile = New-Object VMware.Vim.VmDiskFileQuery
$qDiskFile.Details = New-Object VMware.Vim.VmDiskFileQueryFlags
$qDiskFile.Details.CapacityKb = $true
$qDiskFile.Details.DiskExtents = $true
$qDiskFile.Details.DiskType = $true
$qDiskFile.Details.Thin = $true
$spec.Query += $qDiskFile
$spec.Query += New-Object VMware.Vim.FileQuery
 
$report = foreach($vm in (Get-VM -Name txvadmp01, txvadmp07)){
    foreach($ds in (Get-View -Id $vm.ExtensionData.Datastore | where{$skipDS -notcontains $_.Name})){
        $dsBrowser = Get-View -Id $ds.Browser
        $spec.MatchPattern = "*$($vm.Name)*"
        $result = @($dsBrowser.SearchDatastore("[$($ds.Name)] ",$spec))
        $spec.MatchPattern = $null
        $result += @($dsBrowser.SearchDatastore("[$($ds.Name)] $($vm.Name)",$spec))
        $provisioned = $result.File | %{
            if($_.CapacityKB -ne $null){
                $_.CapacityKB*1KB
            }
            else{
                $_.FileSize
            }} | Measure-Object -Sum | select -ExpandProperty Sum
        $used = $result.File | Measure-Object -Property FileSize -Sum | select -ExpandProperty Sum
  
        New-Object PSObject -Property ([ordered]@{
            VM = $vm.Name
            Datastore = $ds.Name
            UsedGB = [Math]::Round($used/1GB,2)
            VMUsedGB = [Math]::Round($vm.UsedSpaceGB,2)
            ProvisionedGB = [Math]::Round($provisioned/1GB,2)
            VMProvisionedGB = [Math]::Round($vm.ProvisionedSpaceGB,2)
})
    }
$report | export-csv c:\scripts\output\vmSizePerDatastore.csv -notypeinformation
0 Kudos