VMware Cloud Community
MartinO70
Contributor
Contributor
Jump to solution

Orphaned files and folders

Hi,

Refering to LucD's "Orphaned files and folders- Spring cleaning"

http://www.lucd.info/2011/04/25/orphaned-files-and-folders-spring-cleaning/

Is it possible to also report the size of ophaned data inside an orphaned folder, I would like to get the total "Size" of data in an orphaned folder, not just the "Provisioned Size" (length) if possible? If the total size of data in an orphaned folder makes it more difficult would a listing of each file/size also work well.

I would use it purely for reporting for other people to action so the delete option is not required.

Many thanks,

Martin

0 Kudos
1 Solution

Accepted Solutions
LucD
Leadership
Leadership
Jump to solution

If you only want the total size of the files in the folders that can be done.

Try this

function Remove-OrphanedData {

<# .
SYNOPSIS   Remove orphaned folders and VMDK files
.DESCRIPTION   The function searches orphaned folders and VMDK files
  
on one or more datastores and reports its findings.    Optionally the function removes  the orphaned folders
   and VMDK files
.
NOTES   AuthorLuc Dekens
.PARAMETER Datastore   One or more datastores.    The default is to investigate all shared VMFS datastores
.
PARAMETER Delete   A switch that indicates if you want to remove the folders
   and VMDK files
.EXAMPLE
   PS> Remove-OrphanedData
.EXAMPLE
 
PS> Get-Datastore ds* | Remove-OrphanedData
.
EXAMPLE
  PS> Remove-OrphanedData -Datastore $ds -Delete
#>

  [
CmdletBinding()]   param(     [parameter(ValueFromPipeline=$true)]     [PSObject[]]$Datastore,
    [switch]$Delete
  )   begin{     $fldList = @{}     $hdList = @{}     $fileMgr = Get-View FileManager
  }  
process{     if(!$Datastore){       $Datastore = Get-Datastore
    }     foreach($ds in $Datastore){       if($ds.GetType().Name -eq "String"){         $ds = Get-Datastore -Name $ds
      }       if($ds.Type -eq "VMFS" -and $ds.ExtensionData.Summary.MultipleHostAccess){         Get-VM -Datastore $ds | %{           $_.Extensiondata.LayoutEx.File | where{"diskDescriptor","diskExtent" -contains $_.Type} | %{             $fldList[$_.Name.Split('/')[0]] = $_.Name
            $hdList[$_.Name] = $_.Name
          }         }         Get-Template | where {$_.DatastoreIdList -contains $ds.Id} | %{           $_.Extensiondata.LayoutEx.File | where{"diskDescriptor","diskExtent" -contains $_.Type} | %{             $fldList[$_.Name.Split('/')[0]] = $_.Name
            $hdList[$_.Name] = $_.Name
          }         }         $dc = $ds.Datacenter.Extensiondata         $flags = New-Object VMware.Vim.FileQueryFlags
        $flags.FileSize = $true
        $flags.FileType = $true         $disk = New-Object VMware.Vim.VmDiskFileQuery
       
$disk.details = New-Object VMware.Vim.VmDiskFileQueryFlags
       
$disk.details.capacityKb = $true
       
$disk.details.diskExtents = $true
       
$disk.details.diskType = $true
        $disk.details.thin = $true         $searchSpec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec
        $searchSpec.details = $flags
       
$searchSpec.Query += $disk
        $searchSpec.sortFoldersFirst = $true         $dsBrowser = Get-View $ds.ExtensionData.browser
        $rootPath = "[" + $ds.Name + "]"
        $searchResult = $dsBrowser.SearchDatastoreSubFolders($rootPath, $searchSpec)         foreach($folder in $searchResult){           if($fldList.ContainsKey($folder.FolderPath.TrimEnd('/'))){             foreach ($file in $folder.File){               if(!$hdList.ContainsKey($folder.FolderPath + $file.Path)){                 New-Object PSObject -Property @{                   Folder = $folder.FolderPath
                  Name = $file.Path
                  Size = $file.FileSize
                 
CapacityKB = $file.CapacityKb
                 
Thin = $file.Thin
                 
Extents = [string]::Join(',',($file.DiskExtents | %{$_}))                 }                 if($Delete){                   $dsBrowser.DeleteFile($folder.FolderPath + $file.Path)                 }               }             }           }           elseif($folder.File | where {"cos.vmdk","esxconsole.vmdk" -notcontains $_.Path}){             New-Object PSObject -Property @{               Folder = $folder.FolderPath
              Name = $null
              Size = &{                 $dsName,$folderName = $Folder.FolderPath.Split(']')                 $dsName = $dsName.Trim('[ ')                 $folderName = $folderName.Trim(' ')                 New-PSDrive -Name DSTemp -PSProvider VimDatastore -Root "\" -Datastore (Get-Datastore -Name $dsName) | Out-Null
                Push-Location -Path "DSTemp:\$folderName"
               
Get-ChildItem -Recurse | Measure-Object -Property Length -Sum |
               
Select -ExpandProperty Sum
               
Pop-Location
                Remove-PSDrive -Name DSTemp -Confirm:$false | Out-Null
              }               CapacityKB = $null
             
Thin = $null
              Extents = $null
            }            
if($Delete){               $fileMgr.DeleteDatastoreFile($folder.FolderPath,$dc.MoRef)             }           }         }       }     }   } } Remove-OrphanedData -Datastore MyDS


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

View solution in original post

0 Kudos
11 Replies
LucD
Leadership
Leadership
Jump to solution

I don't want to change the function by adding a file list.

The reason is that this would put different objects on the pipeline and that makes it hard to save the results to for example a CSV file.

But you use the produced output to list content of the folders.

Something like this for example

Remove-OrphanedData -Datastore MyDS | %{
  if($_.Folder){
    $ds = $_.Folder.Split(']')[0].Split('[')[1]
    $folder = $_.Folder.Split(']')[1].Trim(' ')
    New-PSDrive -Name DSTemp -PSProvider VimDatastore -Root "\" -Datastore (Get-Datastore -Name $ds) | Out-Null
    Push-Location -Path "DSTemp:\$folder"
    Get-ChildItem -Recurse
    Pop-Location
    Remove-PSDrive -Name DSTemp -Confirm:$false | Out-Null
  } }


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
MartinO70
Contributor
Contributor
Jump to solution

Thanks, that helps but I wouldn't necessarily need the list of all the files in the folder, more important would be to have the total size of the orphaned folder. So from the original script and its output would I just need the Size of folders added as this now comes out blank.

Hope this is possible 🙂

0 Kudos
LucD
Leadership
Leadership
Jump to solution

If you only want the total size of the files in the folders that can be done.

Try this

function Remove-OrphanedData {

<# .
SYNOPSIS   Remove orphaned folders and VMDK files
.DESCRIPTION   The function searches orphaned folders and VMDK files
  
on one or more datastores and reports its findings.    Optionally the function removes  the orphaned folders
   and VMDK files
.
NOTES   AuthorLuc Dekens
.PARAMETER Datastore   One or more datastores.    The default is to investigate all shared VMFS datastores
.
PARAMETER Delete   A switch that indicates if you want to remove the folders
   and VMDK files
.EXAMPLE
   PS> Remove-OrphanedData
.EXAMPLE
 
PS> Get-Datastore ds* | Remove-OrphanedData
.
EXAMPLE
  PS> Remove-OrphanedData -Datastore $ds -Delete
#>

  [
CmdletBinding()]   param(     [parameter(ValueFromPipeline=$true)]     [PSObject[]]$Datastore,
    [switch]$Delete
  )   begin{     $fldList = @{}     $hdList = @{}     $fileMgr = Get-View FileManager
  }  
process{     if(!$Datastore){       $Datastore = Get-Datastore
    }     foreach($ds in $Datastore){       if($ds.GetType().Name -eq "String"){         $ds = Get-Datastore -Name $ds
      }       if($ds.Type -eq "VMFS" -and $ds.ExtensionData.Summary.MultipleHostAccess){         Get-VM -Datastore $ds | %{           $_.Extensiondata.LayoutEx.File | where{"diskDescriptor","diskExtent" -contains $_.Type} | %{             $fldList[$_.Name.Split('/')[0]] = $_.Name
            $hdList[$_.Name] = $_.Name
          }         }         Get-Template | where {$_.DatastoreIdList -contains $ds.Id} | %{           $_.Extensiondata.LayoutEx.File | where{"diskDescriptor","diskExtent" -contains $_.Type} | %{             $fldList[$_.Name.Split('/')[0]] = $_.Name
            $hdList[$_.Name] = $_.Name
          }         }         $dc = $ds.Datacenter.Extensiondata         $flags = New-Object VMware.Vim.FileQueryFlags
        $flags.FileSize = $true
        $flags.FileType = $true         $disk = New-Object VMware.Vim.VmDiskFileQuery
       
$disk.details = New-Object VMware.Vim.VmDiskFileQueryFlags
       
$disk.details.capacityKb = $true
       
$disk.details.diskExtents = $true
       
$disk.details.diskType = $true
        $disk.details.thin = $true         $searchSpec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec
        $searchSpec.details = $flags
       
$searchSpec.Query += $disk
        $searchSpec.sortFoldersFirst = $true         $dsBrowser = Get-View $ds.ExtensionData.browser
        $rootPath = "[" + $ds.Name + "]"
        $searchResult = $dsBrowser.SearchDatastoreSubFolders($rootPath, $searchSpec)         foreach($folder in $searchResult){           if($fldList.ContainsKey($folder.FolderPath.TrimEnd('/'))){             foreach ($file in $folder.File){               if(!$hdList.ContainsKey($folder.FolderPath + $file.Path)){                 New-Object PSObject -Property @{                   Folder = $folder.FolderPath
                  Name = $file.Path
                  Size = $file.FileSize
                 
CapacityKB = $file.CapacityKb
                 
Thin = $file.Thin
                 
Extents = [string]::Join(',',($file.DiskExtents | %{$_}))                 }                 if($Delete){                   $dsBrowser.DeleteFile($folder.FolderPath + $file.Path)                 }               }             }           }           elseif($folder.File | where {"cos.vmdk","esxconsole.vmdk" -notcontains $_.Path}){             New-Object PSObject -Property @{               Folder = $folder.FolderPath
              Name = $null
              Size = &{                 $dsName,$folderName = $Folder.FolderPath.Split(']')                 $dsName = $dsName.Trim('[ ')                 $folderName = $folderName.Trim(' ')                 New-PSDrive -Name DSTemp -PSProvider VimDatastore -Root "\" -Datastore (Get-Datastore -Name $dsName) | Out-Null
                Push-Location -Path "DSTemp:\$folderName"
               
Get-ChildItem -Recurse | Measure-Object -Property Length -Sum |
               
Select -ExpandProperty Sum
               
Pop-Location
                Remove-PSDrive -Name DSTemp -Confirm:$false | Out-Null
              }               CapacityKB = $null
             
Thin = $null
              Extents = $null
            }            
if($Delete){               $fileMgr.DeleteDatastoreFile($folder.FolderPath,$dc.MoRef)             }           }         }       }     }   } } Remove-OrphanedData -Datastore MyDS


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
MartinO70
Contributor
Contributor
Jump to solution

Great, many thanks! Thanks works.

I'm guessing there's no easy way to get the actual "Size" rather than the "Provisioned" (Length) size for thin provisioned disks when making up the total of the folder?

/M

0 Kudos
LucD
Leadership
Leadership
Jump to solution

The method I used is rather "raw". The PSProvider will return the vDisks as a .vmdk file and a -flat.vmdk file.

The size these -flat.vmdk files have are indeed the Provisioned sizes.

Getting the actual size for the vDisks is possible but it will require a different way of calculating the total foldersize.

So yes, it is possible, but not in an easy way I'm afraid.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
MartinO70
Contributor
Contributor
Jump to solution

Ok, it's not a big issue as most of our disks currently are thick provisioned anyway.

Many thanks for your help!

/M

0 Kudos
MartinO70
Contributor
Contributor
Jump to solution

Sorry, one other question...

Is there anyway to also have a "cluster name" column in this report?

0 Kudos
LucD
Leadership
Leadership
Jump to solution

Based on the datastore I presume ?

Part of these folders are relics from VMs that do not exist anymore, so it will be difficult to use the VM.

Do you have datastores shared between multiple clusters ?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
MartinO70
Contributor
Contributor
Jump to solution

Sorry, yes, based on datastore.

There are no datastores that are shared between multiple clusters

0 Kudos
LucD
Leadership
Leadership
Jump to solution

You can add a line like this to the New-Object properties

Cluster = Get-VMHost -Datastore $ds | Select -First 1 | Get-Cluster | Select -ExpandProperty Name


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
MartinO70
Contributor
Contributor
Jump to solution

Ahh, very clever.

Perfect - many thanks again.

/M

0 Kudos