The VM notes in our environment are unfortunately not standardized. Quite often we end up with varying formats of identifying who requested and who created vms. I am currently working on reclaiming resources and need a way to extract the notes field into a csv for one column per newline.
My desired output regardless of what information each line of the notes state is one column per line as shown below for the above example VM notes.
I'm pretty sure the solution is going to be similar to Export all Notes from a VM in one line but with modifications. It's those modifications that I'm struggling with. Any and all help is appreciated.
No, that is the right place.
To test only for lines that have a ':' in there, except in the 1st position we can use
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Can you post what you currently have?
You could do something like this
ForEach-Object -Process {
$obj = @{
VM = $vm.Name
}
$vm.Notes.Split("`n") |
ForEach-Object -Process {
$name,$value = $_.Split(':')
$obj.Add($name,$value)
}
New-Object -Typename psobject -Property $obj
} | Export-Csv -Path .\report.csv -NoTypeInformation -UseCulture
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
LucD
Thank you once again for a speedy reply. I get the following message onscreen:
The csv reads:
I thought it was because of an inconsistancy in variable: VM = $vmName vs $name later in the code. The output was the same when I made the switch to:
Get-VM -PipelineVariable vm |
ForEach-Object -Process {
$obj = @{
VM = $vmName
}
$vm.Notes.Split("`n") |
ForEach-Object -Process {
$vmName,$value = $_.Split(':')
$obj.Add($vmName,$value)
}
$obj
}
That should have been $vm.Name.
I corrected the code above.
For the CSV creation, I corrected the creation of the object.
The error seems to indicate that you double entries in the Notes.
For example:
Created by: xxx Created by: zzz
Is that possible?
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Anything is possible, Luc. I have 3760 VMs in this particular vcenter, and we've had staff turnover over the years. I can't be 100% certain that someone didn't have two Created By enteries in one note.
I strongly suspect the VM records will only contain one "Created By" per VM, however, it is our team's desire to have each VM record with a "Created by" and "Requested For" as well as "Team".
The csv generated this time around only had a list of VM names. No notes. As well there were only 1627 rows (VMs) in the csv. It's entirely possible to have some VMs with no notes whatsoever. The error/message below generated on screen during the running of the script.
I checked my command line to see what the variables gave me for output. This particular VM did not appear on the CSV. I've tried to find a pattern of why one vm name appears on csv vs another and I can't see one. It seems random but that's atypical for your coding. I'm perplexed for sure.
**Addition to above: I just noticed this is in the csv. Not sure why there is a column header "This is for Oracle".
I built in some tests, if a Notes field is empty or if a Notes field contains the same type of entries twice or more, a message on the console will be displayed.
Perhaps this info will allow to do a better analysis of the Notes field.
ForEach-Object -Process {
$obj = @{
VM = $vm.Name
}
if($vm.Notes){
$vm.Notes.Split("`n") |
ForEach-Object -Process {
$name,$value = $_.Split(':')
if($obj.ContainsKey($name)){
Write-Host "VM $($vm.Name) seems to have multiple entries for $name"
}
else{
$obj.Add($name,$value)
}
}
}
else{
Write-Host "VM $($vm.Name) has no notes"
}
New-Object -Typename psobject -Property $obj
} | Export-Csv -Path .\report.csv -NoTypeInformation -UseCulture
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
The script will create a row per VM, where each row contains the VM's name and then a column per 'field' in the Notes property.
For example
VM Created by Requested byMyVM Me you
But I just realise that Export-Csv has this known limitation where it only looks at the first object to determine how many columns there will be.
So if the first VM doesn't have any field in Notes, the columns for subsequent VMs will only have the VM column.
A quick solution for that is to order the objects by number of properties in a descending way.
Like this
ForEach-Object -Process {
$obj = @{
VM = $vm.Name
}
if($vm.Notes){
$vm.Notes.Split("`n") |
ForEach-Object -Process {
$name,$value = $_.Split(':')
if($obj.ContainsKey($name)){
Write-Host "VM $($vm.Name) seems to have multiple entries for $name"
}
else{
$obj.Add($name,$value)
}
}
}
else{
Write-Host "VM $($vm.Name) has no notes"
}
New-Object -Typename psobject -Property $obj
} | Sort-Object -Property {($_ | Get-Member).Count} -Descending |
Export-Csv -Path .\report.csv -NoTypeInformation -UseCulture
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
We are getting closer. I'm willing to do some clean up on our vcenter where there are valid duplicates. It would be nice to get the multiple errors to errors.csv. It will help with data cleanup. That being said, I believe it's generating the multiple entry line on screen when it sees blank lines in the Notes.
I like what you're doing with the csv. Creator, Deployment Date, VM, etc columns. That's awesome. I do see someone used the equal sign vs the colon "Deployment Date=2013-10-22" when I look at the csv. I'm willing to clean up my data in those cases where there are valid duplicates in the Notes entry.
We also need to fix up how the csv is dealing with random notes in the Notes entry. Those notes that aren't prefaced with a colon as shown with blue highlight in the following image.
After all is said in done, I'm going to propose to our team we use tags for standardization of required information on VMs, but that's the future.
This is where we're at, Luc.
I placed the New-Object in a try-catch, which should give us more details when the error occurs.
Also, you can specify multiple characters on the Split, so I added the '=' character.
Let's see if this brings us any further
ForEach-Object -Process {
$obj = @{
VM = $vm.Name
}
if($vm.Notes){
$vm.Notes.Split("`n") |
ForEach-Object -Process {
$name,$value = $_.Split(':=')
if($obj.ContainsKey($name)){
Write-Host "VM $($vm.Name) seems to have multiple entries for $name"
}
else{
$obj.Add($name,$value)
}
}
}
else{
Write-Host "VM $($vm.Name) has no notes"
}
try{
New-Object -Typename psobject -Property $obj -ErrorAction Stop
}
catch{
Write-Host "Error creating the object for VM $($vm.Name)"
$obj.GetEnumerator()
}
} | Sort-Object -Property {($_ | Get-Member).Count} -Descending |
Export-Csv -Path .\report.csv -NoTypeInformation -UseCulture
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Closer.
1. We need to deal with blank lines.
2. If there is a word: make a column for word.
3. If there is a statement with no : then dump into a single column for that vm row. No need to make a separate column for each statement without : in it.
I'd sooner not clean up blank lines in our VMs that have notes that way, even though they're not supposed to.
I want columns for those items that have a colon.
I get this for each VM that has notes with blank lines between rows of information.
This is the corresponding notes:
My guess is the blank lines are being viewed as a multiple entry. I am willing to clean up the valid mulitple entry but yikes!! Tons of VMs with blank rows, and tons without. In the meantime, I'll start the cleanup of valid duplicate data.
No red errors this time. Just script output. Progress, I'd say.
We should be able to get rid of the blank lines with a Where-clause.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Well that gave me an interesting csv. I'm guessing I put your suggestion in the wrong place.
Get-VM -PipelineVariable vm |
ForEach-Object -Process {
$obj = @{
VM = $vm.Name
}
if($vm.Notes){
$vm.Notes.Split("`n") | where{$_ -ne ''} |
ForEach-Object -Process {
$name,$value = $_.Split(':=')
if($obj.ContainsKey($name)){
Write-Host "VM $($vm.Name) seems to have multiple entries for $name"
}
else{
$obj.Add($name,$value)
}
}
}
else{
Write-Host "VM $($vm.Name) has no notes"
}
try{
New-Object -Typename psobject -Property $obj -ErrorAction Stop
}
catch{
Write-Host "Error creating the object for VM $($vm.Name)"
$obj.GetEnumerator()
}
} | Sort-Object -Property {($_ | Get-Member).Count} -Descending |
Export-Csv -Path .\report-noblank.csv -NoTypeInformation -UseCulture
CSV:
How about we ask for those lines in notes that have verbiage with a colon separator? From the looks of it, someone has filesystem information recorded in notes. Skip everything that isn't prefaced with XYZ: (XYZ could be anything as long as it's followed by a colon).
No, that is the right place.
To test only for lines that have a ':' in there, except in the 1st position we can use
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Yay!!!! That did it Luc. I just have to clean up my VM Notes where there are duplicate elements and I'm good to go. For something that seemed like it was going to be easy it sure wasn't. Thank you so much for your help.
Get-VM -PipelineVariable vm |
ForEach-Object -Process {
$obj = @{
VM = $vm.Name
}
if($vm.Notes){
$vm.Notes.Split("`n") | where{$_ -match '^(?!:).*:.*'} |
ForEach-Object -Process {
$name,$value = $_.Split(':=')
if($obj.ContainsKey($name)){
Write-Host "VM $($vm.Name) seems to have multiple entries for $name"
}
else{
$obj.Add($name,$value)
}
}
}
else{
Write-Host "VM $($vm.Name) has no notes"
}
try{
New-Object -Typename psobject -Property $obj -ErrorAction Stop
}
catch{
Write-Host "Error creating the object for VM $($vm.Name)"
$obj.GetEnumerator()
}
} | Sort-Object -Property {($_ | Get-Member).Count} -Descending |
Export-Csv -Path .\report-noblank2.csv -NoTypeInformation -UseCulture