Script to report RPO violations

[Baoth]

Hi all

I have seen a few forums posts in regards to something similar to what I am trying to achieve, and although I understand getting replication status is somewhat difficult as there is no public API, I am hoping that something can be tweaked to get what I need via a script.

I am currently running the below which I found here:

Get-VIEvent -MaxSamples ([int]::MaxValue) | Where { $_.EventTypeId -match "hbr|rpo" } | Select CreatedTime, FullFormattedMessage, @{Name="VMName";Expression={$_.Vm.Name}} | export-csv -NoTypeInformation -Path ([Environment]::GetFolderPath("Desktop") + "\HBR-RPOEvents.csv")

However, that is returning everything under the FullFormattedMessage column. I'd like to filter that so it only contains "Virtual machine vSphere Replication RPO is violated by 1 minute(s)".

If possible, I would like to avoid the 1 minute part of the log though. The reason for this is that Alerts work well enough, and will email me when something is violated, but its constantly sending me the 1 minute messages, which subsequently clear under normal operation.

I'd ultimately like to get to a stage where I'm alerted if something is violated by more than 1 minute, and covers the last 24 hours.

Hope you can help.

Thanks.

[LucD]

On the 1st part of your question, you can extend your Where-clause to filter on the message content.

Something like this

Get-VIEvent -MaxSamples ([int]::MaxValue) |

Where { $_.EventTypeId -match "hbr|rpo" -and $_.FullFormattedMessage -match "Virtual machine vSphere Replication RPO is violated by 1 minute(s)" } |

Select CreatedTime, FullFormattedMessage,

    @{Name="VMName";Expression={$_.Vm.Name}} |

Export-Csv -NoTypeInformation -Path ([Environment]::GetFolderPath("Desktop") + "\HBR-RPOEvents.csv")

[Baoth]

Thanks LucD

I have it a go, and it didn't return anything after a good few minutes.

Tried this section of the script, and nothing is returned in the PS window:

Get-VIEvent -Start (Get-Date).adddays(-2) -MaxSamples ([int]::MaxValue) |

Where { $_.EventTypeId -match "hbr|rpo" -and $_.FullFormattedMessage -match "Virtual machine vSphere Replication RPO is violated by 1 minute(s)" }

If I drop the 2nd match section, I get data returned successfully.

[LucD]

Could you provide a sample of the text in the FullFormattedMessage you want?

[Baoth]

From the email Alerts I set up, with RPO Violations, I get mailed the following:

Alarm Definition:

([Event alarm expression: RPO violated])

Event details:

Virtual machine vSphere Replication RPO is violated by 1 minute(s)

I'm really more interested in "RPO Violated" than anything, and would really like it to ignore the 1 minute warning so to avoid unnecessary concern.

[LucD]

I mean what exactly is in the FullFormattedMessage field?

[Baoth]

Had to generalise some names, but here's the output. The parts in quotes are from the FullFormattedMessage column:

09/10/2019 00:26 "Sync started by VR Scheduler for virtual machine <VM-NAME> on host <HOST> in cluster <CLUSTER> in <DC>." <VM-NAME>

09/10/2019 00:27 "Virtual machine vSphere Replication RPO is violated by 1 minute(s)" <VM-NAME>

09/10/2019 00:30 "Virtual machine vSphere Replication RPO is no longer violated" <VM-NAME>

[LucD]

I forgot the escape the RegEx meta-characters in the condition.

Try like this

Get-VIEvent -MaxSamples ([int]::MaxValue) |

Where { $_.EventTypeId -match "hbr|rpo" -and $_.FullFormattedMessage -match "Virtual machine vSphere Replication RPO is violated by 1 minute\(s\)" } |

Select CreatedTime, FullFormattedMessage,

    @{Name="VMName";Expression={$_.Vm.Name}} |

Export-Csv -NoTypeInformation -Path ([Environment]::GetFolderPath("Desktop") + "\HBR-RPOEvents.csv")

[Baoth]

That's returning data now. Great, thanks!

Would the opening section like this work, to only cover the last 24 hours?

Get-VIEvent -Start (Get-Date).adddays(-1) -MaxSamples ([int]::MaxValue |

Is there any way to have a denominator that is anything other than 1 minute in the second match statement please?

[LucD]

Yes, that only retrieves the events from the last 24 hours.

[Baoth]

Hi Luc

Apologies for the long reply time.

Thanks for that. I'm now using the following which is filtering the results as I would hope to see - last 24 hours of violations that exclude 1 minute:

Get-VIEvent -Start (Get-Date).adddays(-1) -MaxSamples ([int]::MaxValue) |

Where { $_.EventTypeId -match "hbr|rpo" -and $_.FullFormattedMessage -match "Virtual machine vSphere Replication RPO is violated" -and $_.FullFormattedMessage -notmatch "violated by 1 minute"} |

Select CreatedTime, FullFormattedMessage,

    @{Name="VMName";Expression={$_.Vm.Name}} |

Export-Csv -NoTypeInformation -Path ([Environment]::GetFolderPath("Desktop") + "\HBR-RPOEvents_24hrs.csv")

It does take quite a while to run for some reason - sometimes up to an hour. I'm guessing its something to do with the MaxSamples section, as it will be scouring every event available before filtering?

[LucD]

No, the MaxSamples parameter is ignored when the Start or Finish parameters are used on Get-VIEvent.

Which PowerCLI version are you using?

There have been some changes on the Get-VIEvent cmdlet in recent versions.

As an alternative, you might want to try my Get-VIEventPlus function in my Get The VMotion/SvMotion History post.

[Baoth]

Here's what I'm using:

PowerCLI Version

----------------

   VMware PowerCLI 11.0.0 build 10380590

---------------

Component Versions

---------------

   VMware Cis Core PowerCLI Component PowerCLI Component 11.0 build 10335701

   VMware VimAutomation VICore Commands PowerCLI Component PowerCLI Component 11.0 build 10336080

[LucD]

You could upgrade to the latest version, but that will probably not change a lot on the execution time I'm afraid.

[Baoth]

Will do. Can't hurt to see if it does affect anything.

I think I need one more tweak, and it accomplishes what I need. Can the output remove duplicate VM names and expand the column widths?

As it stands, it outputs the following from the last time I ran it:

CreatedTime	FullFormattedMessage	VMName
21/10/2019 13:48	Virtual machine vSphere Replication RPO is violated by 2088 minute(s)	VM1
21/10/2019 13:33	Virtual machine vSphere Replication RPO is violated by 2073 minute(s)	VM1
21/10/2019 13:18	Virtual machine vSphere Replication RPO is violated by 2454 minute(s)	VM2
21/10/2019 13:18	Virtual machine vSphere Replication RPO is violated by 2058 minute(s)	VM1
21/10/2019 13:03	Virtual machine vSphere Replication RPO is violated by 2043 minute(s)	VM1
21/10/2019 13:03	Virtual machine vSphere Replication RPO is violated by 2439 minute(s)	VM2
21/10/2019 12:48	Virtual machine vSphere Replication RPO is violated by 2074 minute(s)	VM3

Cheers

[LucD]

Try something like this.

It will show 1 result per VM, and it will use the most recent one.

Get-VIEvent -Start (Get-Date).adddays(-1) -MaxSamples ([int]::MaxValue) |

Where { $_.EventTypeId -match "hbr|rpo" -and $_.FullFormattedMessage -match "Virtual machine vSphere Replication RPO is violated" -and $_.FullFormattedMessage -notmatch "violated by 1 minute"} |

Select CreatedTime, FullFormattedMessage,

    @{Name="VMName";Expression={$_.Vm.Name}} |

Group-Object -Property VMName |

ForEach-Object -Process {

    $_.Group | Sort-Object -Property CreatedTime -Descending |

    select -First 1

} |

Export-Csv -NoTypeInformation -Path ([Environment]::GetFolderPath("Desktop") + "\HBR-RPOEvents_24hrs.csv")

[Baoth]

Left it run for a few hours, and although it created the file on my desktop, it didnt populate any information within it.

[LucD]

I'm afraid that retrieving events is not the fastest process around.

Did you upgrade to PowerCLI 11.4 in the mean time?
Did you try my Get-VIEventPlus?

[Baoth]

Just having a look at your function now.

Does the use of it change the script other than starting it with Get-VIEventPlus ?

[LucD]

The parameters should be compatible, except for the additional ones.

But are you on PowerCLI 11.4?
I would try that first.