6 Replies Latest reply on Sep 7, 2017 6:27 AM by andreaspa

    PowerCLI and creating alarms

    andreaspa Enthusiast

      Hi,

       

      I have read LucD's excellent blogpost (Alarm expressions - Part 2 : Event alarms - LucD notes ) on how to create alarms and peetz addition to that (https://www.v-front.de/2013/06/creating-vcenter-alarms-with-powercli.html) while I was doing research for a project of mine.

       

      The idea is to separate customer alarms with different folders in vCenter for both hosts and datastores, and create a set of alarms that are the same for each customer but with different recepients. However, I noticed that there was an option I couldn't really find out on my own, and that was advanced conditions for triggers, see this image below:

       

      So, for creating "regular" alarms these settings seems to do it:

      # Create AlarmSpec object
      $alarm = New-Object VMware.Vim.AlarmSpec
      $alarm.Name = "$targetcustomer"
      $alarm.Description = "Host storage status"
      $alarm.Enabled = $TRUE
      
      # Event expression 1 - Host storage status
      # will change state to "Red"
      $expression1 = New-Object VMware.Vim.EventAlarmExpression
      $expression1.EventType = "EventEx"
      $expression1.eventTypeId = "Hardware health changed"
      $expression1.objectType = "HostSystem"
      $expression1.status = "red"
      
      # Attribute comparison for expression 1
      $comparison1 = New-Object VMware.Vim.EventAlarmExpressionComparison
      $comparison1.AttributeName = "Host Connection State"
      $comparison1.Operator = "isEqualTo"
      $comparison1.red = "Disconnected"
      #$comparison1.RedInterval = 300
      $comparison1.yellow = "None"
      #$comparison1.yellowinterval = 300
      
      $comparison1.Value = "1"
      $expression1.Comparisons += $comparison1
      
      # Add event expressions to alarm
      $alarm.expression = New-Object VMware.Vim.OrAlarmExpression
      $alarm.expression.expression += $expression1
      $alarm.expression.expression += $expression2
      #$alarm.expression.expression += $expression3
      
      # Create alarm in vCenter root
      $alarmMgr.CreateAlarm("Folder-group-d1",$alarm)
      

       

      But I have a hard time figuring out how to add these extra trigger conditions.. Any help would be appreciated, this is kind of new ground for me

        • 1. Re: PowerCLI and creating alarms
          LucD Guru
          Community WarriorsUser ModeratorsvExpert

          To create alarms for the Hardware Health states you have to use the com.vmware.vc.cim.CIMGroupHealthStateChanged eventtypeID.

          In fact I already did a post on that, a long time ago, in Alarms – Cody’s Abandon Ship

          That created a general alarm, that fired when anything in the Hardware Health status turned red.

           

          The ones you discovered are for specific groups of Hardware Sensors.

          These groups are listed under the HostNumericSensorType enumeration.

           

          Once you know the group, it is straightforward to create an Alarm.
          For example, this one will fire when one of the SystemBoard sensors fires.

           

           

          $si = Get-View serviceInstance

          $alarmMgr = Get-View -Id $si.Content.AlarmManager

           

          $alarms = Get-View -Id ($alarmMgr.GetAlarm($si.Content.RootFolder))

          $ha = $alarms | where{$_.Info.Name -match "Host hardware"}

           

          $spec = New-Object VMware.Vim.AlarmSpec

          $spec.Name = 'TestHW2'

          $spec.Description = 'Test HW alarm'

          $expr1 = New-Object VMware.Vim.EventAlarmExpression

          $expr1.EventType = 'EventEx'

          $expr1.EventTypeId = 'com.vmware.vc.cim.CIMGroupHealthStateChanged'

          $expr1.ObjectType = 'HostSystem'

          $expr1.Status = 'red'

           

          $exprComp1 = New-Object VMware.Vim.EventAlarmExpressionComparison

          $exprComp1.AttributeName = 'group'

          $exprComp1.Operator = 'equals'

          $exprComp1.Value = 'SystemBoard'

           

          $exprComp2 = New-Object VMware.Vim.EventAlarmExpressionComparison

          $exprComp2.AttributeName = 'newstate'

          $exprComp2.Operator = 'equals'

          $exprComp2.Value = 'red'

           

          $expr1.Comparisons = $exprComp1,$exprComp2

           

          $spec.Expression = $expr1

           

          $alarmMgr.CreateAlarm($si.Content.RootFolder,$spec)

           

           

           

          The layout for these Hardware Sensor alerts is always the same, you specify the group ($exprComp1).

          And then you specify the state on which the alarm shall fire ($exprComp2).

           

          Needless to say, you can combine multiple of these HW Sensor expressions with AndAlarmExpression and OrAlarmExpression

          • 2. Re: PowerCLI and creating alarms
            andreaspa Enthusiast

            Thanks, that helped a lot!

             

            In the spirit of sharing scripts, this is the full script as it is now, I will share the finished one once it's done.

             

            function Add-StorageAlarms($targetcustomer) {
            
                $folder = Get-Folder -Type HostAndCluster -Name $targetcustomer | Get-View
                #$folder = get-folder -type HostAndCluster -Name Test1 | Get-View
            
                $alarmMgr = Get-View AlarmManager
            
                # Create AlarmSpec object
                $alarm = New-Object VMware.Vim.AlarmSpec
                $alarm.Name = "$targetcustomer - Hardware"
                $alarm.Description = "Host hardware status"
                $alarm.Enabled = $TRUE
                $alarm.action = New-Object VMware.Vim.GroupAlarmAction
            
                $trigger1 = New-Object VMware.Vim.AlarmTriggeringAction
                $trigger1.action = New-Object VMware.Vim.SendEmailAction
                $trigger1.action.ToList = "anpa@itgarden.se"
                $trigger1.action.Subject = "$targetcustomer - ESXi Hardware status changed"
                $trigger1.Action.CcList = ""
                $trigger1.Action.Body = ""
            
                # Event expression 1 - Host hardware status
                # will change state to "Red"
                $expression1 = New-Object VMware.Vim.EventAlarmExpression
                $expression1.EventType = "EventEx"
                $expression1.eventTypeId = 'com.vmware.vc.cim.CIMGroupHealthStateChanged'
                $expression1.objectType = "HostSystem"
                $expression1.status = "red"
            
                # Event expression 2 - Host hardware status
                # will change state to "Yellow"
                $expression2 = New-Object VMware.Vim.EventAlarmExpression
                $expression2.EventType = "EventEx"
                $expression2.eventTypeId = 'com.vmware.vc.cim.CIMGroupHealthStateChanged'
                $expression2.objectType = "HostSystem"
                $expression2.status = "yellow"
            
                # Event expression 3 - Host hardware status
                # will change state to "Yellow"
                $expression3 = New-Object VMware.Vim.EventAlarmExpression
                $expression3.EventType = "EventEx"
                $expression3.eventTypeId = 'com.vmware.vc.cim.CIMGroupHealthStateChanged'
                $expression3.objectType = "HostSystem"
                $expression3.status = "green"
            
                $trans1 = New-Object VMware.Vim.AlarmTriggeringActionTransitionSpec
                $trans1.StartState = "green"
                $trans1.FinalState = "yellow"
            
                $trans2 = New-Object VMware.Vim.AlarmTriggeringActionTransitionSpec
                $trans2.StartState = "yellow"
                $trans2.FinalState = "red"
            
                $trans3 = New-Object VMware.Vim.AlarmTriggeringActionTransitionSpec
                $trans3.StartState = "red"
                $trans3.FinalState = "yellow"
            
                $trans4 = New-Object VMware.Vim.AlarmTriggeringActionTransitionSpec
                $trans4.StartState = "yellow"
                $trans4.FinalState = "green"
            
                $trigger1.TransitionSpecs += $trans1
                $trigger1.TransitionSpecs += $trans2
                $trigger1.TransitionSpecs += $trans3
                $trigger1.TransitionSpecs += $trans4
            
                $alarm.action.action += $trigger1
            
                # Add event expressions to alarm
                $alarm.expression = New-Object VMware.Vim.OrAlarmExpression
                $alarm.expression.expression += $expression1
                $alarm.expression.expression += $expression2
                $alarm.expression.expression += $expression3
            
                $alarm.setting = New-Object VMware.Vim.AlarmSetting
                $alarm.setting.reportingFrequency = 0
                $alarm.setting.toleranceRange = 0
            
                $alarm | Format-List
            
                # Create alarm in vCenter root
                #$alarmMgr.CreateAlarm("Folder-group-h1271",$alarm)
                $alarmMgr.CreateAlarm($folder.MoRef,$alarm)
            
            }
            
            $dc = get-datacenter SharedVC01
            $hostfolders = Get-Folder -Type HostAndCluster -Location $dc -Name host | Get-Folder -NoRecursion
            #Write-host Host folders: $hostfolders
            $storagefolders = Get-Folder -Type Datastore -Location $dc | Get-Folder -NoRecursion
            #Write-host Storage folders: $storagefolders
            
            $validhostfolder = $false
            $validstoragefolder = $false
            
            while ($validcustomer -ne $true) {
                Write-host "Enter customer name exactly as shown in vCenter, including any spaces."
                $customer = Read-Host -Prompt "Customer"
                foreach ($hfolder in $hostfolders) {
                    if ($hfolder -like $customer) {
                        Write-Host "- Host folder found" -ForegroundColor Green
                        $validhostfolder = $true
                    } else {
                        #Write-Host Host: "$hfolder" does not equal "$customer" -ForegroundColor Red
                    }
                }
                foreach ($sfolder in $storagefolders) {
                    if ($sfolder -like $customer) {
                        Write-Host "- Storage folder found" -ForegroundColor Green
                        $validstoragefolder = $true
                    } else {
                        #Write-Host Storage: "$sfolder" does not equal "$customer" -ForegroundColor Red
                    }
                }
                if (($validhostfolder -eq $true) -and ($validstoragefolder -eq $true)) {
                    Write-host -ForegroundColor Green "All folders for customer $customer found!"
                    $validcustomer = $true
                } else {
                    if (!$validhostfolder) { Write-Host -ForegroundColor Red "- No valid host folder found, please create one and try again." }
                    if (!$validstoragefolder) { Write-Host -ForegroundColor Red "- No valid datastore folder found, please create one and try again." }
                }
            }
            
            Write-Host -ForegroundColor White "Creating Host alarms for $customer"
            #Add-HostHardwareAlarms($customer) | out-null
            #Add-HostStatusAlarms($customer) | out-null
            #Add-HostStatusAlarms($customer) | out-null
            
            Write-host -ForegroundColor White "Creating storage alarms for $customer"
            #Add-StorageAlarms($customer) | out-null
            

             

            This script asks for a customer (folder) name, this folder must exist for both hosts and datastores under the Datacenter called "SharedVC01" on a vCenter server.

            If the customer (folder) not is found, it will alert the user and ask for a new input.

             

            If valid, it will call a couple of functions to create different alarms. The only function done at the moment is the one that will notify anpa@domain.tld of any host hardware status changes.

             

            On my todo list is to create alarms for datastore usage and other host alerts that our regular monitoring system doesn't catch, and perhaps add some VM folder specific alerts as well (hence the commented out sections off Add-*Alarm parts in this script.

             

            I blame any odd formating on my lack of skill on how to copy from VSCode

            • 3. Re: PowerCLI and creating alarms
              LucD Guru
              User ModeratorsvExpertCommunity Warriors

              Thanks for sharing!
              It would be nice if you post the final version on GitHub - vmware/PowerCLI-Example-Scripts

              • 4. Re: PowerCLI and creating alarms
                andreaspa Enthusiast

                So, I thought it was time to revisit this topic

                 

                I'm making some progress, and now that I'm back from summer vacation I'm revisiting this script and trying to get the final parts to work.

                 

                I have the following function that gives me an error while being called:

                 

                function Add-HostConnectionStatusAlarms($targetcustomer) {

                 

                   $folder = Get-Folder -Type HostAndCluster -Name $targetcustomer | Get-View

                   #$folder = get-folder -type Datastore -Name Test1 | Get-View

                   $alarmMgr = Get-View AlarmManager

                   # Create AlarmSpec object

                   $alarm = New-Object VMware.Vim.AlarmSpec

                   $alarm.Name = "$targetcustomer - Connection status"

                   $alarm.Description = "Host connection status"

                   $alarm.Enabled = $TRUE

                   $alarm.action = New-Object VMware.Vim.GroupAlarmAction

                   $trigger1 = New-Object VMware.Vim.AlarmTriggeringAction

                   $trigger1.action = New-Object VMware.Vim.SendEmailAction

                   $trigger1.action.ToList = "noreply@noreply.se"

                   $trigger1.action.Subject = "$targetcustomer - ESXi Host Connection"

                   $trigger1.Action.CcList = ""

                   $trigger1.Action.Body = "{eventDescription}"

                   # Event expression 1 - Host connection status

                   $expression1 = New-Object VMware.Vim.StateAlarmExpression

                   $expression1.Operator = "isEqual"

                   $expression1.StatePath = "runtime.connectionState"

                   $expression1.Type = "HostSystem"

                   $expression1.Yellow = ""

                   $expression1.Red = "notResponding"

                 

                   # Event expression 2 - Host connection status

                   $expression2 = New-Object VMware.Vim.StateAlarmExpression

                   $expression2.Operator = "isUnequal"

                   $expression2.StatePath = "runtime.Powerstate"

                   $expression2.Type = "HostSystem"

                   $expression2.yellow = ""

                   $expression2.red = "standBy"

                 

                   # Add event expressions to alarm

                   $alarm.action.action += $trigger1

                   $alarm.expression = New-Object VMware.Vim.AndAlarmExpression

                   #$alarm.expression.expression += $expression1

                   $alarm.expression.expression += $expression2

                   $alarm.setting = New-Object VMware.Vim.AlarmSetting

                   $alarm.setting.reportingFrequency = 300

                   $alarm.setting.toleranceRange = 0

                   $alarm | Format-List

                   # Create alarm in vCenter root

                   #$alarmMgr.CreateAlarm("Folder-group-h1271",$alarm)

                   $alarmMgr.CreateAlarm($folder.MoRef,$alarm)

                }

                 

                But, when it's being called it gives me this error:

                Exception calling "CreateAlarm" with "2" argument(s): "A specified parameter was not correct: "

                At C:\Scripts\<hidden>\Set-CustomerAlarm.ps1:160 char:5

                +     $alarmMgr.CreateAlarm($folder.MoRef,$alarm)

                +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException

                    + FullyQualifiedErrorId : VimException

                 

                I've hit my head against the desk now for some time trying to figure it out. Disabling expression1 or expression2 gives the same result, so I guess it's something wrong there..

                 

                I have been looking at this auto-created alarm:

                PS C:\Scripts> $alarm = (Get-AlarmDefinition -name "Host connection and power state")
                PS C:\Scripts> $alarm.ExtensionData
                
                
                Info           : VMware.Vim.AlarmInfo
                Value          : {}
                AvailableField : {}
                MoRef          : Alarm-alarm-1
                Client         : VMware.Vim.VimClientImpl
                
                PS C:\Scripts> $alarm.ExtensionData.Info
                
                
                Key              : alarm-1
                Alarm            : Alarm-alarm-1
                Entity           : Folder-group-d1
                LastModifiedTime : 2016-11-01 14:48:54
                LastModifiedUser :
                CreationEventId  : 0
                LinkedView       :
                Name             : Host connection and power state
                SystemName       : alarm.HostConnectionStateAlarm
                Description      : Default alarm to monitor host connection and power state
                Enabled          : True
                Expression       : VMware.Vim.AndAlarmExpression
                Action           :
                ActionFrequency  : 0
                Setting          : VMware.Vim.AlarmSetting
                PS C:\Scripts> $alarm.ExtensionData.Info.Expression
                
                Expression
                ----------
                {VMware.Vim.StateAlarmExpression, VMware.Vim.StateAlarmExpression}
                
                PS C:\Scripts> $alarm.ExtensionData.Info.Expression.Expression
                
                Operator  : isEqual
                Type      : HostSystem
                StatePath : runtime.connectionState
                Yellow    :
                Red       : notResponding
                
                Operator  : isUnequal
                Type      : HostSystem
                StatePath : runtime.powerState
                Yellow    :
                Red       : standBy
                

                 

                Any assistance would be appreciated.

                 

                EDIT:

                I have tried to run it line by line, and this is how the object looks like:

                 

                (in this example I tried commenting out yellow and red statements, I have also tried without "runtime." added in front of statepath)

                PS C:\Scripts>$alarm | Format-List
                
                
                Name            :  - Connection status
                SystemName      :
                Description     : Host connection status
                Enabled         : True
                Expression      : VMware.Vim.AndAlarmExpression
                Action          : VMware.Vim.GroupAlarmAction
                ActionFrequency :
                Setting         : VMware.Vim.AlarmSetting
                
                PS C:\Scripts> $alarm.Expression
                
                Expression
                ----------
                {VMware.Vim.StateAlarmExpression, VMware.Vim.StateAlarmExpression}
                
                PS C:\Scripts> $alarm.Expression.Expression
                
                
                Operator  : isEqual
                Type      : HostSystem
                StatePath : ConnectionState
                Yellow    :
                Red       :
                
                Operator  : isUnequal
                Type      : HostSystem
                StatePath : PowerState
                Yellow    :
                Red       :
                

                 

                (in this example I tried commenting out yellow and red statements, I have also tried without "runtime." added in front of statepath)

                 

                Also, apologies for all the editing.. The forum software keeps removing my formatting, and inserting tables everywhere..

                • 5. Re: PowerCLI and creating alarms
                  LucD Guru
                  vExpertUser ModeratorsCommunity Warriors

                  If you don't want to use the Yellow state in the expressions, don't fill it in.

                  And you need to specify the TransitionSpec

                  Like this

                   

                     $trigger1 = New-Object VMware.Vim.AlarmTriggeringAction

                     $trigger1.action = New-Object VMware.Vim.SendEmailAction

                     $trigger1.action.ToList = "noreply@noreply.se"

                     $trigger1.action.Subject = "$targetcustomer - ESXi Host Connection"

                     $trigger1.Action.CcList = ""

                     $trigger1.Action.Body =  "{eventDescription}"

                     $trigger1.TransitionSpecs +=New-Object VMware.Vim.AlarmTriggeringActionTransitionSpec

                     $trigger1.TransitionSpecs[0].FinalState = [VMware.Vim.ManagedEntityStatus]::red

                     $trigger1.TransitionSpecs[0].StartState = [VMware.Vim.ManagedEntityStatus]::yellow

                   

                     # Event expression 1 - Host connection status

                     $expression1 = New-Object VMware.Vim.StateAlarmExpression

                     $expression1.Operator = [VMware.Vim.StateAlarmOperator]::isEqual

                     $expression1.StatePath = "runtime.connectionState"

                     $expression1.Type = "HostSystem"

                     $expression1.Red = [VMware.Vim.HostSystemConnectionState]::notResponding

                   

                   

                     # Event expression 2 - Host connection status

                   

                     $expression2 = New-Object VMware.Vim.StateAlarmExpression

                     $expression2.Operator = [VMware.Vim.StateAlarmOperator]::isUnequal

                     $expression2.StatePath = "runtime.powerState"

                     $expression2.Type = "HostSystem"

                     $expression2.red = [VMware.Vim.HostSystemPowerState]::standBy

                   

                  1 person found this helpful
                  • 6. Re: PowerCLI and creating alarms
                    andreaspa Enthusiast

                    Thanks for the quick reply, I updated the script and noticed that the StatePath also was case sensitive.

                     

                    Now it works great! Again, thanks! As soon as the script has been finalized I'll upload it to the git repository you posted about earlier