We are trying to capture the timings of a spurious disk IO latency issue in some VMs.
I though to try creating an alarm as follows:
For the VM an alarm was created that is triggered when total storage latency exceeds 50ms for 0 seconds (so even a spike) is captured for a warning. For critical the trigger is 100ms for 30s.
For the action I have "Run a command", in the command field I have "logger "High Storage IO Latency Detected"". The command works from the VCSA CLI > shell interface.
I am seeing the syslog entries for when the alarm transitions from green to yellow and to red and back to green, and I see a syslog entry which basically says:
Alarm {alarm name} on {entity - VM name} triggered an action
But I am also seeing:
Alarm {alarm name} on {entity - VM name} did not complete script. A general system error occurred: Failed to run script: Failed to invoke command: Stream is not initialized
Is this because I am not trying to run a script but a direct command. Can the "command" field not actually be a command but needs to be a PY or SH script?
For our purposes the alarm is working, as we can get the time of events from the output above, but it would be useful to know how to send more useful Syslog output in response to alarms, for example eventually to include some of the alarm metadata, but would these necessitate the creation of a script on VCSA or can it be encapsulated in the command field?
