dhyedrated
Contributor
Contributor

Query Alerts in VROPs 8 timeouts

I'm trying to use the queryAlerts REST API to get CRITICAL, active alerts for HOSTSYSTEMS and DATASTORES.  Initial testing of this method looked promising, however now I only receive successful results when my query results in no alerts.  When I confirm that an an alert exists in the dashboard, I use the api to get it's resourceId and translate the date into their required UNIX EPOCH MS and submit a pretty simple request.  However, I've either received 400 Bad Request or my request times out.  Even if i set the HTTP timeout  value to >300 seconds and/or set the pageSize value to <5

Many times the status code 400 errors are JSON syntax errors but I believe I've gotten all of those, however, a few times I can trace a timeout to a syntax error too, was that a coincidence or is that also a realistic result?  If so, does VROPs use a stricter JSON parser than https://jsonlint.com/?

I've tried pretty simple queries, and the docs say everything is optional:

{

"compositeOperator ": "AND ",

"resource-query": {

"adapterKind": ["VMWARE"],

"resourceKind": ["HostSystem",

"Datastore"

],

"resourceId": idArray

},

"startTimeRange": {

"startTime": unixEPoch,

"endTime": unixEPochEnd

}

}

Is there a administrator log that could shed some light on this issue?

Does the VROPS api have default rate limits?  If so where are they configured?

Thank you

0 Kudos
13 Replies
KabirAli82
Hot Shot
Hot Shot

Im not a 100% clear what you want the end result to look like.

Why do you:

I use the api to get it's resourceId and translate the date into their required UNIX EPOCH MS and submit a pretty simple request.

When you get the alert id, you can use that to query the alert. I dont understand why you are translating the Epoch timestamp.

Maybe this post of mine will help you: https://www.kablog.nl/2020/05/18/trend-micro-vrealize-operations-dashboard-api/


Was I helpful? Give a kudo for appreciation!
Braindumping @ http://kablog.nl/
Tweeting @ https://twitter.com/_Kabir_Ali_
0 Kudos
dhyedrated
Contributor
Contributor

Thanks KabirAli,

I apologize if i wasn't clear.  The "translating" I am doing right now is strictly to  ensure my proof of concept is working.  The end goal is have this small C# program poll for new CRITICAL/ACTIVE alerts  from 100 ESXI hosts and their attached datastores.  When an alert is found further logic is performed; we are on a closed network without internet or mail server access so much of the "out of the box" automation isn't applicable.

As you say this should be a pretty simple query, however it wasn't working, so I took a step back and tried a small number of resourceids and a date I knew had an alert and wrote a simple console application:

1) Acquires a token

2) submits a POST query with a small JSON blob

3) output results

Even with a multi minute timeout this fails, but if I tweak the enddate value of my startdaterange to something earlier, I get no alerts almost instantly

I hope that makes more sense

Thanks for your code, I will take a look

0 Kudos
KabirAli82
Hot Shot
Hot Shot

Hi,

No worries. I think I understand your end goal. How does your URL look like? Then i'll try that in my lab as well.

Also which versions are you using?


Was I helpful? Give a kudo for appreciation!
Braindumping @ http://kablog.nl/
Tweeting @ https://twitter.com/_Kabir_Ali_
0 Kudos
dhyedrated
Contributor
Contributor

Thanks,

I'm trying to POST to:

const string destination = "{0}/suite-api/api/alerts/query";                 //?page={1}&pageSize={2}";  //I've tried with and without these query parameters

string queryStr = string.Format(destination, _baseUrl);                         //, page, pageSize);

var httpWebRequest = GetWebRequest(queryStr, "POST", _vToken);

/*token is acquired earlier in the program, and as mentioned, sometimes works, it's just this query that fails more often than it succeeds

*/

Then i encode that JSON blob I sent earlier as UTF-8

{

"compositeOperator ": "AND ",

"resource-query": {

"adapterKind": ["VMWARE"],

"resourceKind": ["HostSystem",

"Datastore"

],

"resourceId": idArray

},

"startTimeRange": {

"startTime": unixEPoch,

"endTime": unixEPochEnd

}

}

I just tried it again, and I'm requesting less than 24 hours of data over 3 resourceids,  if I change the endTime value to an hour earlier (before an active alert I can see in the webUI/dashboard) i get a 200 response with an empty alert array.  Move the endTime value ahead an hour (including my alert) and the query times out.

Do i need a certain role from my sysadmin?  This api works, sometimes I've gotten as many as 17 alerts from time to time, however but it's more likely to fail when there is an alert.

Thanks for taking a look

0 Kudos
KabirAli82
Hot Shot
Hot Shot

Can you show me what the values are for

"startTime": unixEPoch,

"endTime": unixEPochEnd

I might have an idea where you went wrong... at least thats where I went wrong just now testing this.


Was I helpful? Give a kudo for appreciation!
Braindumping @ http://kablog.nl/
Tweeting @ https://twitter.com/_Kabir_Ali_
0 Kudos
dhyedrated
Contributor
Contributor

I've fiddled with them a bit but something like:

startTime: 1597227600000

endTme: 1597241347535

Hmm that example is only 3 hours, but it still shows the issue

0 Kudos
KabirAli82
Hot Shot
Hot Shot

Ok thats from Wednesday 12 August 2020 10:20:00 until Wednesday 12 August 2020 14:09:07.535 right?

Im posting this:

{

    "resource-query": {

       "resourceKind": ["virtualmachine"]

        },

    "activeOnly": "True",

    "alertCriticality": ["CRITICAL","IMMEDIATE","WARNING","INFORMATION"],

    "startTimeRange" : {

    "startTime" : 1597227600000,

    "endTime" : 1597241347535

  }

}

And it returns me 1 hit:

{

    "pageInfo": {

        "totalCount": 1,

        "page": 0,

        "pageSize": 1000

    },


Was I helpful? Give a kudo for appreciation!
Braindumping @ http://kablog.nl/
Tweeting @ https://twitter.com/_Kabir_Ali_
0 Kudos
dhyedrated
Contributor
Contributor

Is there a public facing server I can test my code against?  Obviously our system will have different results, but the sysadmins tell me there's a load balancer doing some redirection so I wonder if that's causing some issues.

Also, I see you have "activeOnly": "True", I think I saw that in the Postman example here(Postman Client Collection for vRealize Operations REST APIs - Samples - VMware {code} ) however the api docs  have:

activeOnlyFilter to return only Active alerts, if true query will exclude Suspended and Cancelled Alertsxs:boolean

Isn't JSON boolean syntax lower case with no quotes?  i.e "activeOnly" : true

I wonder if that's causing my problem.

Finally, what roles does your user have?  I'm using a ReadOnly account, but this is is a Read Only POST so I assume it should still work.

0 Kudos
KabirAli82
Hot Shot
Hot Shot

There is no public server to test on as far as I know. Might be that the LB is messing things up.

Strictly speaking the quotes are not needed, but I ones ran into a wierd problem and from that moment on I use quotes even for a single value. The documentation states;

  },
 
"activeOnly" : true,
 
"alertTypeSubtype" : [ ],
 
"alertCriticality" : [ "CRITICAL", "IMMEDIATE", "WARNING", "INFORMATION" ],
 
"alertStatus" : [ ],
 
"alertImpact" : [ ],
 
"alertControlState" : [ ],
 
"startTimeRange" : {
  
"startTime" : 1589278447,
  
"endTime" : 1589288447
 
},

So you are right on that, but with " " it also works.

I use this account to do more than just API calls so its set a bit wider. But be sure to select Administration -> Rest APIs. And I think it wouldn't hurt to select Alerts as well.


Was I helpful? Give a kudo for appreciation!
Braindumping @ http://kablog.nl/
Tweeting @ https://twitter.com/_Kabir_Ali_
0 Kudos
dhyedrated
Contributor
Contributor

Are there any whitespace requirements with the JSON?  Turning  on special characters, I see I have tabs and windows line endings throughout.  However i've also tried putting the entire JSON on one line.

I've removed all the boolean keys from my blob just to prevent confusion, but now I'm getting 400 Bad request errors

0 Kudos
dhyedrated
Contributor
Contributor

Any other tips I can pass on to my sysadmin?  I can't get my POST queries to work in Insomnia (program similar to Postman, but less restrictive with certificates)  however he confirms I have

REST-API/Read Access to APIs

REST-API/Read Access to metering API

The spec for queryAlerts is pretty loose but is there any key that's required or makes the search much faster?

Thanks again

0 Kudos
dhyedrated
Contributor
Contributor

Hi KabirAli82

Have you had any other thoughts?  My sysadmin contact got pulled off to something else, but doesn't know of any configuration settings to tweak on his end.

It's very strange as now I can't even issue GET requests to get alerts and he says nothing has changed. 

Appreciate your previous answers, but I'm still stump, any guidance you can share will really help.

0 Kudos
KabirAli82
Hot Shot
Hot Shot

Alright so I did some testing in my homelab. I've setup permission just for the API (Administration -> REST-API) and selected all the object when configuring the account.

As for the trailing spaces;

PS C:\Windows\system32> $jsonbody = '{

    "resource-query": {

       "resourceKind": ["virtualmachine"]

        },

    "activeOnly": "True",

    "alertCriticality": ["CRITICAL","IMMEDIATE","WARNING","INFORMATION"]

}'

PS C:\Windows\system32> $izziq = Invoke-RestMethod -Method POST -Body $jsonbody -Uri ($vROPsURL+"alerts/query") -Headers $vROPSSessionHeader -ContentType $Type

PS C:\Windows\system32> $izziq.alerts.count

483

PS C:\Windows\system32> $jsonbody = '{

    "resource-query": {

       "resourceKind": ["virtualmachine"             ]

        },

    "activeOnly": "True",

    "alertCriticality": ["CRITICAL","IMMEDIATE","WARNING","INFORMATION"]

}'

PS C:\Windows\system32> $izziq = Invoke-RestMethod -Method POST -Body $jsonbody -Uri ($vROPsURL+"alerts/query") -Headers $vROPSSessionHeader -ContentType $Type

PS C:\Windows\system32> $izziq.alerts.count

483

PS C:\Windows\system32> $jsonbody = '{

    "resource-query": {

       "resourceKind": ["virtualmachine"             ]

        },

    "activeOnly": "True",

    "alertCriticality": ["CRITICAL","IMMEDIATE","WARNING          ","INFORMATION"]

}'

PS C:\Windows\system32> $izziq = Invoke-RestMethod -Method POST -Body $jsonbody -Uri ($vROPsURL+"alerts/query") -Headers $vROPSSessionHeader -ContentType $Type

Invoke-RestMethod : The remote server returned an error: (400) Bad Request.

At line:1 char:10

+ $izziq = Invoke-RestMethod -Method POST -Body $jsonbody -Uri ($vROPsU ...

+          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException

    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

So as long as the value is without spacing you are good.

Also CAPS are no problem;

PS C:\Windows\system32> $jsonbody = '{

    "resource-query": {

       "resourceKind": ["VIRTUALMACHINE"]

        },

    "activeOnly": "True",

    "alertCriticality": ["CRITICAL","IMMEDIATE","WARNING","INFORMATION"]

}'

PS C:\Windows\system32> $izziq = Invoke-RestMethod -Method POST -Body $jsonbody -Uri ($vROPsURL+"alerts/query") -Headers $vROPSSessionHeader -ContentType $Type

PS C:\Windows\system32> $izziq.alerts.count

486

---------------

Are you now testing without the LB? Did you also try Postman with the vROPs collection? Postman Client Collection for vRealize Operations REST APIs - Samples - VMware {code}


Was I helpful? Give a kudo for appreciation!
Braindumping @ http://kablog.nl/
Tweeting @ https://twitter.com/_Kabir_Ali_
0 Kudos