VMware Cloud Community
JFCooleCC
Contributor
Contributor
Jump to solution

How to query events with constraints + limit parameter via REST API?

I'm making a fairly simple workflow in vRO that will simply take an input of the 'context' field, connect via the REST API, and query for all events that have that value in the context field, to output the logs associated with provisioning a system in vRA. The idea being I can either email the results on a failure, publish to a dashboard, whatever.

I'm using VLI 4.0.0 currently, and I'm referencing the documentation at Log Insight API documentation

I can easily do the GET query to get events and set the limit parameter to whatever I want, e.g :

GET /api/v1/events?limit=1000&timeout=15000

But, when I go add the constraints that I mentioned above, and attempt to add the limit argument, it's throwing errors.

https://[host]/api/v1/events/context/CONTAINS%20[context value]/product/vra/product/vro/timestamp/%3E0

This works, but only returns 100 entries, and I know there are more. If I try to do either of these whether URL encoded or not, it errors out:

https://[host]/api/v1/events/context/CONTAINS%20[context value]/product/vra/product/vro/timestamp/%3E0?limit=1000

{"errorMessage":"invalid_constraints: timestamp GT [0?limit=1000]"}

https://[host]/api/v1/events/context/CONTAINS%20[context value]/product/vra/product/vro/timestamp/%3E0/?limit=1000

{"errorMessage":"missing_argument: ?limit=1000"}

https://[host]/api/v1/events?limit=1000/context/CONTAINS%20[context value]/product/vra/product/vro/t...

{"errorMessage":"Handler not found for request GET /api/v1/events%3Flimit%3D1000/context/CONTAINS%20SaU96w79/product/vra/product/vro/timestamp/%3E0"}

Is anyone aware of if it is possible to query with constraints and the limit argument in a single go?

I would hate to have to query a pile of stuff and then parse it by the context field afterward.

Tags (1)
1 Solution

Accepted Solutions
admin
Immortal
Immortal
Jump to solution

You have incorrect URL encoding. The characters / and ? have meaning in urls. You need to URL-encode them in field names, operators and values, not in path or query delimiters. The urls in your post are not as they appear, but we can guess what the real urls were by the error messages.

Following tests are done using the apiclient.py inside a Log Insight virtual appliance, connecting to localhost, and logging in as the local "admin" user.

# /usr/lib/loginsight/application/lib/python/apiclient.py -a localhost -u admin -i

Combining constraints in the /path/component with the limit query (?) parameters works like this:

>>> get("/events/text/CONTAINS%20opid?limit=5")

{u'duration': 1134, u'complete': True, u'events': [{u'text’: ...

This is a valid query over the whole time range, and I have a lot of data so it hits the default 30-second timeout before finding anything from VRA/VRO:

>>> get("/events/text/CONTAINS%20opid/product/vra/product/vro/timestamp/%3E0/?limit=5")

{u'duration': 30000, u'warnings': [{u'progress': 0.65040056903264443, u'id': 128, u'details': u'Request timeout expired'}], u'complete': False, u'events': []}

The following are incorrect URLs demonstrating various failure modes.

URL-encoding the ? makes it part of the field value; numeric as in your example, which produces invalid_constraints because “5?limit=1000” isn’t a number. This matches the first error you reported:

>>> get("/events/numberfield/%3E5%3Flimit=1000")

APIException: APIException Status: 400 Message: {"errorMessage":"invalid_constraints: numberfield GT [5?limit=1000]"}

If you did the same thing with a text field you would see different behavior, because "opid?limit=1000" is a valid text field value. This is a syntactically valid search for the “text” field containing “opid?limit=1000” - it doesn’t, so this is an empty result set.

>>> get("/events/text/CONTAINS%20opid%3Flimit=1000")

{u'duration': 520, u'complete': True, u'events': []}

The second error you observed, "missing_argument", is caused by a url-encoded question mark immediately after a non-urlencoded slash. You’re declaring a constraint (/field/contains value) with the field name “?limit=1000” without a comparison operator or value.

>>> get("/events/numberfield/%3E5/%3Flimit=1000")

APIException: APIException Status: 400 Message: {"errorMessage":"missing_argument: ?limit=1000"}

If you did include a value, you would get an invalid_constraints failure because "?limit=1000" is not a valid field name.

>>> get("/events/numberfield/%3E5/%3Flimit=1000/CONTAINS%20value")

APIException: APIException Status: 400 Message: {"errorMessage":"invalid_constraints: ?limit=1000 CONTAINS [value]"}

For the handler-not-found case, you url-encoded the question mark and appended it to the “events” url base directly. The url fragment “events%3Flimit=4” is not the same as “events?limit=4”. If you replace the url-encoded question mark with any other character that’s not a slash or question mark, you’ll get the same error.

>>> get("/events%3Flimit=irrelevant/text/after/here")

APIException: APIException Status: 404 Message: {"errorMessage":"Handler not found for request GET /api/v1/events%3Flimit=irrelevant/text/after/here"}

>>> get("/events_irrelevant/text/after/here")

APIException: APIException Status: 404 Message: {"errorMessage":"Handler not found for request GET /api/v1/events_irrelevant/text/after/here"}

If you didn't url-encode the question mark, all the text following it is interpreted as part of the "limit" query parameter, with the value “5/text/contains/ opid”. Which would be rejected with "invalid_limit":

>>> get("/events/?limit=5/text/CONTAINS%20opid")

APIException: APIException Status: 400 Message: {"errorMessage":"invalid_limit: 5/text/CONTAINS opid"}

View solution in original post

3 Replies
parmarr
VMware Employee
VMware Employee
Jump to solution

Please see if you can get any information from :

https://blogs.vmware.com/management/2016/05/log-insight-3-3-query-api.html

Sincerely, Rahul Parmar VMware Support Moderator
0 Kudos
admin
Immortal
Immortal
Jump to solution

You have incorrect URL encoding. The characters / and ? have meaning in urls. You need to URL-encode them in field names, operators and values, not in path or query delimiters. The urls in your post are not as they appear, but we can guess what the real urls were by the error messages.

Following tests are done using the apiclient.py inside a Log Insight virtual appliance, connecting to localhost, and logging in as the local "admin" user.

# /usr/lib/loginsight/application/lib/python/apiclient.py -a localhost -u admin -i

Combining constraints in the /path/component with the limit query (?) parameters works like this:

>>> get("/events/text/CONTAINS%20opid?limit=5")

{u'duration': 1134, u'complete': True, u'events': [{u'text’: ...

This is a valid query over the whole time range, and I have a lot of data so it hits the default 30-second timeout before finding anything from VRA/VRO:

>>> get("/events/text/CONTAINS%20opid/product/vra/product/vro/timestamp/%3E0/?limit=5")

{u'duration': 30000, u'warnings': [{u'progress': 0.65040056903264443, u'id': 128, u'details': u'Request timeout expired'}], u'complete': False, u'events': []}

The following are incorrect URLs demonstrating various failure modes.

URL-encoding the ? makes it part of the field value; numeric as in your example, which produces invalid_constraints because “5?limit=1000” isn’t a number. This matches the first error you reported:

>>> get("/events/numberfield/%3E5%3Flimit=1000")

APIException: APIException Status: 400 Message: {"errorMessage":"invalid_constraints: numberfield GT [5?limit=1000]"}

If you did the same thing with a text field you would see different behavior, because "opid?limit=1000" is a valid text field value. This is a syntactically valid search for the “text” field containing “opid?limit=1000” - it doesn’t, so this is an empty result set.

>>> get("/events/text/CONTAINS%20opid%3Flimit=1000")

{u'duration': 520, u'complete': True, u'events': []}

The second error you observed, "missing_argument", is caused by a url-encoded question mark immediately after a non-urlencoded slash. You’re declaring a constraint (/field/contains value) with the field name “?limit=1000” without a comparison operator or value.

>>> get("/events/numberfield/%3E5/%3Flimit=1000")

APIException: APIException Status: 400 Message: {"errorMessage":"missing_argument: ?limit=1000"}

If you did include a value, you would get an invalid_constraints failure because "?limit=1000" is not a valid field name.

>>> get("/events/numberfield/%3E5/%3Flimit=1000/CONTAINS%20value")

APIException: APIException Status: 400 Message: {"errorMessage":"invalid_constraints: ?limit=1000 CONTAINS [value]"}

For the handler-not-found case, you url-encoded the question mark and appended it to the “events” url base directly. The url fragment “events%3Flimit=4” is not the same as “events?limit=4”. If you replace the url-encoded question mark with any other character that’s not a slash or question mark, you’ll get the same error.

>>> get("/events%3Flimit=irrelevant/text/after/here")

APIException: APIException Status: 404 Message: {"errorMessage":"Handler not found for request GET /api/v1/events%3Flimit=irrelevant/text/after/here"}

>>> get("/events_irrelevant/text/after/here")

APIException: APIException Status: 404 Message: {"errorMessage":"Handler not found for request GET /api/v1/events_irrelevant/text/after/here"}

If you didn't url-encode the question mark, all the text following it is interpreted as part of the "limit" query parameter, with the value “5/text/contains/ opid”. Which would be rejected with "invalid_limit":

>>> get("/events/?limit=5/text/CONTAINS%20opid")

APIException: APIException Status: 400 Message: {"errorMessage":"invalid_limit: 5/text/CONTAINS opid"}

JFCooleCC
Contributor
Contributor
Jump to solution

Thank you for the excellent reply and explanation. Based on this I realize what my error was and I adjusted the call in vRO and it's now pulling the data that I want. Thanks a ton!

0 Kudos