Retrieving more than 100 items

[pizzle85]

Im attempting to fetch a list of objects from vRA where there might be more than 100 objects returned. Im executing an OdataRequest but its still limited to 100 return results. Is this expected or should this allow me to return more than 100 items?

service = cafeHost.createCatalogClient().getCatalogConsumerResourceService();
var filter = new Array();
filter[0] = vCACCAFEFilterParam.substringOf("organization/subTenant/id", vCACCAFEFilterParam.string(bg.id));
var query = vCACCAFEOdataQuery.query().addFilter(filter);
var odataRequest = new vCACCAFEPageOdataRequest(1, 10000, query);
resources = service.getResourcesList(odataRequest);

Request was denied due to exceeded resource size limit. The maximum number of resources allowed is 100.

If this "work around" does not allow me to return more than 100 objects in a single query does anyone have any usable examples of iterating through pages and concatenating the results into a large array?

[iiliev]

Hi,

Could you check if the following code works for you (it fetches objects in chunks of 100 items and concatenates them into the array named resources)

const PAGE_LIMIT = 100;
var resourceService = vcacCafeHost.createCatalogClient().getCatalogConsumerResourceService();
var filters = [];
filters[0] = vCACCAFEFilterParam.substringOf("organization/subTenant/id", vCACCAFEFilterParam.string(bg.id));
var query = vCACCAFEOdataQuery.query();
query.addFilter(filters);
query.setTop(PAGE_LIMIT);
query.setSkip(0);
var oDataRequest = new vCACCAFEPageOdataRequest(query);
var resources = [];
do {
   var resourcesList = resourceService.getResourcesList(oDataRequest);
   resources = resources.concat(resourcesList);
   System.log("Found Resources: " + resourcesList.length);
   query.setSkip(resources.length);
   oDataRequest = new vCACCAFEPageOdataRequest(query);
} while (resourcesList.length > 0);
System.log("Number of resources: " + resources.length);

[qc4vmware]

I'm not sure if this is a bug or expected behavior but on large data sets I noticed I end up with duplicates getting returned unless I add an ordering to the query with something like this:

query.addAscOrderBy(["name"]);

for example when I am querying catalog resources.  Is this maybe because order is not guaranteed?  Its strange because if say I have a query to pull all of the virtual machine resources for a given subtenant where there are 2500 vm's i'll get back 2500 results but around 25% end up being duplicates and for as many duplicates I'm missing some systems.  When I pull them in some order I get all expected 2500 unique items.

Anyway iiliev​ can test this?

[iiliev]

Returning duplicates sounds like a bug to me, which makes this API pretty much unusable if you want to enumerate all items without ordering. Looks like when there is no order specified explicitly, the API directly returns the items as returned by the database query, which almost certainly will include duplicates.

It would have been more deterministic if there was some default ordering (ie. by object ID) to be used if there is no user-provided ordering, but probably CAFE plug-in doesn't have such.

[stevedrummond]

You have to explicitly add your ordering criteria or paging will not work in vRA 7.x.

I raised this internally and was told "its not possible to order by ID as a default with postgres"; this is of course, complete nonsense.

This is the default behaviour of the API and not a vRO plug-in issue.