VMware Cloud Community
qc4vmware
Virtuoso
Virtuoso
‎12-14-2017 11:47 AM

com.vra.reservations.getReservationsForUserAndComponent throwing 404 error when more than 12 subTenants vRA/vRO 7.3

I'm trying to run the com.vra.reservations.getReservationsForUserAndComponent action and it seems to blow up when I give it more than 12 subTenants.  I don't know if it has to do with the size of the filter that gets created or something along those lines.  I'm currently getting around this by creating my own action and looping through each subTenant and doing a single query.  I haven't tried testing this directly against the vRA api but if someone could investigate this it would be great. We have 28 subTenants in our environment that our service account has access to so this will always explode on us out of the box and I'd think the same in most larger deployments.

3 Replies
jasnyder
Hot Shot
Hot Shot
‎12-14-2017 03:30 PM

It appears to be a bug with the OData request object or the REST client object underneath in the Java implementation.  I think it has to do with the length of the OData filter query string, and the limit just happens to be between 12 and 13 subtenants.  12 works; 13 doesn't.  It blows up at this call:

reservations = reservationService.getAllReservations(reservationFilter).getContent();

First, I verified that an independent call to the REST API with the same query this generates will work.  I dumped the query string that it builds with the query object, URL encoded it and tested it both with an independent request to the reservation-service/api/reservations URI and also with the vcacHost.getReservationClient.get(URI) method in vRO.  Both worked with the extended URL encoded query string.

The non-encoded query string was:

$filter=(subTenantId eq '9c239473-1f78-4d41-af78-4f1b5d87fe2e' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere') or (subTenantId eq 'fe1a9d55-5f7e-45a8-8808-3a9bc5ba4113' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere') or (subTenantId eq '8cea0f65-1a74-44f6-b3d0-881a6b4138ec' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere') or (subTenantId eq 'be5ee67e-ce0b-4024-9c47-c68a338cb164' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere') or (subTenantId eq '1d9e7acd-c3c1-44aa-9b4f-b745cc8aaabf' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere') or (subTenantId eq '1d99ec43-c3dc-49e9-93a6-d05844793420' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere') or (subTenantId eq '0b2676c2-82e7-4fe3-ae37-ce1cbf1f46a3' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere') or (subTenantId eq '03761b44-7830-4d65-9eda-0819d3ca6fa7' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere') or (subTenantId eq '32171dd0-762a-47de-ab18-7810d9473403' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere') or (subTenantId eq '74618239-167c-4765-855d-7eaab4d527cc' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere') or (subTenantId eq '26bdd4d0-08c5-4b5b-8c4f-2a8bd10b5732' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere') or (subTenantId eq '31a74d17-a923-49b5-b221-e453465cde60' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere') or (subTenantId eq '8ee1fbcf-acf3-42a7-a7c1-4f5b47db84c7' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere') or (subTenantId eq '73b5c799-dcab-4dc1-be91-f432fff28929' and enabled eq true and reservationTypeId eq 'Infrastructure.Reservation.Virtual.vSphere')

The encoded query string I used was:

%24filter%3D(subTenantId%20eq%20%279c239473-1f78-4d41-af78-4f1b5d87fe2e%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)%20or%20(subTenantId%20eq%20%27fe1a9d55-5f7e-45a8-8808-3a9bc5ba4113%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)%20or%20(subTenantId%20eq%20%278cea0f65-1a74-44f6-b3d0-881a6b4138ec%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)%20or%20(subTenantId%20eq%20%27be5ee67e-ce0b-4024-9c47-c68a338cb164%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)%20or%20(subTenantId%20eq%20%271d9e7acd-c3c1-44aa-9b4f-b745cc8aaabf%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)%20or%20(subTenantId%20eq%20%271d99ec43-c3dc-49e9-93a6-d05844793420%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)%20or%20(subTenantId%20eq%20%270b2676c2-82e7-4fe3-ae37-ce1cbf1f46a3%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)%20or%20(subTenantId%20eq%20%2703761b44-7830-4d65-9eda-0819d3ca6fa7%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)%20or%20(subTenantId%20eq%20%2732171dd0-762a-47de-ab18-7810d9473403%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)%20or%20(subTenantId%20eq%20%2774618239-167c-4765-855d-7eaab4d527cc%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)%20or%20(subTenantId%20eq%20%2726bdd4d0-08c5-4b5b-8c4f-2a8bd10b5732%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)%20or%20(subTenantId%20eq%20%2731a74d17-a923-49b5-b221-e453465cde60%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)%20or%20(subTenantId%20eq%20%278ee1fbcf-acf3-42a7-a7c1-4f5b47db84c7%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)%20or%20(subTenantId%20eq%20%2773b5c799-dcab-4dc1-be91-f432fff28929%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27Infrastructure.Reservation.Virtual.vSphere%27)

This works direct to the API, but fails when passing it to the reservationService.getAllReservations(reservationFilter).getContent();

I found as you did that reducing the number of subtenants to 12 makes it work.  Increasing to 13 makes it break.

But I also found that if you modify the filter to have less conditions (I made it only filter on subTenantId and forgetting for the moment about enabled = true or reservationTypeId = vSphere) that more subtenants suddenly works.  The modified filter becomes:

?$filter=(subTenantId eq 'ea60e940-4c8e-498b-a92c-5c35b7f19edf') or (subTenantId eq '31a74d17-a923-49b5-b221-e453465cde60') or (subTenantId eq '32171dd0-762a-47de-ab18-7810d9473403') or (subTenantId eq '73b5c799-dcab-4dc1-be91-f432fff28929') or (subTenantId eq '1d9e7acd-c3c1-44aa-9b4f-b745cc8aaabf') or (subTenantId eq '8cea0f65-1a74-44f6-b3d0-881a6b4138ec') or (subTenantId eq '9c239473-1f78-4d41-af78-4f1b5d87fe2e') or (subTenantId eq '0b2676c2-82e7-4fe3-ae37-ce1cbf1f46a3') or (subTenantId eq '8ee1fbcf-acf3-42a7-a7c1-4f5b47db84c7') or (subTenantId eq 'be5ee67e-ce0b-4024-9c47-c68a338cb164') or (subTenantId eq '03761b44-7830-4d65-9eda-0819d3ca6fa7') or (subTenantId eq '1d99ec43-c3dc-49e9-93a6-d05844793420') or (subTenantId eq '26bdd4d0-08c5-4b5b-8c4f-2a8bd10b5732')

This leads me to believe the issue is with the length of the filter query string generated from the ODataQuery object.  Even if you shortened the way I did to make more subtenants work, it too, would eventually hit the character limit with enough subtenants.  I'm not sure where the limit is being imposed, but I am assuming somewhere in the RestClient internals of the vRA SDK or VCACCAFE plugin underpinning the ReservationServiceClient.  I doubt there is a fix for this without the devs making a fix, except to modify the action and make your own, wherein you get the query string, and tack it on the end of a generic get request using the ReservationServiceClient.get(URI) method.  That's probably more effort than it's worth.  IF you wanted to shortcut it you could just loop through the subtenants and append another block of the above query, already encoded, just switching out the subtenant id.

For example, replacing the content of the action:

var client = host.createCompositionClient();

var compositionService = client.getCompositionCompositeBlueprintService();

var blueprint = compositionService.getBlueprint(blueprintId);

//create services

var subTenantService = host.createAuthenticationClient().getAuthenticationSubtenantService();

var reservationService = host.createReservationClient().getReservationReservationService();

// get sub tenants for user

var userPrincipal = splitUser(user);

var principal = new vCACCAFEPrincipalId(userPrincipal.username, userPrincipal.domain);

subTenants = subTenantService.getSubtenantsByPrincipalAndRole(tenant, principal, null, true, new vCACCAFEPageOdataRequest(1 , 2147483647));

// get reservation filtered by sub tenants

//var reservationFilter = getReservationFilterForSubTenants(subTenants);

var reservations = [];

var resClient = host.createReservationClient();

reservations = resClient.get("/reservations" + getQueryString(subTenants)).getBodyAsJson().content;

for(var r in reservations) {

    System.log(JSON.stringify(reservations[r]));

}

return reservations;

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

function splitUser(userParam) {

    var indexOfAt = userParam.lastIndexOf("@");

    if(indexOfAt > 0) {

        return {

            username: userParam.substring(0, userParam.lastIndexOf("@")),

            domain: userParam.substring(userParam.lastIndexOf("@") +1)

        }

    }

}

function getQueryString(subTenants) {

    var queryString = "?page=1&limit=2147483647&%24filter%3D";

    for(var s in subTenants) {

        queryString += "(subTenantId%20eq%20%27" + subTenants[s].getId() + "%27%20and%20enabled%20eq%20true%20and%20reservationTypeId%20eq%20%27" + System.getModule("com.vmware.vra.reservations").getReservationTypeIdForComponent(host, blueprintId, componentId) + "%27)";

   

         if(s != subTenants.length - 1) queryString += "%20or%20";

    }

   

    return queryString;

}

The output of this looks like (only showing one, for example's sake):

[2017-12-14 18:24:03.070] [I] {"@type":"Reservation","createdDate":"2017-02-01T04:26:33.380Z","lastUpdated":"2017-02-02T02:33:18.250Z","version":"1","id":"fad9e070-11be-450e-b731-76b623797072","name":"MTD-BG-1--2","reservationTypeId":"Infrastructure.Reservation.Virtual.vSphere","tenantId":"MTD","subTenantId":"8ee1fbcf-acf3-42a7-a7c1-4f5b47db84c7","enabled":"true","priority":"1","reservationPolicyId":null,"alertPolicy":{"enabled":"false","frequencyReminder":"0","emailBgMgr":"true","recipients":[],"alerts":[{"alertPercentLevel":"80","referenceResourceId":"storage"},{"alertPercentLevel":"80","referenceResourceId":"memory"},{"alertPercentLevel":"80","referenceResourceId":"cpu"},{"alertPercentLevel":"80","referenceResourceId":"machine"}]},"extensionData":{"entries":[{"key":"reservationMemory","value":{"type":"complex","componentTypeId":null,"componentId":null,"classId":null,"typeFilter":null,"values":{"entries":[{"key":"computeResourceMemoryTotalSizeMB","value":{"type":"integer","value":"393216"}},{"key":"computeResourceMemoryReservedTotalSizeMB","value":{"type":"integer","value":"204800"}},{"key":"memoryReservedSizeMb","value":{"type":"integer","value":"204800"}},{"key":"computeResourceMemoryAllocatedTotalSizeMB","value":{"type":"integer","value":"8192"}}]}}},{"key":"reservationStorages","value":{"type":"multiple","elementTypeId":"COMPLEX","items":[{"type":"complex","componentTypeId":null,"componentId":null,"classId":null,"typeFilter":null,"values":{"entries":[{"key":"permissible_value_selected","value":{"type":"boolean","value":"true"}},{"key":"storageReservedSizeGB","value":{"type":"integer","value":"2000"}},{"key":"storageReservationPriority","value":{"type":"integer","value":"1"}},{"key":"storageAllocatedSizeGB","value":{"type":"integer","value":"0"}},{"key":"computeResourceStorageTotalSizeGB","value":{"type":"integer","value":"3934"}},{"key":"isStorageCluster","value":{"type":"boolean","value":"false"}},{"key":"storagePath","value":{"type":"entityRef","componentId":null,"classId":"Storage","id":"34eb11ea-11be-4744-ad2e-1d44c81f458d","label":"NFS-01"}},{"key":"storageEnabled","value":{"type":"boolean","value":"true"}},{"key":"computeResourceStorageReservedSizeGB","value":{"type":"integer","value":"2000"}},{"key":"isSDRSEnabled","value":{"type":"boolean","value":"false"}},{"key":"computeResourceStorageFreeSizeGB","value":{"type":"integer","value":"2265"}},{"key":"diskCost","value":{"type":"integer","value":"0"}}]}}]}},{"key":"machineQuota","value":{"type":"integer","value":"0"}},{"key":"computeResource","value":{"type":"entityRef","componentId":null,"classId":"ComputeResource","id":"fb0fc0e3-578d-4aa5-a585-89d39de4a1cb","label":"-02 (vCenter)"}},{"key":"reservationNetworks","value":{"type":"multiple","elementTypeId":"COMPLEX","items":[{"type":"complex","componentTypeId":"com.vmware.csp.iaas.blueprint.service","componentId":null,"classId":"Infrastructure.Reservation.Network","typeFilter":null,"values":{"entries":[{"key":"networkPath","value":{"type":"entityRef","componentId":null,"classId":"Network","id":"fbf75746-6952-45a6-86dd-4b65aea12d0b","label":"VMPublic"}},{"key":"networkProfile","value":{"type":"entityRef","componentId":null,"classId":"networkProfile","id":"0187070e-4798-483c-90f2-787589377fdb","label":"VMPublic"}}]}}]}}]}}

Not sure if that approach is appealing to you or not.  The resultant object type may or may not map directly to how the old action was returning objects, so I had it output the JSON string representation of what is returned for each reservation so you could see if this would work for you.

qc4vmware
Virtuoso
Virtuoso
‎12-14-2017 04:23 PM

Very thorough testing!  I just didn't have time in my day to get as detailed!  I did end up just looping through the subTenants as my bandaid.  Hopefully the VMware team will put this in as a bug and comment on this thread to let us know they did.  I'll mark your answer as correct once I know this has been addressed.

Thanks!

0 Kudos
jasnyder
Hot Shot
Hot Shot
‎12-14-2017 05:34 PM

So I went back and I think my solution still doesn't work.  It produces a query string that should work, but it doesn't.  I didn't look closely enough at the results, but it was returning more than it should.  However, if I take the same query string out and put it straight to the API, it does work.  The reservation client seems to be throwing away the filter query parameter for some reason (logs show the request not making it through to the API with the $filter parameter).

In any case, in my more in-depth analysis after the last post, I came to the conclusion that the issue is in fact on the server-side.  It looks like the query to the reservation API results in a query from the reservation service to the to the IaaS proxy service, which queries the IaaS model manager, which makes sense because reservations used to live solely in the IaaS side, but have begun being migrated into the CAFE side.  So a query to the CAFE API ends up resulting in a query to the IaaS model manager.  It seems like when the filter is too long, the model manager kicks a 404 back to the IaaS proxy service which kicks it back upstream.  I was in fact able to reproduce this with a raw call to the API without going through vRO.  And I was able to remove a filter parameter and watch it go through successfully.

So with that said, I think the only solution is the one you have at this time.

0 Kudos