ToDo queue

[xian_]

I need to collect deferred activities in a queue and try to perform them until a defined timeout. The usecase is to connect to a REST API and make some calls but retry for a while (few days) if the connection is unavailable.

My idea: try to make the call as usual, and in case of error put the call input into a queue. Run a scheduled vRO task and pick the queue elements, retry until success or predefined aging exceeded. My question is how would you implement this queue? In a DB? In resource elements? Message queue? Any recommendations appreciated.

[eoinbyrne]

The ResourceElement is the simplest one and it has no external dependencies. The only thing to be careful of would be how many entries. I've stored ~100,000 generated MAC addresses in a JSON formatted text element with no issues previously. We used this approach to build a MAC address assignment pool for server provisioning with an integration to DHCP during server provisioning for a client. Depending on the amount of data each retry record requires this could work nicely (just make sure you use workflow Locking for all reads & writes!)

Database would obviously be fine and would permit large amounts of data (say Request & Response attempt logs for each retry) as well as the retry table your primary requirement indicates. The external system requirement is the only piece to consider but it might be warranted if you have the need.

I've never used the MessageQueue plugin but my understanding is that it is trigger/event based which might make the delay period part of this difficult. Even if you went this way and just put a message back on the queue if it's retry date was not met your system would just spend all it's cycles firing messages to itself (if I understand how this works correctly at least)

[GeiAll]

Hi xian

We are using a SQL database for the same reason.
Queuing if any system is down like:
RestAPI:  ServiceNow, VAPI, PHPIPAM and vRA.
XaaS workflows started from vRA (eg: it might fail because of lack of disk space etc)

If you have a little SQL experience it's not so hard to set up, and you can add transaction or locking to avoid problems if you have multiple workflows running at the same time.

Be aware of the scheduled task will run under the scheduled user and not the original user starting the workflow though, but you can save the username and trick this if needed.

I would avoid the RabbitMQ (MessageQueue) if you are using it from vRA (we tried that, with did not go so well). It might work better for a standalone RabbitMQ or another message broker.

I never tried to store in the the ResourceElement, but that should work aswell.