I vaguely remember I've seen somewhere that SRM plug-in requires vRO to be configured with vCenter Single Sign-On authentication.
Not sure if it is indeed a requirement and I don't have a suitable environment to test it, but if it is true, it could explain why you are having issues with the plug-in, as the vRO embedded in vRA is configured with vRA/vIDM authentication.
It might be worth the efforts to setup a standalone vRO, configure its authentication to be vCenter Single Sign-On, and verify if the plug-in works there.
Thank you for your reply
Tested out with standalone vRO where in the config I've set the auth method to be vSphere and I can login with my SSO credentials.
Still the same issue where I didn't get anything under my SRM inventory even though the workflow runs successfully.