It's been a while since we set this up and have only been using the App where the steps are:

User types in User name

User generates a OTP on the mobile App

User enters OTP and selects OK

User Enters AD password

User is successfully logged in

This is all using the setting 'radius-auth' on it's own (I believe the RADIUS server caches the AD credentials) and it works fine.

We also have a bunch of tokens/user on the same solution that do not use the app, they use the SMS method explained previously. Will these co-exist? I don't see how from your explanation that UAG/RADIUS will know that in the user doesn't have a passcode yet and will fallback to the AD password in the first instance. We haven't been able to get the SMS method working yet, nothing seems to happen,