Refresh Page to see sync status never goes away even though the sync has finished

[sbeaver]

Good day all,

I have a 3 three node vIDM cluster that has green lights across the board but the syncs never show up as completed in the UI although the logs seem to indicate that the sync was finished.  On one of the directories I see the sync button momentarily and then it switches back to "Refresh Page to see sync status" and the other directory never changes from the Refresh Page link and message.  Anyone seen anything like this?

Thanks

Steve

[David1Black]

Hi Steve, we are seeing the same issue over the last couple weeks. I've spoken to VMware support and they believe that it has to do with the RabbitMQ server/cluster.  But so far they don't have a fix for it. I have to go thru a series of reboots.

We're running 2.8.1, what version are you on?

David

[sbeaver]

Hey David,  Mine is 3.0.0.  Just curious, is your setup running in a cluster?  I have a three node cluster and the interesting thing is this was working when this was running as a single node.  I also have a support case open.  I have the value in the database that should be referenced for that and the value in the DB is false which should indicate the sync is over.  Kinda interesting.  I will post the resolution of the support case and ask if you would also please.

[atoerper]

I see the random buggy issues when running vIDM in a cluster have not gone away in version 3. We abandoned the cluster setup months ago and couldn't be happier.

[David1Black]

Yes, we have a 3 node cluster in 2 data centers; so 6 in total. Things were running great up until a couple weeks ago. No idea what has changed that would cause this.  VMware Tech Support is pointing to RabbitMQ being the culprit, but as they've pointed out to me that service was gone in 2.9.  So if you're still having the issues then that blows that theory out of the water.  I have a working session with them today and I hope this gets solved.

We need 3 appliances on each side as we have almost 40K users and 1 appliance won't handle the load.  Plus we're adding more people every week.

David

[sbeaver]

I have the same deal.  Large number of users  I will update anything new that comes from my SR

[atoerper]

We are at 50K synced users and have moved from 3 nodes with SQL to one appliance with Postgress DB with a lot of compute assigned to the VM. It has never performed better. Our concurrent user count is relatively low though so this probably a much different scenario than yours.

[David1Black]

Just got off the phone with development.  They identified an issue with our second datacenter not being set as "read-only" so the Calculator service, which syncs things in the database, was pulling a lot of data across the WAN. On each appliance in the secondary site we changed the following file:

/usr/local/horizon/conf/runtime-config.properties

At the end of this file we added the line: 

read.only.service=true

After that we did a restart of the horizon-workspace service on all 3 appliances and the sync completed.  We were able to do a full manual sync which took about 10 minutes.  I'm hoping the automatic sync tonight also works, and I think we'll be safe to say the fix was correct.