We used to experience this. What we found was that the session was typically hung logging off in the VDI while the connection brokers thought it was still active. The user would try to reconnect and the connection server would try to connect the user but since it was still logging off they could not connect.