Yeah that's weird... you start and end a ssh session it's logged so we can trust the connection close well.
Have you tried to set a timeout for ssh sessions ? (even though the starting/ending of your ssh sessions are logged so it means they close well, we can verify if a timeout change anything.)