Please review the Firewall Rules with a Custom Layer 3 Protocol section of the NSX Administration Guide that may assist on resolving this issue - https://docs.vmware.com/en/VMware-NSX-for-vSphere/6.3/com.vmware.nsx.admin.doc/GUID-293B8FB3-8261-4867-844F-370BE264C39B.html
Suspecting the TCP timeout mismatch between the Server the Firewall can you check what is the tcp timeout set in the Linux server and mach that exactly to the ESG and see how the behavior is.
you can refer to the below KB article for getting the TCP timeout value and to set TCP timeout value.
vCNS/NSX Edge Firewall TCP Timeout Values (2101275)
I would recommend you to modifies it on Server and match to ESG and see. give a try ! :
Do you see this issue only after you have changed the routing from static to dynamic (OSPF) or you also have this issue when you were on static routing?
If it's only on OSPF, check if the OSPF is dropped at the same time your SSH is dropped.
You can try debug OSPF packet from Edge for example.
I have seen some dynamic routing issues in some firewalls, I don't have much experience with pfSense tho.
On the vDS side, how many vmnics do you have and what kind of load balancing policy do you have?
Not sure if this is acceptable in your environment, but if you have multiple vmnics you can try removing/disconnecting one of the vmnic from the vDS dvUplink to eliminate vDS load balancing issueBayu Wibowo | vExpert NSX, VCIX6-DCV/NV, Cisco Champion, AWS-SAA
Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
https://nz.linkedin.com/in/bayupw | twitter @bayupw