You're not the only one. Something is blocking inbound connections from the host to any guest on a bridged network after some amount of time. If you initiate the connection from the guest, it all starts up again. I've taken to running "ping <gateway>" from inside my guest(s) which seems to keep the network up for inbound connections.
