I used to have the same problem. Tried everything, but never really solved the problem. Later I moved from USB connecton to ethernet connection.
Since then that issue never happend again.