If you tell your follower to wait for too long, they can tire of waiting and stop following you, returning to they were hired. Do they still keep any stuff that you gave them?
As a stealth fighter, I am wont to ask my follower to stay behind so I can ninja properly without interference by their artificial unintelligence. Unfortunately, there is no easy way to prevent my follower from waiting for long periods. How dare they tire of lugging my crap!