My answer is no longer correct, even though, at the time of posting, it was. Look to some of the other answers (particularly the bounty winner) for more up to date information.
Most important rule: when a chunk is unloaded, so are the mobs (and all other items) on it, although passive mobs won't be removed.
There is also a limit on the number of mobs that can be loaded at one time: 200 hostile and 15 neutral mobs (I don't know if there's a limit on passive mobs). Also, hostile mobs are removed if they leave the 9x9 chunks that they're allowed to spawn in.
Running away will usually allow you to stay out of the reach of hostile mobs, since a chunk needs to be loaded a certain amount of time before mobs will start to spawn, but the chunks immediately behind you could have the max allowable mobs, making it dangerous at best to turn around.
Take a look at the wiki for more information.