Notice: This post only applies to games up to HeartGold/SoulSilver. The EV system has changed since Black/White. For more details, see DragonLord's answer.
Technically, every 4 EVs gained will result in 1 extra stat at level 100. For example, if you took two level 100 Snorlax, with identical natures and IVs, but one had 4 EVs in Attack and the other had 0 EVs in Attack, the former would have 1 more point in its Attack stat than the latter.
The EV-effected stats gained during level-up don't all happen all at once. DisgruntledGoat described what occurs in a good way:
On level up, the game will calculate how much of a stat point increase you would have at that level and adds it on. For example, if you have a level 50 Gyarados and you earn 200 EVs (50 stat points), you won't get +1 extra point per level up. Instead, you'll get around +25 on the first level up (because at lv50 you would have earned about half the points), then +1 for roughly every two levels after that.
Of course, if your Gyarados was level 99, had cleared EVs, and then you battled all those low level Pokémon, you would probably see a very large stat boost from 99 to 100.