There's three things you want in a split push champion.
The first is the ability to take down towers quickly. You only have so much time before the enemy team figures out you're there, and you need to make the most of it. Similarly, you want to be able to clear out minion waves quickly, though this is a smaller issue as a lot more champions can do this easily.
The second is dueling capability. Often times the enemy team might send only one or two champions to stop your push while the rest try to continue what they're doing. Certain champions excel at winning 1v1's and even some 1v2's, and if you're playing one of those, you can just kill them and continue pushing.
The third quality is escape ability. Sooner or later the entire enemy team is going to come after you, and at that point it'd be a great idea to try and get out alive.
A very good (and very cheap) split pusher is Master Yi. His abilities include a massive AD boost, a massive AS boost, an incredible movement boost, and even resistance to slows. This is literally everything a split pusher could ever ask for, as you'll be able to murder towers and lone enemy champions, and he's nearly impossible to catch.
However, there's three other things you should do to increase the chances of succeeding with your split push.
The first is to choose your spots well. If the enemy team is currently teamfighting your allies in the bottom lane, by all means, go take out a couple top lane towers. If three of them are dead and you have vision on the other two, again, push to your hearts desire. If they're all live and none of them are visible? That's a bad time to push.
The second is to ward well. There's this nifty 75 gold item known as a sight ward, that lets you see a small area of the map. Plant these things at choke points around the area you plan on pushing, and you'll know when the enemy team is coming for you. This makes escaping much easier, and is likely to save your life several times through a match. (They're also very important for normal laning and teamfighting, but that's another issue.)
The third and final thing is the teleport summoner spell. If you've got a huge minion wave pushing top but you're currently back at base, teleport there and you'll get the tower down before they know what hit them. On the contrary, if you've just taken down an inhibitor and you know they're coming for you (thanks to those wards you placed earlier), you can use it like a much faster recall to get out of there... possibly to another lane you can push down.
On the flip side, keep in mind that teleport is also an amazing way to stop a split push as well. If you see three enemy champions with teleport, you might be better off just trying to teamfight. Similarly, if they have abilities that can effectively emulate a teleport (Twisted Fate, Nocturne, or Pantheon's ultimates), you need to be a lot more careful, as they can appear at any time and try to kill you.
One last thing, although split pushing is a valid and extremely effective tactic, keep in mind that there are times where it's just better to teamfight. If the enemy team's main push would kill your nexus before your backdoor can destroy theirs, you need to go back and help out your team. This can also apply to fights around Baron, as winning that buff is often enough to seal the game. Then again, in the EU vs NA all star game, a split push during a baron fight took out everything in NA's base short of the Nexus itself, so use your own judgement.
Good luck, and happy split pushing!