Is it for a faster jungle clear? Because a certain buff camp doesn't not do as much for them when jungling? For lvl 2 ganks? Or any other reason I'm missing?
Usually champions that rely heavily on their mana or abilities to jungle get Blue first, so that they can use abilities without worrying about mana consumption or to lessen their coolodwns by 20%. Also, the Blue buff gives enough xp to go from level 1 to 2 quickly.
Champions that gank really early, such as Shaco, take Red buff first to slow down the enemy and deal extra damage. Another reason for starting at red, is that some champions do not require mana for their abilities, for example Lee Sin and Shyvana.
If you want to pull off a level 2 gank with any chance of success you need the red buff. If done properly you can gank lane at level 2 while enemy is still level 1, meaning better base stats, more abilities and a slow from red buff that can be applied over and over again till they die.
Blue buff is the typical jungle start if you aren't going to level 2 gank.
Blue buff starts are for champions that are either mana dependent or cannot gank effectively at low levels to warrant starting at red. Some champions also start blue in order to steal the enemy jungle's red buff due to a faster jungle speed. Popular champions that do this are Shyvana and Udyr.
Red buff starts are for champions that have high gank potential and/or do not utilize blue buff effectively. Oftentimes, a red buff jungler will gank at lvl 2, or go pass blue buff to their AP mid early.
Blue Buff: This unit regens 25 flat mana regeneration per 5 seconds and 0.5% of their maximum mana (or energy) per second and has 20% cooldown reduction on their abilities. Source
Red Buff: This unit's physical attacks apply a debuff that slows the target's movement speed by 8 / 16 / 24% for 3 seconds and a DoT that deals 10-50 (10 at levels 1/2, 15 at 3/4, 20 at 5/6, etc) true damage twice. Source
It's common to see Lee Sin (and other energy junglers) start at red buff. This gives him a very strong gank early in the game. He's not mana-dependent, so he doesn't need blue buff to sustain him through the jungle.
Champions like Gangplank or Amumu are very mana dependent. In order for them to initially clear the jungle AND have the mana to gank, they require blue buff.
These routes are not set in stone. If a jungling Gangplank knows the enemy team expects him to start blue, he may start at red instead. Likewise, Lee Sin may start blue for the extra experience and CDR. By varying from expected routes, the enemy team can be confused or caught off guard.
tl;dr: Junglers dependent on mana start blue, those who want to gank early start red. Some champions vary from their expected start to confuse the enemy.
Well the most obvious reason why a jungler don't start at blue is : they do not need it. Some champions are really mana dependent for their jungle, think Amumu, Maokai. Some aren't like Shaco. There are other champions like Tryndamere who don't have mana at all.
If you can afford not taking blue, red is a real advantage for early gank for the slow it provides.