I disagree that it should be the largest minions first really. Only in the early game, where your defence is very little, and red/blue can deal a lot of damage to you, should you probably aim for the bigger one first, however, depending on your champion, if you're using a champion that is very good at, or designed for jungling, then you could probably survive the damage that the big minion does to you, whilst you lessen the total damage by attacking the others.
Assuming you won't get ganked, and your champion isn't squishy, or has a sustain, it's almost always better to take the little ones first, because if you want to get the red or blue buff, you don't want to waste it's precious time, by fighting the minions which you don't need. You'll be able to utilize the maximum time you have the buff for, if the lesser minion are already gone.
Even when facing super minions in the lanes, getting the lesser minions means more money and exp, then you can focus on the big one, otherwise your minions would kill the majority of the lesser minions, thus wasting all that nice gold and exp.
From mid game and onward, unless you think someone else will try steal your buff, I highly recommend that 99% of the time you attack the lesser minions first. Especially with champions like Warwick and Xin Zhao, who have sustain abilities.