My feeling about Cheese is that is just as legitimate as camping in other types of games, and irritates other players for much of the same reason. I like Daryl's explanation that it incorporates luck more strongly than the more stable builds many players prefer, but I'd like to extend that it works principally by exploiting to maximum advantage the asymmetries between the opposing forces.
A particular feature of cheese is that it works well before the opponent is able to get a reasonable sized, balanced and stable army, because it exploits the compromises players have to make in the early part of their builds. By mid-game, those weaknesses will be covered by the composition of the opponents army.
For this reason, you see cheese play mostly in non mirror matchups. There are more chances for asymmetric advantages in a TvP, PvZ or ZvT matchup than in TvT, PvP, and ZvZ. That said, there are still some interesting things that can happen in mirror matchups that would typically count as cheese.
As a particular example, I'll compare TvT where the defending player is using a 1/1/1 build. A cheesy terran player might bank on the opponent making marines out of their barracks until the factory is up in order to defend against early aggression. Unfortunately for the defender, marines aren't very good against reapers. If the cheesing player arranges for a handful of reapers to arrive before any addons are down on a factory or barracks, it will be very hard for the defending player to hold it off.
On the other hand, the cheesing player could be very unlucky and the defending player got a tech lab on his barracks immediately. A very small number of marauders can hold off many reapers because reapers do so little damage to armored units.