Generally speaking, ilvl 61+ legendaries are significantly better than rares. However... most of them still have the potential to roll useless stats.
Then again, so does every single rare.
The difference between legendaries and rares is that legendaries have a pool of fixed attributes, in addition to one or more random attributes. These fixed attributes can be assigned a range of values, just like any other item, but generally speaking, the fixed attributes at least give the legendary some focus.
For example, some legendary weapons always include sockets, or critical damage bonuses, or lifeleech, or life on hit. Really good ones may include more than one (check out Skorn, for example... high crit damage and a socket, AND a high primary stat value). Where these items really shine, however, is if the random attribute(s) roll something particularly good (for example, a Skorn that also rolls lifeleech, or a Butcher's Sickle that gets good damage and a socket, in addition to its guaranteed lifeleech and crit damage).
As other answer mentioned, some legendaries allow you (guarantee you) to have attributes in a slot not normally available: movement speed on pants, chest, ring or a weapon; crit chance on pants or belt; increased attack speed on belt or helm; magic find on a weapon, etc..
These items typically have some value, as even a poorly rolled Witching Hour is still a belt that has attack speed and critical damage bonus, plus other useful stats. You can find rare belts that may be better than a witching hour, but you'll never find a rare belt that has attack speed or crit damage on it, and the vast majority of rare belts you find will have far worse stats in general.
Other legendaries may have attributes that are attainable by rares of the same type, but will have guaranteed useful combinations. Vile Ward shoulders, for example, always have 70+ all resistances, bonus armor, and a decent range of values for a primary stat. A really bad Vile Ward will still be better than an average rare shoulder. A rare shoulder could conceivably roll with better stats than a Vile Ward, but the chances of that happening are pretty meager. The fixed pool of attributes makes it far, far more likely that a Vile Ward will be "good" or even "great", than a random rare would be comparable, even though the random rare has the potential to be better (you'd need 5 or 6 "perfect" rolls on the shoulder out of a list of dozens of possible stats, vs getting a good roll on the primary stat, plus 1-2 "perfect" rolls on random attributes, for a really good Vile Ward).
Set items, as well, provide potentially useful, or even invaluable, bonuses as you get more items from a set. Legacy Natalya's set bonus, for example, is tremendously useful to Demon Hunters.
Finally, some legendaries have attributes that are unique, as well. Puzzle Rings can summon treasure goblins when you get hit, which provides some interesting farming opportunities. Stone of Jordan rings can provide large bonuses to class-specific abilities. Windforce bows knock enemies back. The Hellrack Crossbow can root foes. The sword Maximus summons a useful minion.
However, there are many legendaries that seem poorly designed, and seem like they could never be good items. Others require very lucky rolls to be decent. Since the market is a bit glutted since they increased the rate at which legendaries can be found, the vast majority of legendaries you find won't be worth much, if anything. I salvage far more than I use/sell.
Still, just think about how many rares you salvage/vendor, and remember that the proportion of useful legendaries to useless legendaries is far higher than the proportion of useful rares to useless rares.
