These augments are extremely important for endgame content. They are one of the keys to progressing into the upper greater rifts.
As the accepted answer states, you get +5 for every rank of the gem.
The end result is a unique blue stat name on your item called "Caldesann's Despair" (the name of the cube recipe for augmenting), as well as the rank and bonus to the stat.
So, for example, let's say you used a level 42 legendary gem with an emerald, the added blue stat would be:
+210 Dexterity (Caldesann's Despair Rank 42)
These are the correlations for gems to stats:
- ruby (red): strength,
- amethyst (purple): vitality,
- emerald (green): dexterity,
- topaz (yellow): intelligence
- diamond (white): not available
Having enough gems is important. You should be taking your off stat gems (as in, if you are a Demon Hunter and don't use Topaz) and using Convert Gem to get them to your main stat type.
- If you want to augment a weapon, the legendary gem you use must be at least rank 30.
- If you want to augment jewelry, the legendary gem you use must be at least rank 40.
- If you want to augment armor, the legendary gem you use must be at least rank 50.
This doesn't mean that you should use a 30, 40, or 50 rank gem though. This means that once you reach grift 30, you should absolutely augment your weapon if it is ancient, and then every 5-10 grifts re-augment it.
The same goes for jewelry and armor. By 60 you may be augmenting every several grift levels you earn, as each +5 or +10 is noticeable during progression.