I've got a cool blue gem which allows me to summon up to 3 zombies from dead enemies' corps. It's now level 4, but I haven't noticed any difference in the spell (no more zombies, the summoning time looks pretty much the same, ...) And when hovering the gem, there are no help or tooltip explaining what upgrade will come with the next level. So what's the point of leveling these gems ?
Levelling up a gem is like levelling up a skill in many other games. It gives you higher damage, more effects, higher requirements, etc.
As you can see, burning arrow gains 3% physical damage, and 1% greater chance to ignite enemies. It also now requires you to be level 2 to use it.
I believe higher levels of Raise Zombie increase zombie health and damage. For other (non-summon) skills, the stats of the new level were shown on the gem at one point during beta, but do not appear to be shown now. The reason minion statistics aren't shown on the skill is because they don't exist on the skill, they exist on the minion.
You get more zombies from the passive tree.
Skills are your bread and butter. As long as you have them socketed in your equipment, they will gain experience. You don't even need to use them! This means if you have a free slot, socket it if you got a gem; no harm in doing so, and it might help you in the future.
Skill levels are exactly like Diablo 2's skills; put more points into them, they get better.
You can see my Ice Nova at Level 1 here:
And it again at level 2:
As you can see, it doesn't tell you how it gets better; the only thing it tells you is what the requirements for the next level are going to be. Ice Nova got a +1 to Cold Damage for level 2, apparently. The higher levels will require more demanding stats, such as Intelligence for spells, Dexterity for traps and such, and Strength for raw power skills.
Since skill gems are completely interchangable, and none of them are class-specific, that means anyone can use your gems, as long as they meet the requirements. They're even easy to unsocket and re-use with other equipment!
The best part is your stash is shared, so you can socket a gem, level it up a bit, dump it in your stash, and hand it off to the person who'll really make good use of it.
Some gems have stat limits. i.e. you may need a minimum of 30 dex to use them and as they increase level that limit increases. The reason you may not want to increase their level further is that this locks in any equipment you may be using that gives that stat a boost.