I think that all gems receive 10% of each amount of EXP your gain. But on the same wiki's page, you can read:
Gems gain experience and levels alongside the character wearing them.
And I've got the feeling that gems on my second weapon set (which I rarely use) don't level up as fast as the ones on my first weapon set... Any clue ?