From Skill Gem on Wiki:
Once a gem has enough experience to level up, it will stop gaining experience and a level–up icon will appear on the right of the screen and in the inventory menu. Left–clicking it will level up the gem, right–clicking the icon will hide it on the side of the screen, but it will still be accessible in the inventory menu. Not levelling up a gem is useful in some situations.
However, on PS4 I cannot find this option, gems just keep leveling up automatically!
How can I disable auto level up for gems?