I've found some Emerald, Diamond and Amethyst but haven't seen what I can do with it. I'm still in the beginning stages, trying to outfit myself in gold armor and weapons. I was curious if I should sell these minerals or keep them around for crafting purposes.
You can use ten of the same gemstone, plus 20 Meteorite Bars, to craft a phaseblade. All phaseblades are identical except for color, which is based on the color of the gemstone. You only need ten total gemstones, all of one type. You do not need ten of each.
As of 1.05, they have other uses! Six to ten each of two gemstones are required to craft Jungle Armor set pieces. You will need ten emeralds and sapphires, eight amethysts and topaz, and six rubies and diamonds. (for a total of 48 gemstones.) Additionally, rubies are used to create the Robe and Gold Crown, while topaz are used to create the Sandgun. (Source: http://wiki.terrariaonline.com/)
As of 1.1, each of the gemstones can be combined with 3 torches to produce 3 colored torches (shows off Terraria's new lighting engine!).
As of 1.2, each of the gemstones has an associated ore that it can be crafted with to create a "___ Rod" item, which is a rod made out of 15 of your gemstones and 10 bars of a particular ore. The Diamond and Ruby rods are First-and-second best, and use Platinum and Gold bars respectively.
All gemstones can now also be used to craft new grappling hooks that don't require a Hook to create, making them ideal for new characters. The Ruby and Diamond hooks also have a longer reach than the Ivy Whip.
In addition, these gems can be used to create Large Gems for the Capture-The-Flag game, so it would be wise to hold onto some in case you ever wish to host said game for some of your friends.
As of patch 1.04, Rubies are used in two non-phaseblade recipes -- the "Robes" and the "Golden Crown" vanity items.
The robes requires 3 rubies and 30 silk, while the Crown requires 1 ruby and 30 gold bars.
As of patch 1.05, Gemstones are used to create the pieces of Jungle Armor as well.
You'll need one of each to craft the whole set.
5 Topaz can be used to make the Sandgun with Antlion mandibles and Illegal Gun Parts.
Finally, as of 1.1, each of the gemstones can be crafted with 3 torches (no workbench, etc. required) to produce colored lights (same respective colors as the phaseblades).
The main thing that you can craft with them is the phaseblade, which takes 10 of one type (changes color), but other than that, sell sell sell.