From the Diablo Wiki:
What does the gem in chat do?
Nothing, other than change colours and put up the occasional unusual
message. As was explained to us by a Blizzard employee, it was not set
up to send data to the server, and that would be necessary in order
for it to have any other function than to just "look good."
If you get desynched/disconnected from Battle.net sometime while you
are in the channel, try clicking on the Gem. It will still change and
give you the very rare "Perfect Gem Activated" message from time to
time, even though it's obviously not sending anything to or from the
server, since you are disc'ed.
There could be two theories on the original intent: Blizzard put it in
there to taunt and amuse us, or it could be that the gem was intended
to have a function, but for some reasons that never happened, and they
just left the gem in the chat since it's fun to poke at, and they know
how people love to run with the wacky rumors and speculation.
Further information from the Blizzard Insider (also taken from the Diablo Wiki):
The edition of the Blizzard Insider for June 24, 2001, had a question
about the Gem in Chat answered by Bill Roper:
Q: I want to know what the Gem does in the Diablo II Battle.net chat
room, and don't tell me you can't say -- just tell me what it
does!!!!!!!!!!!!!!!!
A: After a solid year of silence, we just can't keep the secret
anymore. The long awaited answer to one of the most burning questions
about Diablo II is simple: It does what it does. When you click on the
Gem, it works perfectly every time, and that is what we have always
said. To paraphrase Sigmund Freud, "Sometimes a Gem is just a Gem"!
Further information, especially regarding the gem on the Diablo III site:
The Chat Gem has made its appearance on Diablo III's web page,
although when clicked, it does nothing but turn purple. The gem can
also be found on Warcraft III's World Map Editor. When clicked, the
green gem turns purple with the following message "Gem Activated". Any
units deleted after the gem has been activated will display their
"death animation" upon deletion.
In an interview at Gamescom 2011 by GameTrailers, Jay Wilson stated
that "the chat gem is not coming back" to Diablo III. He also said, "I
will definitively say, the chat gem did nothing" in regards to its
functionality in Diablo II.
To sum it all up: It doesn't really have any effect other than being something you can click on and get a random result that ultimately doesn't do anything.