The AI seem to be very fond of paying me 15 - 25 gold to establish an embassy in my capital very early in the game. I'm quite fond of free money, so I've generally taken the deal, but is that actually the right decision? What advantages / disadvantages does an embassy offer to me when playing against AI?
Jmee has the correct idea, but missed two important factors. Letting an AI civ have an embassy in your capital gives them a diplomatic buff toward you. Not a very big one, but it could make the difference between dealing with one war or two.
The one downside to accepting an embassy is it shows the civ the location of your capital. This isn't important in a single player game, because AI civs will not straight rush your capital, preferring to move gradually through border cities. However, in a Multiplayer game, revealing your capital can cause a great deal of difficulty for you. This is particuarly important because it reveals when you build wonders there. If your happy-hungry opponent is looking forward to that Notre Dam and it pops up in your capital when they have 2 turns left... good luck.
There is a downside of accepting an AI embassy I hadn't realized when I first posted this. If your capital is near to them, allowing them to see your capital can cause the Covets your land diplomatic penalty to appear. If they haven't seen your capital and are a warmongering civ it can occasionally be useful to deny them vision of your capital temporarily to prevent the Covet debuff, and therefore a war.
TLDR: AI, usually accept it. Multiplayer, don't accept it unless you need for trade.
Take the money. I can't see any reason not to:
If the civ is going under then at least you got something out of their demise.
If the civ is sticking around, then the embassy opens up the possibility of trade; they're paying you plus opening more options. It's a darn sight better position than when they demand you to pay for an embassy.