The default rate is 1/turn, for hostile city states it is 1.5/turn.
It can be reduced by 25% with the patronage social policy and by 50% with the Greek unique ability.
Another factor is the social policy "United Front" from the Order tree, it increases the rate of decay for other players by 33% (thanks to Raven Dreamer).
If you conquer too many city states they will band together to defend against you (you'll get a notification about that). At this point, influence with non-hostile city states will decay 100% faster. Curiously, influence with hostile city states decays at the usual rate, so they are actually cheaper then. Beware that you don't need to attack many city states to trigger this event, 2 city states on normal size/Prince triggered it already. (Thanks to Sean)
I've checked some other factors using FireTuner, and neither map size, difficulty level nor current era seem to influence the decay rate.