When I conquered a city state I chose to turn it into a puppet. The dialogue promised that I'd be able to annex it at any point. How do I do so?
Just left click it =)
As Jens says, if you click on the city name, a dialog box should pop up asking if you want to annex the city, or if you want to leave it as a puppet (and it should indicate the Happiness impact of such a decision).
Is this not the case for you?