You have to set the Datavalue of the button too, as this tells the button which direction to face. Luckily in the newest snapshots it is really easy to determine the correct datavalue by pressing F3, and then looking at the button you want to place.
At the top right corner it says: Block: minecraft:stone_button/1 which you want to place. So the full command would be:
setblock ~1 ~ ~ minecraft:stone_button 1