Hammers and their interaction with walls is definitely one of the more annoying aspects of building in Terraria. Unfortunately, there's not really a good fix.
If I recall correctly, hammering a tile seems to be pixel-perfect. That is, if you click on a transparent part of the block (the "air" of a triangle shaped tile, for example), the wall behind that block will be affected instead. Additionally, wall-tiles themselves actually extend into the neighboring tiles, making their "hitbox" much larger than a single tile. Therefore, you need to be careful where you click when working with hammers. In particular, make sure that smart-targeting is turned off, as this can occasionally target tiles and walls that you don't want to touch.
In my experience, high-tier hammers not not really useful for construction - they are much better suited for destruction. This is because with the highest tiers of hammer, a single mouse click often lasts long enough to get two or three ticks of hammer action in before the click releases. If you're looking to put the finishing touches on a structure, I would recommend using a lower-tier hammer, perhaps Adamantium/Titanium or less, so that single clicks equate to single hammer actuations.