On the Map
Alarm boxes appear on your minimap rather than the main map. Unfortunately when you're attacking an outpost the minimap's zoom level may not let you see some or all of the boxes.
An alarm box appears as an icon of a white speaker facing to the right with a white arc representing its sound. Once triggered it will change, pulsing pink or yellow if I recall correctly (someone who has FC4 installed, please edit). You can see three alarm box icons in this image.
In the World
But even when I can see one of the speaker icons, I find it only generally helpful to show me the right direction. It's much better to scout their locations in the real world, ideally from above the outpost or fortress.
Alarm boxes are almost always mounted on tall poles with conical speakers near the top. Learn to look for the speakers. You can see examples here above the crossbow, and here... well, it's hard to miss in this image.
They can appear anywhere in the outpost, though they're not as often right on the perimeter. They aren't always directly guarded, though most at least have guards that walk past them occasionally.
Taking Them Out
You can take the box out from a distance with a rifle, arrow, or even a throwing knife. Up close you can shoot them or simply interact with them for stealth.
If guards are near one when you take it out they might cry out, but that doesn't count as a detection. If they're not near the box when you take it out, a guard patrolling near it in the future is likely to yell something about the box being broken. It's fun to hear them yell about it when they run to trigger a broken box—though of course, it's more fun to cap the outpost without being detected.
