It's possible to break enemy troops out of an overwatch state but is there a way to stop Mutons from counterattacking when melee attacked?
Flashbangs, according to this XCOM Wiki page (emphasis mine):
Prevents use of any and all special abilities - Stun Lancers will not use their blades, Officers will not use grenades, Mutons cannot even counter-attack in melee, if Disoriented.
I have not been brave enough to confirm this for myself, however.
In addition, some Psi Operatives abilities (Insanity and Void Rift) may Disorient enemies for the same effect, but this is somewhat less reliable.