The only way to CQC a guard without alerting him, is by grabbing him. Simply keep the CQC button pressed when you see the hand symbol on screen.

Once you've grabbed him, you can either strangle him<sup>**1**</sup> (non-lethal, silent, slow), knock him out<sup>**2**</sup> (non-lethal, loud, fast), or kill him<sup>**3**</sup> (lethal, silent, fast).

---
<sup>**1**</sup>: release the CQC button just long enough so you can press it again, then keep it pressed until the guard loses consciousness. According to @Lucas Leblanc, you can choke faster if you mash the CQC button instead of simply holding it.  
<sup>**2**</sup>: Double tap the CQC button while pressing the left analog stick (or the arrow buttons) in any direction.  
<sup>**3**</sup>: Press the contextual action button.