The only way to CQC a guard without alerting him, is by grabbing him. Simply keep the CQC button pressed when you see the hand symbol on screen.
Once you've grabbed him, you can either strangle him1 (non-lethal, silent), knock him out2 (non-lethal, loud), or kill him3 (lethal, silent).
1: release the CQC button just long enough so you can press it again, then keep it pressed until the guard loses consciousness.
2: Double tap the CQC button while pressing the left analog stick (or the arrow buttons) in any direction.
3: Press the contextual action button.