When you "Counter" a spell it essentially never happens. This can only be done during the time the spell is being cast.
A spell that is countered is put into the graveyard instead of doing its effect.
In Magic the Gathering, everything but land is a spell. So whether your opponent is playing a creature, instant, sorcery, enchantment, etc. it can be countered.
As far as why your specific counter did not work, perhaps you targeted the wrong card? Or maybe your opponent used a counter on your Cancel (countering your counter). Another reason could be that the spell specifically cannot be countered. It is also possible that you attempted to counter first and the instant was play on the stop of the stack, meaning it resolves first. In order to know for sure, we would need to know what spell the opponent was playing as well as anything on the field.