I just discovered on my Demon Hunter that I can cast his abilities by keeping pushed down the corresponding key, instead of clicking it like a madman. Overall I've noticed an improvement in damage output and responsiveness, is that just an impression? Or was my click-spamming really being an obstacle?
It is possible. The inverse is not true however. You can't click faster than you can cast by holding down (you used to be able to, but they patched that out), however if you click fast enough you can do the same DPS.
It is possible that you aren't clicking fast enough to keep up the same DPS, however it should only be necessary to click as fast as your attack speed, whatever that is. If you have a lot of +attack speed gear than it is very possible that you simply can't keep up. If you are clicking faster, then you shouldn't be at a disadvantage (? someone clarify if I'm wrong here).
In short: No disadvantage to holding down your button. No disadvantage purely for clicking, but there is a possible disadvantage if you can't click fast enough.
Actually, the opposite is true. You can get higher attack speeds through clicking than through holding the button down. You can see the question "Is there a difference in DPS depending on how you attack?" for more details.
However, Bashiok (D3 community rep) acknowledged that this was a bug, and would be fixed. I do not know if the fix is live, but I believe the goal is that no matter how you trigger your skills, they do the same DPS.