The answer to this is really simple, ask yourself the following question :
"Do I get a benefit from interrupting him ?"
- if yes do it !
- else don't !
For your lane opponent, if he just hard pushed the lane and you see him back, prevent him from doing it ! Because if he does recall, he'll barely lose any cs so make him lose time for you to prepare the push for when he is gone.
Other case, if he wants to recall because he is out of sustain (pots, flask, low health) or because he is behind in items and need to shop, PREVENT THE RECALL. You have an advantage on the lane, don't let it go.
If he is too far, and you need to overextend or lose cs, ask yourself the same question again, does it still gives you a benefit to do it ?
And for the jungler, if he is backing on a ward he doesn't know about, it might not be a good thing to prevent the recall and give the information that it is warded there, because when he comes back to your lane, he will know what to avoid. But if he is doing a stupid back in a visible place, go for it.