If my brother plays one of my games under his account, do I get the cards, or does he? Do the cards get counted towards my card-allotment, or his? Or are cards simply only handed out when the owner plays?
Neither. From a buyers point of view, it's not fair if someone else gets the card you paid for. From a borrowers point of view, it's not really fair that someone else gets the cards with your playtime.
You said it yourself: you cannot get cards for games you haven't paid for. This holds also for games with the family sharing feature. If your brother plays a foreign-library game, which he hasn't bought yet, he will get no card drops for this game. There are even bug reports on this matter, since those game will show up in your badge process, with 0 card drops, but you're not eligible for a booster pack.
All in all, only those who paid for the game will get cards when they play the game.