Correct. Any DLC/Crown items are directly linked to the account that was logged in when purchasing them from the store (applies to both PS and XB consoles).
AFAIK, there is no way to add more than one connection between your ESO account and your PSN/Live account. One account to one account. I can't forsee any option to link multiple PSN/Live accounts to a single ESO account. That would promote people doing what you are trying to do, that being buy once, and many play. Zen would loose money that way and its not gonna happen.
So, if your brother's account was the one you bought it on and thats the one associated in the ESO account, thats the only one with access to what you bought. You should have your own ESO account and then you can associate your PSN/Live to that ESO. The upside to that is that if yall split up (like you are over at a friend's place) you can log in and so can he and the two of you play together. Can't happen if you only have one account.