The short answer is... you aren't playing a male dragonborn -- at least, as far as your clothes are concerned!
An adventurer's raiment does not consider voice, pronouns, or anything other than which "body type" is selected at character creation. And Baldur's Gate 3 has a finite selection in that respect.


This is much clearer for elves, but the crux of it is: picking 1 or 3 defaults to a "Female" identity and a "Feminine" voice (the even numbered voices), and picking 2 or 4 defaults to a "Male" identity and a "Masculine" voice (the odd numbered voices)
Likewise, picking body type 1 on a dragonborn...

Defaults to female identity and voice.
And so the short of it is, clothing ignores the "identity" selected by the player.
This is almost assuredly a technical limitation -- the fact that there's no way to deform the body skeleton at all implies each piece of clothing has specific render models (one per type) rather than a single model that is transformed to fit the given skeleton.
So if you want to be a dragonborn with the smaller build? You're stuck with the bra.