I have the radio at Castle working and it has been for a while, but my settlement isn't growing, which leads me to think I might need to also build a recruitment beacon. But that seems like a flavor fail. Do I just need to wait for longer or should I build a beacon?
No, it doesn't work as a recruitment beacon. The castle radio will unlock...
the Radio Freedom radio signal instead as long as you have reached the relevant part of the Minutemen questline. While you can unlock the Castle as a settlement before that, the radio will not be active.
To recruit additional settlers you'll have to construct the Recruitment Beacon just as you would for any other settlement, at which point you'll pick up the Settlement Recruitment Beacon station.