Can I build in, say, one of the forests and get people to join my settlement so we can grow? Or do I need to join the settlements that I already own / already placed? I want to design the whole thing myself, like the ones you find in the town.
No, you can only build in the areas designated by the workshops you find in settlements. The entire area within the green is able to be built in, but once you leave that, you leave build mode.