Beware this wall of text offers no TL;DR
Food... food. Food! > Fuel > Housing > Tools > Clothes
Set food max at half a million and never look back; this is what you trade and what keeps you alive almost no matter what (build more than enough barns to store it, and then some).
Farms should be 5x9 and worked by a single worker, paying special attention to orientate them so that when they randomly collect the harvest, they're only going back and forth on those 5 squares. Whatever dimensions you pick, should be a rectangle with this in mind.
Multiple fisheries interfere with each other, work best with the more water their 'circle' covers and with the maximum four workers. So just have one per area, keeping it fully employed throughout the year. They really help even-out the winter months.
Always have a wood cutter and foresters in operation. Always. IMO, there's only two things that will make you lose; lack of food and fuel. These two products are never something you want to be in a panic to get. Tasked, but not working anymore, workers revert into laborers, so don't worry that their times being wasted.
Try to keep a collect resources area in effect, whether or not you're employing actual laborers (of which there should always be a few, to automatically replace the deceased), so that there's always something to be done by the idlers.
Everything except tools and clothes gets set to at least 2000; those I set to 200~500 (anything higher takes up space unnecessarily). When their quota has been met, they're free to be re-tasked, just keep an eye on the stock.
Don't be afraid to send laborers way out there to get stone and iron, you don't have to carry it. One of the first things I do is build the four main roads out of town, as far as they'll go, dropping stockpiles along it every so often.
Any building that requires or produces products, should have a means of storage for those items directly adjacent to it; less walking. This is key for fisheries, as the best spot is usually a bit out of the way. Build a marketplace ASAP (employing one person per ~50 population), to keep everything flowing and your workers working, not walking around making deliveries.
Obviously you need housing and if your over-population dies from the lack of it, so be it; you weren't ready for them. Try not to accept nomads if you don't have a ridiculous amount of food stored and houses that are only half full. You need a buffer until they get settled and start contributing. Nomads are also the primary reason tools and clothes are set way above the initial 50; they'll bleed you dry of them.
The caveat being, don't ever run out of tools or clothes; people will die in the winter and work slow throughout the year. This leads to the poor production of everything. By the time you can make steel tools and warm coats, you'll have it made.
Don't be tempted to fully employ the mines even though you can throw lots of labor at them. You should have completely exhausted the map's resources before you really need them. One guy mining coal is enough for me and two or three getting stone or iron, if I can spare them. When the game gets going, this is where you can throw the fifty new people who just showed up, until you get them real jobs.
3~4 (5x9) farms is not enough, you need more like ten. I'd say at least a fifth of your population should be farming, on top of those who do husbandry, fishing, ect. Orchards should only be counted on for variety, not sustenance.
The 'outposts' consisting of a gatherer, hunter, forester and herbalist should also have one or two houses and a barn.