I had this problem at first too, coming from a SimCity background where the costs were all different. But over time I've learned how to avoid going into debt.
It's as simple as not overbuilding at the start.
Most things you don't need right away. To begin with you need a power plant, a water and sewage system, and some roads to support a residential neighborhood and a small commercial/industrial segment. You can also immediately set the power and water budgets to 50% to halve the upkeep since a starting town won't use nearly as much power or water (though keep an eye on the power production as power consumption ramps up pretty quick).
A common mistake is overbuilding roads. Don't build too many roads or even the super expensive roads at the start, that's how you get high upkeep before you can support it. Your population 200 town can even work just fine on dirt roads, and you can go back later and upgrade them all to asphalt when you have the money.
Once you have this initial setup, just wait for people to move in, and only build what's necessary until your weekly income turns positive. With this small start, that should happen around 250-400 people (slightly more with the Snowfall DLC because you have to supply more power when it gets cold).
The first milestone you unlock allows you to set your city's taxes. You can safely up all taxes to 12% without any adverse impact on your city.
As you unlock milestones, more service buildings become available, but again, you don't need all of them right away. An early school helps out because education is required for the first three residential building levels and higher building levels generate more income. You'll also likely need garbage and police soon after you get them. Hospitals and fire you can hold off on for a while (until a fire breaks out). Cemeteries you don't technically need until you either have a dead person waiting for transport or a lot of seniors in your city. (The game doesn't seem to allow people to age to senior until after you unlock Cemeteries, so at the start you shouldn't have to worry about this.)
Essentially, as your city grows, make sure your services stay within your budget. If you only make a net of $100/week, don't build that extra service building unless it's absolutely necessary. Wait for your buildings to level up or more people to move in so you have a bigger income buffer before expanding your budget.
Once you hit the Big Town milestone and unlock high density zoning and the University (allowing residential to progress beyond level 3), your income should begin to skyrocket, and money will become far less of an issue.
Because most services have a big impact on land value, and a building running at 50% budget is actually only 25% effective, the only services I really adjust the budget for are power and water. If I have too many hospitals I'd rather just turn one hospital off temporarily instead of tweaking the budget, so it doesn't destroy the land value in my high-value districts. (I also up the hospital and garbage night budgets to 110-120% in larger cities so the service vehicles can keep up with demand when traffic is lower, but that's tangential to the saving money discussion.)