Determine the root cause
It's important to inspect your buildings with the query tool on some of the dilapidated or abandoned buildings. Does it indicate not enough demand or does it also indicate abandonment due to commute time?
A lot of the behaviour you're describing can be ascribed to flaws in the default simulation that rear their head for larger cities.
Cycles
A clue is whether or not you have 'cycles' where buildings dilapidate and restore themselves. This points to a 'desirability' problem. Generally, R$$$ high-rises develop in a moderately-attractive area. The surge of people causes traffic to increase, and then commute times as well. Services can't handle and lower in quality. The desirability drops below 50 and buildings start to dilapidate and now house even more R$$ sims. This makes the problem worse and the situation escalates. Now the buildings house a gigantic amount of R$ sims. (Take Ong condos for example, it houses around 5,000 R$$$ sims but around 20,000 R$ sims). At this point we now get a lot of R$ sims. However, in a very wealthy city without I-D, I-M, I-R and with very little CS$, there isn't a lot of work space for all these poor people. So they leave and abandon because of no work. When all the people leave, the area becomes more attractive again and the cycle starts anew.
Of course, this is mathematically a complex dynamic system with chaotic properties so the reality of it is a little more involved. It may be possible to achieve a somewhat stable equilibrium depending on the model's parameters (e.g. a balance of various stages of dilapidation and abandonment that is 'just right' for a neighbourhood where it maintains itself).
The best advice to give here is the same advice when dealing with logistic models and balancing step size: develop slowly. Don't re-zone huge tracts of the city to high density in one go as this can trigger 'chaotic' dynamical systems where you have a self-reinforcing cycle of abandonment that overwhelms other effects. Aim for no more than 10-20% of one neighbourhood at a time. Let the simulation run for a few years until it stabilizes, then continue, and fix problems as they crop up.
If you have an area that is currently in such a cycle and you want to fix it I recommend reducing the density until it stabilizes. First to medium, then maybe even low density. You can start reintroducing higher density after the cycle stops.
Desirability
There's a problem in SC4 where the threshold where buildings actually will 'build' is far lower than the threshold required to 'maintain' their stature when it comes to desirability. R$$$ has quite steep requirements to not dilapidate when it comes to commute time, traffic, crime, land value, and so on (that collectively determine desirability) but will build anywhere. The result is that if left to build naturally most cities will eventually feature a large collection of dilapidated buildings.
This is actually quite realistic if you want your city modelled after a gritty place like (certain parts of) New York, Chicago, or Detroit.
You can check your 'desirability' area view for areas that are not so green. Fix the problem(s) by removing pollution and traffic, and placing public services (parks, police, education, etc.).
You can also check by inspecting a building using the query tool. If a building houses lower wealth sims than it usually does then it has been dilapidated. If it's not completely 'full' of sims (e.g. amount housed < capacity) then it's progressing towards (further) dilapidation. A building will dilapidate once half of it's inhabitants leave, while buildings of the lowest wealth type will abandon. You can also visually see clues things aren't going so well, such as clotheslines appearing in the back yard of a mansion. R$$$ becomes R$$ becomes R$ before abandon. Similar for CS$$$ to CS$. CO$$$ can become CO$$ and then abandons. The industry types abandon straight away (no wealth progression).
The disparity between 'build' desirability and 'maintain' desirability is also fixed by the 'CAM' mod amongst other things. Also see this thread over at SC4Devotion.
The CAM then has the side-effect of making the simulation a bit less chaotic, the effects described in the previous section are less pronounced.
Over-education
If you build a lot of education buildings, the EQ of your citizens will gradiually over time move to 200. A stable population without an influx of new immigrants will slowly become very educated. Over-education occurs when you don't have many low-education workers but a lot of educated workers.
The highly educated don't want to work in I-M, I-D, or I-R at all and wealthy highly educated don't like lower-paying commercial jobs (CS$ for R$$ and CS$$ and CO$$ for R$$$). However the latter do employ higher-wealth sims (a small fraction of who works there is $$$), so not having any residents with lower education will prevent workplaces from finding workers. No one wants to be a janitor, but we still need them.
Taxes too high
If you 'tax away' part of your demand (like you did with I-D or R$) this can cause inherent problems. Most of your region lives in that one city, and you're artificially removing a natural part of its population. A city is a complex interlocking mechanism, and the posh higher-wealth services like CO$$$ indirectly and/or directly depend on more mundane zones like R$ to survive. While you can use taxes to influence demand, I strongly emphasize that moderation is important. Bump up by a few percent or tenths of a percent to modify 'where' (in which city tile) people settle certain businesses but don't try to use taxes to for example 'expel' dirty industry from your region completely. That's generally bad for the economy. This topic over at SC4devotion has some really nice in-depth info on the way the jobs and residences demands interact.
Taxes are a flat 'modifier' on demand. When they are low relative to other cities in the region, demand is 'poached' from these other places to your city and vice versa. In addition, when taxes are 'too high', then demand is reduced in general. The changes are done by modifying those census drive numbers.
By the way, the reason why you can and do get some R$$$ at the start (when the only office/industry is I-D that only employes R$) is that many government buildings employ them, and they also get employed by CS$$ as well as CS$$$. The former is attracted by R$$ customers and the latter by R$$$.
Much like in SC3000, if your city becomes very large, you should lower the taxes. The 'Natural' tax rate starts out at 9% up to about 600,000 inhabitants and then moves down to 4% at 6,000,000 inhabitants in a sort-of-linear fashion (in small 0.1% increments every so many inhabitants).
Eternal Commuters
This problem (depending on your city topology) might be caused by what is the most prominent known bug in Simcity 4: Eternal Commuters. You say you rely on your neighbouring cities for work.
- Do these neighbouring cities have residents?
- Are they interconnected?
- Are they connected to eachother close to the 4 'corners' of their tiles?
If you answered yes to all three questions, you may have eternal commuters. You can also check with the traffic query tool and add up the numbers and get more commuters than you actually have people working. Here's how it works:
1) When a sim decides where to work, they go to the closest unused workspace.
2) All workspaces in another tile are located exactly at the boundary to that tile (e.g. at each of the connection tile(s)).
Now suppose you have sim A going to work from town A. There's no available workspace in town A, so they look at the borders and decide to go to town B. So from the perspective of town A, you get one commuter navigating towards the border to town B.
When you now simulate town B, you get one commuter incoming from town A. They won't go back to town A, but they look for a job in town B. Now suppose that they came in right at the corner. That means the border to town C is actually 'closer by' than your industry zone. Hence, the commuter will go to town C.
And in town C, they discover the same scenario: town 'D' is closer. And in town 'D' they might decide to go to 'B' again via 'A'. At this point you've got a loop; the amount of commuters will keep increasing while you switch cities. It causes exactly the kind of 'recession' you may be experiencing.
If you want further clarification, also see this post on Simtropolis. As explained there, in order to fully prevent this problem, you can design your transport networks as a 'tree'.
Draw a graph with each of your cities as a node. If there's a connection, connect them with a line. If there is any way to draw a route in the resulting picture that starts in any one point and ends in the same point and does not use any line twice (and isn't the trivial 'empty route') then you do not have a 'tree' graph and eternal commuters may occur.
These eternal commuters or 'phantom commuters' as I'd like to call them effectively come from nowhere and take up jobs. Hence you get a never-ending spiral of traffic and infinite industrial demand but far less residential demand.
Demand Caps
There are hard caps to the amount of people and jobs of each wealth type (R$, R$$, CO$$, I-HT, etc.) you can have in your region, determined by a fixed number. These caps can be raised by certain buildings. E.g. airports, seaports, and network connections will raise the CO caps, while parks will raise the R$$$ cap. Have you tried if building say 10 large plazas nets you an extra 90,000 people? (each plaza is worth 9,000) if so, this can be the culprit. Simply build the required buildings.
Also see the relevant Simtropolis forum thread
The 2km/h speed limit
Sims actually travel at 1.2 km/h on roads in cars in simcity 4 (buses go 2 km/h). They walk at 0.2 km/h. This causes with the default setting that you need to build neighbourhoods quite close, as there's a limit of 2.5 hours to commute time, so sims can't travel more than 3 km on road. In actuality if we use the real speeds: 31 km/h for roads, 5km/h for walking, it computes to 6 minutes. Subways run at 6 km/h (commute time) or 150 km/h (real) so can cover up to 15 km (about 2 large tiles diagonally, use manhattan distance).
Congestion worsens these values. Fully congested roads run at only 30% of the speed of free roads, worsening the issues.
In order to resolve these problems and to be able to build large specialized cities I recommend installing the NAM mod. See its documentation. An in-depth analysis of the subject can be found over at sc4devotion. This is quite a technical post, you may wish to read up on wikipedia about the pathfinding used to gain a full understanding.
