Many people will tell you as the very first thing to avoid two-way tracks and use a pair of one-way tracks, one in each direction. Doing so will improve throughput, but isn't strictly necessary to make a system that doesn't jam.
What you need to do to not jam is make sure not to place too many rail signals: a rail signal must not be placed in a location where, if a train stopped before it, the train ahead blocking that train could also not make progress. (Or that same situation but with more than 2 trains.)
The usual way to set this up is to think of the rail network as made up of parts that are intersections, and regular non-branching track. You must never place a rail signal inside an intersection; only after an intersection. Every non-branching section (which can have as many signals as you like) must be long enough that no train which stops on it will be sticking into an intersection.
You will probably have to tear down your track and redesign it, because right now you have way too many spread-out connections that are not part of widely-spaced intersections. If you don't have enough room, see if you can make sure that trains only pass through the area in one direction — that way they might be blocked by trains ahead temporarily, but they can't deadlock.
That will prevent jams. The next step is to improve throughput by using chain signals. Chain signals allow you to break intersections into smaller parts so multiple trains can use an intersection at once — the trick is that a train will not enter a chain-signaled area until it can reserve a clear path through the entire intersection (or rather, the entire track up to a non-chain signal).
The usual way this is described is that you should:
- Place chain signals at the entrances to an intersection.
- Place rail signals at the exits from an intersection.
- Place chain signals within an intersection wherever they make usefully smaller blocks. (Additional chain signals may do nothing but cannot make the intersection jam.)