I have used all of them, but I still cannot figure out which one is the best overall. What is the best overall power plant/ power source to use?
I recommend the Solar Power Plant due to low upkeep and pollution. However, it really depends on what you mean by "best".
The best plant in terms of cost per MW and upkeep per MW is the Fusion Plant. To build it, however, you have to unlock and build several unique buildings. Assuming you want to build something other than the Fusion Plant:
- In terms of initial cost per MW, the Nuclear Power plant is best. The Oil, Coal, and Solar plants take 2nd, 3rd, and 4th place, respectively.
- In terms of weekly upkeep per MW, the Solar Plant is best. Advanced Wind, Wind, and Nuclear plants take 2nd, 3rd, and 4th place, respectively.
- In terms of polluting factor, Solar, Advanced Wind, Wind, Nuclear, and Hydro are all zero pollution.
- In terms of noise pollution, the Solar plant is best, followed by a tie between Nuclear, Oil, Coal, and the Incineration Plant.
Hydro is a bit of a wildcard as the cost and upkeep vary wildly. Overall, it's not a reliable bet.
Personally, I maximize for upkeep and pollution/noise pollution as initial cost isn't a huge concern for me. Because of this, I build Solar plants as soon as they are unlocked. Until then, I usually build Advanced Wind or Wind turbines as they have reasonable upkeep (assuming you can find a windy spot to place them).
FYI: The SkyLines wiki contains a table listing all power plant options. You can sort the columns to learn more about the best plant for each category.