I think you're missing a few key (but subtle) features of the game that are hindering your happiness management. Stellaris has many interconnected moving parts.
If you come from Civilization-like games to Stellaris, the amount of stats and control you exert over your empire is a bit daunting. I suspect that you may have been glossing over a few new elements that aren't common in other 4X games.
- Planet habitability is a happiness cap.
Every planet has a habitability score. Your species' homeworld is 100%, other planets of the same type as your homeworld are 80%, the other types in the same category are 60%, other categories are 20%, gaia worlds are 100%.
If you settle a planet with 20% habitability, that 20% becomes the cap for happiness. Even if you do everything right and your citizens should be running at 60% happiness, their happiness will be capped at 20% due to planet habitability.
Low habitability has more drawbacks than just happiness, but happiness is the most relevant for this question.
In the beginning, you only want to colonize planets of >50% habitability because you lack the tech to improve the planets. When you unlock the tech, you can terraform planets to change their habitability.
Robotic populations get a flatout +200% to habitability so that they are not affected by planet habitability. If there's a low-hab planet with plenty of minerals on it, you could establish a robot population there instead of an organic one.
More info on habitability can be found on the wiki.
- You can only control a limited amount of core sector worlds.
This means planets that are under your direct control. The default is 3, though there are trait, traditions and techs that can seriously increase that number.
Note that this is a soft cap. You're able to colonize more planets than your cap allows you to, but you suffer a penalty when you do. According to the wiki, the penalty is for influence and energy; but I wouldn't be surprised if this can indirectly influence population happiness too.
It is possible to colonize more planets than your core sectors allow, but you have to assign them to sectors (think of it as provinces of your empire, with a local governor). Any planet that is assigned to a sector no longer counts as a core sector planet. Mineral and energy gains from a sector are lowered (you can regulate the tax on them), science and resources are given to you without a reduction.
The wiki page has more information on both core systems and sectors.
- Population that belongs to a faction has a base happiness equal to that of the faction.
Before modifiers are applied, every pop starts off at 50% happiness. However, if a pop is part of a faction, it will start off at whatever the faction's happiness is. If you spend no effort on keeping your factions happy, your pops are going to be rather unhappy.
On top of the base happiness, there are a ton of modifiers that can influence the final happiness rating of a pop.
The most common negative modifiers come from: planet modifiers, being enslaved, badly chosen civics and rights, starvation.
The most common positive modifiers come from: planet modifiers, ruler traits, buildings, edicts, traditions.
More information on happiness on the wiki.
- Stellaris has an in-built wiki browser for a reason.
I'm not trying to be a hard-ass here; but I do feel it's relevant to address this. If you click on the question mark button (bottom right of the game screen), it will open an in-game browser to the Stellaris wiki.
The game has so many features and stats that it's hard to keep everything apart from time to time, especially in your first few campaigns.
I've played for 60+ hours and I still regularly use the wiki when I'm unsure of something. Most of what I have told you in this answer comes from reading the wiki, more so than anything else.
The wiki is of course also available outside of the game.
