You should use exp books to level up your characters.
In Genshin Impact, the primary way to advance character level is to use experience books (Hero's Wit, Adventurer's Experience, and Wanderer's Advice). You get them either as rewards from quests and then later mainly as rewards for completing leylines, these are the blue cloud icons on the map. Killing monsters gives only a mostly negligible amount of exp.
For all of the methods to gain character exp see the fandom-wiki:
Character EXP gained from quests, defeating enemies, and claiming Boss rewards is given to each member of your party, regardless of participation. Dead party members will still receive Character EXP. Characters not in your active party will not receive Character EXP.
To use exp books go to your character (press c on pc) and click on "level-up." You will need to ascend them to gain further progress.
You gain experience rewards from collecting boss/leyline rewards with the active team. This means that you can switch away from your active max-level main team to a different team low-level team and collect the rewards with it, netting you a small but steady exp-gain from challenges that your second team couldn't have finished.
Not only do you need to keep up with character levels, but you should also level the weapons and artefacts that they are using and the talents of your characters. These affect your character-strength more than the character levels.