I'm having some serious trouble with being overburdened on my Knight all the time. If I find a better armor than my starting ring mail and put it on and drop my ring mail, I'm overburdened! What's the most reliable way to carry more stuff early on in the game?
The most obvious way is to gain strength. Various ways to do that are listed on the Nethack wiki.
In addition to that having a blessed (or even uncursed) Bag of Holding helps greatly, so doing Sokoban early may help (though of course there's only a 50:50 chance that you'll get a Bag of Holding there).
Other than that the best way to not be overburdened, is to stash the things that you don't need right now somewhere and come back to it later when you can carry more.
If you get an early wish it's also a good idea to wish for dragon scale mail. Not only are they the best armour in the game, they're also the the lightest. (You could of course also wish for a Bag of Holding, but if you only have one wish, I'd rather go with the dragon scale mail since that's also useful beyond losing weight).
For knights it is also helpful to know that you'll have maximum carrying capacity when riding.