Managing your inventory and carry weight is one of the primary gameplay mechanics of Skyrim.
One thing Skyrim has done to make this easier compared to previous entries in the series are convenient exits at the end of dungeons. Many dungeons are basically circles, where the last room is actually right next to the entrance. If you're stopping halfway through the dungeon and turning around to take stuff back to town, you're doing it wrong.
Instead, realize that you don't need to, and shouldn't, pick up everything. Many players who are new to this series and similar games make the mistake of picking up everything. Most of it is just junk for atmosphere. Another mistake is paying too much attention to the gross value of an item, while ignoring its weight.
Penny Arcade has even created a comic commenting on this behavior:
When you see an item, consider its value per pound. Iron Armor is worth 125 gold, but weighs 30 pounds. That's just over 4 gold per pound. Alto Wine is only worth 12 gold, but it only weighs 0.5 pounds. That's 24 gold per pound. This means that Alto Wine is worth almost six times as much as Iron Armor. Pick up the wine, not the armor.
I usually use a rule of thumb of picking up things worth 10 or more gold per pound for most of the game, just because it's easy to calculate and tends to work out. If you happen to get full, drop some heavy stuff and keep going. Once you get to the end of the dungeon, it's usually easy to exit quickly and fast travel back to town.
Increasing Carry Weight
If you really, really want to grab everything, there are things you can do to improve your carry weight. Also note that you only need to be unencumbered when you initiate fast travel, so a very temporary increase like you might get from a potion is enough to get you from point A to point B.
Increasing your stamina increases your carry weight, there's a standing stone that increases your carry weight, and your followers can carry things for you. There are also perks that reduce or nullify the weight of worn armor, and potions that temporarily increase carry weight.
Your follower has a carry capacity, but can pick up anything if you give them an order to pick it up. Some may consider that to be exploiting a glitch.
Also, you can fast travel while on a horse even when encumbered.