I have lots of 'rubbish' from enemy drops, as my current equipment is better than anything I've found so far (only a few hours in).
Is there any reason to keep hold of these items, from Dark Souls I you couldn't sell items, can you now?
Actually, if you progressed far enough in the game, you could sell items in Dark Souls I and you can do so in Dark Souls II. Eventually, you'll make your way over to No Man's Wharf, where you'll find an NPC named Gavlan behind a door on the second floor of a house (location may differ for SotFS). You can sell items to Gavlan in return for souls.
There is also no need to throw items away since your carry weight only applies to weapons and armor that you have equipped, not everything in your inventory.
Keep in mind some items that may seem like junk are actually useful in some way or another. You may regret selling them, or not live long enough to.
You also have the item box, which you can access from any bonfire from the start of the game. Instead of selling your extra items, consider simply storing them for later.
You can use "junk" weapons to craft special boss weapons using the soul items obtained for defeating them. Boss weapons need an upgraded weapon of a specific type -- greatsword, ultra greatsword, whip, etc -- as well as the boss's soul involved in the recipe. Keeping a weapon of each type is a pretty good idea if you don't know any of the boss weapons' attributes, in case you find that you want to make a particular one. You'll be able to craft these special weapons once you talk to specific NPCs.
EDIT: Nevermind, in DkS2 you only need the boss soul to make the item in question. I was thinking of DkS1.