Is there any reason to keep items I don't need anymore? Or should I always discard them? And if I discard them, should I "drop" or "destroy"?
That depends what kind of items are you talking about here.
If you're talking about weapons you don't need, then I'd suggest you keep them anyway for as long as you can. Because if you run out of ammo for the weapon type you use as your primary you can use other ones. If you however have stronger weapon that uses the same ammo type as the weaker one, you might want to keep weaker one anyway in case your stronger one gets destroyed (either via durability loss, or if it gets disintegrated).
If you have extra armors which you don't need because they're weaker then the one you're currently using, you might want to keep them anyway for when your main armor (the one you're using) gets destroyed (it will happen sooner or later). If you really need more inventory space then i would suggest you get rid of extra armor since they do take decent amount of space.
Most materials can always be useful, so i wouldn't get rid of them even if i don't need some of them at the moment. Plus most of them take only a block or two of inventory space. However if you do know all recipes in the game, and you're sure in which materials you might not need later then just get rid of them.
Food, Medicine and Ammo
Never drop any of those. NEVER. (I think that's pretty obvious though)
Biomods and Mutation Serums
I don't even bother picking those up unless i have that item which can identify them. The risk of losing stats is not worth it.
If you are 100% sure you wont need them later on then there is no reason not to destroy them. If you however feel like you might need them anytime soon but you need to get rid of them just to get more inventory space then I'd suggest you drop them. However do note that if you already descended few floors, it's not worth coming back for items you droped because then you'll be wasting your food (hunger sucks).