I found and identified some poisonous berries. Are they something I should hang on to, or should I just dump them? Same for poisonous mushrooms.
At the moment, not really. You can eat them, of course :) (And it sounds like fbueckert has been doing just that!) Eating them still provides nourishment, but also causes food poisoning.
Some folks have collected them for sale at the junk market, too. I suspect most just discard them, though.
Time permitting, I may add new uses for them in the future (e.g. crafting recipes), but for now, that's all they do.