It depends on what you are looking for.
When I started playing, all I wanted was scrolls (wisdom and portal), magic and rare items. So I picked up pretty much everything and sold all the items I didn't care about.
Right now, I'm more looking for shards and orbs because I'd like to enhance my gear, and in order to obtain those, I have to follow the famous vendor recipes.
For instance, I need Armourers Scraps and Blacksmith's Whetstons which require superior quality equipment, and also Chromatic Orbs which require items with 3 slots of 3 different colors linked together. In addition of that, I also pick up blue stuff (4 or less slots, like ChrisHateZ saidlike ChrisHateZ said) and yellow/orange stuff when I can find some.
I identify everything before selling my stock because
- Scrolls of Wisdom can be found super easily
- You can get more interesting stuff by selling identified items (once again, check the vendor recipes)
So what should you pick up ? You should pick up what you need: powerful gear (blue/yellow/orange) and everything that might be necessary for one of the recipes you are following.