**It depends on what you are looking for.**  

When I started playing, all I wanted was scrolls (wisdom and portal), magic and rare items.  So I picked up pretty much everything and sold all the items I didn't care about.

Right now, I'm more looking for [shards][1] and [orbs][2] because I'd like to enhance my gear, and in order to obtain those, I have to follow the famous [vendor recipes][3].  
For instance, I need *Armourers Scraps* and *Blacksmith's Whetstons* which require superior quality equipment, and also *Chromatic Orbs* which require items with 3 slots of 3 different colors linked together.  In addition of that, I also pick up blue stuff (4 or less slots, [like ChrisHateZ said][4]) and yellow/orange stuff when I can find some.

I identify everything before selling my stock because

 1. *Scrolls of Wisdom* can be found super easily 
 2. You can get more interesting stuff by selling identified items (once again, check the [vendor recipes][5])

So what should you pick up ?  You should pick up what you need: powerful gear (blue/yellow/orange) and everything that might be necessary for one of the recipes you are following.


  [1]: http://en.pathofexilewiki.com/wiki/Vendor_Recipes#Shards_and_Fragments
  [2]: http://en.pathofexilewiki.com/wiki/Vendor_Recipes#Orbs_and_Scrolls
  [3]: http://en.pathofexilewiki.com/wiki/Vendor_Recipes
  [4]: http://gaming.stackexchange.com/a/102804/15726
  [5]: http://en.pathofexilewiki.com/wiki/Vendor_Recipes