In general, I completely ignore whites and grays - typically they're worth less than 10 gold a piece, which is not even worth dragging back to town. Do note that some "special" objects such as health potions and gems have white text, which might be confusing if you're completely ignoring that font color categorically.
In the very early going, when you have equipment slots that are completely empty, they're worth grabbing just so you're not completely naked. In particular, when you're just starting out, keep an eye out for a Targe shield - it's got typically around 50 defense, which outclasses most of the defense you'll find on any other item at this stage of the game.
In the very early phase of the game, you can occasionally find a white weapon/armor piece that has slightly better DPS than your current weapon - I've found white weapons better than blues I had from previous dungeons in the early levels previously. For this reason, sometimes I'll grab (or do a quick check by hovering over it and pressing CTRL, as bwarner notes) the first white item of a type I find, just to see if it's a class better than my current gear.
Past about level 5 or so, though, you're better off leaving them where they drop. Once you have items that give you beneficial stat boosts beyond the basic defense/DPS rating of the item, whites and grays are almost always going to be worthless to you. Even if you can get a few extra DPS or defense points out of a "higher tier" white, it probably won't offset the utility of the other bonuses on your current gear.