Is Golden God unlocked by unlocking every non-DLC item, or do you need to also find and pick them up?
Unlocking all non-DLC items is the condition to get Golden God. I got the Golden God achievement as soon as I unlocked the last non-DLC item.
No, you must collect all the items in addition to unlocking them in order to get Golden God.
I unlocked this achievement after completing Challenge 1 in Wrath of the Lamb after picking up the (very dreadfully bad) Dead Cat item after a boss fight (?!). Sure, completing the challenge unlocked the Candle — but that's a DLC item, so it doesn't count.