What is the smallest amount of things I need to survive (feed myself, build the majority of things) without ever seeing the light of day (or dark of night) in Minecraft? Could I start in a cave with just a pick (to make things easier; yes, I could tunnel through stone given enough time) and survive?
All that you really need to bring with you to survive completely underground is going to be a sapling. Without wandering around aimlessly hoping you run into a mine shaft you will need a source of wood for tools and weapons. A single sapling, assuming you get at least one sapling from its leaves, will be enough to allow you to survive underground.
A secondary concern is going to be food. It's possible to survive on zombie meat and/or apples depending on the sapling you chose to bring underground with you. However, bringing a few wheat seeds or finding a pair of mushrooms will make this part a bit more under your control.
That is basically it for the bare minimum. Can you bring sheep, cows, pigs, or chickens? Yes. Can you survive without them? Yes.
Having done the "nether only if not dead" experience, you at least have water - it'll be easier :)
The bare minimum, as James' answer states, is one sapling. If you want something resembling the "full" experience:
Tools - you need wood for nearly every tool (except shears). Need: at least one sapling, probably 4-5 just in case. At this point, you might as well chop up one tree. You can replant if you're adamant about not touching the landscape.
Light - while you can make a small shaft that ensures that you're always under spawn distance until you find lava, it's much easier if you bring enough sticks once you find coal in order to be able to use them - but you will need a few to get started. Torches, probably half a stack (32).
Food - while harvesting apples will technically allow you to survive (and spiders encountered underground will give you string for fishing poles), apples and fish make for a rather rotten experience for food - you will end up with lots of wood just trying to stay fed or spend a lot of time fishing. Rotten flesh is better if you can convert a zombie spawner into a grinder. You're probably going to want either melons or wheat. Melon seeds are in abandoned mineshafts, which is an excellent place to start even if you're doing an above-ground experience. Seeds are only found above ground, unless you conduct grass down from the surface (which means you'll be opening a hole in the surface & covering it with glass, or hoping an Enderman comes to visit holding a grass block.) Seeds aren't strictly necessary, but they make your game easier. Bring along: seeds. I'd bring at least 4 just so you don't lose your entire crop to bad lighting.
Meat - you can bring chicken eggs if you want livestock. Cows & Pigs are going to have to be imported, but chickens are portable in item form. You should bring enough eggs that throwing your stash against the wall gives you the expected value of 1 chicken. I would assume 2 stacks (32) would be overkill.
Dye - Squid generally don't spawn underground, so black is out. Flowers can be generated much like seeds can - bone meal on grass. Getting grass down into the hole can be problematic, but Endermen can be helpful here. Otherwise you're going to have to make do with a lot of blue and light blue. If you want green dye, bring cactus. That opens up cyan, at least. You don't actually need wool, because you can spin it with string.
Metal - material for armor & better tools are all found underground. No leather? No problem! It's the weakest anyway.
Nether - you don't even need to find diamonds: you can at the bare minimum construct a portal with 4 iron, one piece of flint, a couple stacks of dirt, a lava lake of 14 blocks or larger, and patience.
Potions - sand (for glass) may be a bit difficult to find underground but it's possible to find it near the surface; you may end up breaking through to the surface if you are not careful, however. Sugarcane can only be found on the surface (but it'll grow underground.) Bring along: one sugarcane stalk. Slimeballs, water, blaze rods, spider eyes, redstone, melons, gold nuggets, glowstone, gunpowder, netherwart, and anything else I've missed can all be found in the nether or underground.
XP - if you build your zombie grinder right, you can stand near your burning zombies and hit them once with a stone sword or better. This gives you the XP, rare iron drops, and the rotten flesh. (This is also possible with skeletons, but you need to take care of the case that you may get shot with flaming arrows.)
TLDR: The 4 renewables you will want are: Sapling, seeds, sugarcane, cactus - but only the first one is strictly necessary; you will need torches to start your tree farm (glowstone is out of reach, and lava is too dangerous). You will need a renewable, infinite source of wood - it's very easy to exhaust an abandoned mineshaft.
Adding on to previous answers, I'd like to mention that seeds (melon, wheat) are needed. Also, bring some wool to make a bed. If you make a staircase of grass down into your dwelling, you might be able (granted that you have enough light) to grow grass underground and spawn animals. That will probably help in getting food, but it isn't a requirement. Either that, or lure a few farm animals down with wheat and then breed them.
Also, while torches are easy to make, they do not produce much light. If you can, use furnaces for light by smelting stone or other items. Leave furnaces around your house instead of torches. Glowstone, while difficult to procure, is probably the best solution in getting light underground. If you cannot make enough, then burning netherrack lamps may be your next best bet. Another option is lava lamps (made with glass and lava), but they are difficult to make without burning yourself, and can leak during creeper explosions.
Going back to the smelting idea. Mining for resources may seem easy, but after a while, you'll notice that you're going very far away from you home to just get a hold of coal. Once this happens, it's best to already have a large tree farm so that you can make charcoal. The more furnace action you have, the better: furnaces = light + smelted goods.
If you want to feed yourself, I'd advise using the following food sources in chronological order: