So far I've been earning a healthy wage in Alliance credits, but I'm not sure what I can do with them. All my time is spent on the battlefield on missions or kicking back in the Normandy, so there's not much opportunity for shopping, or am I missing something?
Many of the stores are on the Citadel. Search every floor for panels which are often stores. Most of them are in Presidium Commons, and there's a Medi-gel upgrade store in Huerta Memorial Hospital.
Once you activate the store for the first time, it will show up in the Procurement Interface so you can buy them any time from the Normandy Armory (at 10% markup). That's on the bottom floor. Armor is a major money-sink, most armor sets cost 50,000 credits. Specre Procurements has crazy expensive guns for you, the "best" pistol, sniper and shotgun are there for ~200,000 credits.
You can also upgrade all of your weapons multiple times, which will eat up a fair amount of credits. There's a weapons bench where you can upgrade your guns in the Armory as well.
What cannot be done with credits?
They increase your effectiveness in battle if spent on the right thing, e.g. gun upgrades. You can also buy fish and flowers and all that junk, i see no use in them so far but they can be bought :)