The guy who laughs rather creepily in the Skyloft Bazaar has bombs in stock, but says I can't buy them unless I have a Bomb Bag. Where can I go to buy/receive the Bomb Bag?
The Bomb Bag is received inside the second dungeon. It's a required item, so you can't miss it.
More specifically, one of the Mogmas will tell you his Bomb Bag is missing and asks you to get it back for him, at which point he tells you to keep it.