I just posted my first item in the auction house! It removed the item from my inventory, but where does it go if it doesn't sell? What if I don't have room in my inventory/stash? Does it stay unlisted in the auction house until I claim it?
This was a bit confusing to me as well. My initial assumption was they ended up in your stash immediately, and this is not the case. Items you buy, as well as items you fail to sell or cancel both end up on the "Completed" tab when you're finished with the transaction.
If this gives you an idea for a crazy auction house storage scheme you may be interested in this answer as well.
From the Auction House FAQ:
Items in this list won't be lost over time, but the maximum is a total of 50 items (including both your unsold and purchased but not stashed items). It's not possible to list new items once you reach 50, although you can still list items even if the total of your active auctions + the number of items on your completed tab is greater than 50. For example, you can have 49 items on the tab, plus 10 items in auctions that expire or are cancelled, and the end result will be 59/50 items on the completed tab.
According to a pre-release blog post, they were considering counting canceled/expired aunctions on the completed tab against your 10 active auctions (search for "items that have expired" to find this clause), but according to the current contents of the auction house FAQ, it appears that this restriction has been removed.
Once the auction is complete, the item will appear in the Completed tab of the AH. From here, you can send it to your stash. If your stash is full, you'll have to free some space before completing the operation.