The first house available in the game is in White run. It's a great house to have, cheapest and actually probably the most useful.
White run is by far the best town for selling, crafting, etc, in other words a place to quickly pop over to in between dungeons, quests, or anytime you get a full load of loot.
The house is literally like 15 steps from the fast travel location. And next door on either side are the blacksmith and general goods store, as well as the potion shop.
This blacksmith (war maidens) is fully stocked with smelter, tanning rack, workbench, grindstone, and of course the forge. Shopkeeper restocks with more than a thousand gold each day.
It's perfect, you travel in, stop at shops very quickly to offload loot and your house for anything you wish to keep. Then whenever you need to you can do all your crafting very conveniently.
Also travelling to dragons reach keep allows you to conveniently use an enchanter, and the wizard there is stocked with all sorts of spells and soul gems as well.
I'm pretty far in the game, having completed many major quest lines and visited all the cities, and I can say that this house is by far the most useful.
That being said, it's nothing to look at and for aesthetic reasons it's probably the worst! But as a storage location it's the best.
Although I do wish there were more chests to store things in, to keep them sorted better.