Houses are a deliberate game mechanic, in The Elder Scrolls Oblivion. Please keep in mind that, while there may be some alternate accommodation available, the game has a specific list of buildings that count as 'player houses'.
As such, only these specific buildings would constitute "Places for good display of items", as in my experience, it is only in these buildings that your items can not 'reset'
Ultimately, to cover "Big space" and "Good display of items", as well as the standard "Containers with safe storage" and "Usable beds and able to wait"; your earliest, cheapest housing is going to be Benirus Manor.
Affording a House
Being able to afford a house, early, should not be difficult. 2,000 septims, IIRC, is a small sum. Personally, I was always able to purchase the Bravil house, immediately. I always interpreted this house as one of the best, and considered it a quest reward, as opposed to property I had to actually purchase.
If gold is short, hold off on buying a house. The only real reason you need a house is to store items; but early on, you should favor selling some of your loot, to make the septims you require to purchase a house.
If you are short, simply find the alchemy merchant in the city respective of where you intend to purchase a house. Buy all their ingredients, apart from the ones that are substantially expensive. Mix the ingredients into potions, and sell the potions back. These potions should easily be worth more then the cost of the ingredients. Find a place to sleep, and repeat the process, until you can afford to buy the house.
You can purchase a house in the Waterfront district of the Imperial City for 2,000 septims. Vinicia Mellissaeia will sell it to you, providing your disposition is above 50. This is easily the first house you will be able to afford, and can be later upgraded, for another 2,000 septims.
The Middle Ground: My Bravil House - 4,000 septims
You can purchase a house in Bravil for 4,000 septime. Count Regulus Terentius will sell it to you, providing your disposition is above 50. This house is a little cheaper then the house in Anvil; but offers far more quality then the standard house in the Imperial City. It can be later upgraded for another 3,900 septims.
The Most Affordable: Benirus Manor - 5,000 septims
You can purchase a house in Anvil for 5,000 septims. Velwyn Benirus will sell it to you, though he acts suspicious in doing so.
The first thing you will want to do, in the Benirus Manor, is sleep. Upon waking up, you will be greeted by three ghosts, and start the quest "Where Spirits Have Lease". After completing this quest, the house returns to normal, and becomes quite an impressive housing for a measly 5,000 septims.
