I have a power plant that produces 10 per cycle. It tells me that when I upgrade it, it would produce 12. I am assuming that when I build a second room, it will produce 10, too, and it even would be much cheaper to build another power plant than upgrading the existing one. So why would I upgrade?
Because the price increases with each room. While for example, a storehouse costs 300 bottlecaps, a second storehouse costs 375, and a third 450. The upgrade costs constantly stay on the same level, so after a while, upgrading is cheaper than building new rooms.
Also, a new room requires new workers, while upgrading one just increases the amount of stuff you get, but do not require any additional manpower.
So, to put it all together:
Downsides of building new rooms and not upgrading:
- More dweller required
- More power required (which requires more dweller too)
- Waste of space (You have to build elevators to get deeper) and limited space available
- Becomes expensive after a while since the prices rise for every room
If you build new rooms, you need more people to staff those rooms. If you haven't got the dwellers to spare, upgrading the old room is a good way to increase production and storage.
As the other answers have pointed out, each additional room of the same type costs more.
However, you also have limited space and so upgrading rooms means you don't end up using another floor (and so requiring an elevator as well) to keep building more rooms.
Upgrading certain rooms would give you the advantage of having free space to build rooms that are more important to your shelter's survival. I usually upgraded the living quarters, power stations, and water treatment plants to have space for the other higher yield rooms.