The primary influences for this is your Speech skill and the associated perks.
Each item has a base value, stated in the item details when you look at it. A speech skill of 0 will see you paying around three times this to buy (actually 3.3x), and receiving around a third when selling. A skill of 100 reduces this to twice the worth to buy, and half the value when selling.
The Haggling perk adjusts this by 10%-30% for ranks 1-5 (10% first, then +5% each additional rank of the perk), and the Allure perk will give you a 30% improvement against the opposite sex.
There are also Fortify Barter potion and enchantments that you can find or brew, or get from a the Blessing of Zenithar's shrine which gives 10% for 8 in-game hours. (see here for more details).
The Skill adjustment is applied first, then each of the perk/fortify barter modifiers are applied cumulatively. A Speech skill above 100 doesn't provide any further benefit for shop prices.
The equation for working out the factor of impact for these is:
Here, Haggle, Allure, etc. are the relevant percentage point vales for these factors. Again, even if your speech skill is above 100, don't put more than 100 in the equation.
Then, for buying you multiply the value of the item by this factor, and for selling you divide it.