Look for the stables outside of the major cities. You can buy them there.

You can see where the stables are in relation to you (if there are any nearby) by keeping an eye out for the following horseshoe icon in your HUD compass:
![enter image description here][1]

They also show up with a similar icon on your map:
![enter image description here][2]

And finally, here's what the stables look like when you're in visual range:

![enter image description here][3]


You can also steal horses (found at stables or in the wild), but there's a 50 gold bounty placed on your head each time a guard is made aware of you mounting one. Stolen horses also run away when dismounted, while owned horses stay where you leave them.


  [1]: https://i.sstatic.net/9HicB.jpg
  [2]: https://i.sstatic.net/yLG0N.jpg
  [3]: https://i.sstatic.net/ynutW.jpg