So far I caught my current horse early on in the game and now I can buy them at the merchant. Are there any advantages of buying a horse?
The advantage on buying a horse is that you will be able to call that -kind- of horse back even when it dies. Thus, if you buy a nice horse such the Kentucky Saddler, American Standardbred or Hungarian Half-bred you won't need to break them again to use them.
By the way, once you buy a horse it won't automatically come to you .. you must open your inventory and use the item you bought representing the horse... then, the horse you were using will go away and the 'item' horse will come to you. This item representing the horse has unlimited uses.
When you purchase the deed to a horse, and "Use" the deed you will have that horse when you whistle for it (and the deed never goes away and you can use it as many times as you like). So purchasing the deeds to horses are the only way to switch to new horses without losing your old ones.
I would suggest you purchase the deeds to the Kentucky Saddler, Hungarian Half-Bred and American Standardbred (but make sure you get them at stores where you have the discounted rate - $750 each)
Some horses are better than others, for instance the bright white Hungarian Half-Bred and solid black American Standardbred. These horses are among the best in the game, but can't be purchased unless you've broken at lest one of the same type in the wild.
After that, a horse is a horse. For a given breed, the only difference is in the wild they're free, but if you purchase a deed it looks like you can re-use one over and over to respawn a horse of the same type if your current one dies.