I have just been given a follower and I was wondering if I steal or buy a horse can I get her to hold it when I dismount so it doesn't wander off?
No, horse behaviour is not affected by your followers. Your follower can only hold items and equipment for you.
Horses will always wander off randomly. The only difference in this is that when you do not actually own the horse, it will wander towards its home. But, that said, even when you do own the horse, it might still wander off. You can easily retrieve it by going to a nearby stable, though. This is not the case with stolen horses, which are never flagged as 'yours'.