How do I make my followers do magic? Do I have to manipulate their inventory in order for them to do that?
Yes. See which followers specialize in casting spells in the UESP wiki's "Followers" article.
Do I have to manipulate their inventory in order for them to do that?
There is no need to do this for followers that can cast spells. These followers will automatically cast spells in combat. However, when given staffs, followers (even non-spell casting ones) can cast the spells of those staffs*, provided those staffs are charged with souls. In a way, you can manipulate a follower's inventory to have them cast spells.
*Related question: Which staffs will be used by followers, when given to them?
Yes followers will cast magic spells, assuming they know them.
However, you can't teach them new spells without using the console.
They usually have sone sort of ranged spell and one for melee, similiar to bow and sword. Even Vicero from the Brotherhood has a variety of abilities including stealth. You are not able to command them to cast a specific spell though.