How do I make my followers do magic? Do I have to manipulate their inventory in order for them to do that?
Yes. See which followers specialize in casting spells in the UESP wiki's "Followers" article.
Do I have to manipulate their inventory in order for them to do that?
There is no need to do this for followers that can cast spells. These followers will automatically cast spells in combat. However, when given staffs, followers (even non-spell casting ones) can cast the spells of those staffs*, provided those staffs are charged with souls. In a way, you can manipulate a follower's inventory to have them cast spells.
*Related question: Which staffs will be used by followers, when given to them?