What determines what abilities human enemy units have in Final Fantasy Tactics?
As far as I know, they are generated randomly based on your highest character's level, as well as the chapter you are in. See this answer for levels. So if an enemy wizard shows up in a random battle, he can have abilities that would be reasonable for someone your level to have (without grinding). There are certain abilities that are blacklisted of course, such as something like Ultima.
I'm not sure if there is an official algorithm but, in practice, something like a wizard would not be generated with something like Bolt 4, if the ability would easily overpower your top level character. As long as you are on equal footing on equipment, which can become a problem if you out-level your equipment without stealing from human enemies.
As far as the types of human enemies, it appears that there are a few lists out there that can help you pinpoint a specific job to target. So the types appear to be generated randomly from a few types that are available based on location. See here and here