It seems fairly random what enemies like to hear and don't like to hear. I very frequently fail to negotiate with enemies and either just get some item or have them just attack me again or even call for backup. For instance, I had to hold up Apsaras 4-5 times before I successfully negotiated with her and Onmoraki called for backup after attacking me when I failed negotiations twice in a row, but I was able to negotiate with Slime on my first try.
Are there any sorts of signs I should be looking for in their dialogue or the options I'm given when I'm holding enemies up and attempting to get them to lend me their power? How do I know which dialogue choice is the one that they will like to hear?