The dialog options you pick can have a large impact on the game - in many cases small choices can have larger effects further down the line. In fact, your choices in earlier Mass Effect games can and will come back to haunt you - even though it's been years since you picked them.
The clearest of these is the "Paragon" / "Renegade" system, which usually is up-right and down-right on the conversation wheel. Usually the "Paragon" option is the "boy scout/goody two shoes" option, and "Renegade" is the "punch stuff and listen to music your parents don't approve of" options.
Picking one of these two options will usually reward you with either Paragon or Renegade points. This, alongside your reputation (which is similar, but neutral instead of good or bad) can make it easier to influence people at certain key decision points.
There are many decision points in the game though that aren't clearly called out as such. Generally speaking you're going to want to get all the information you can before you make a choice in a conversation, because sometimes people will live or die based on these actions.
It's also a very good idea to consult with all of your crewmates between missions, as between almost every mission you've got a chance to talk with them, learn something new about them, and potentially influence them. Some missions and powers are only available if you have a close relationship with your squadmates.
It can be a staggering burden to always have the fate of the galaxy in your hands, so you might want to plan on making many saves, or plan on making several runthroughs of the game to experience all the different things that can happen. Don't feel paralyzed though - it's hard to paint yourself completely into a corner, although it is possible to lock yourself out of the "best ending" if you're not careful.
Action mode takes away your ability to choose, but it does not take away the effects of these decisions. You'll be steered along a particular course that has been laid out for the "action experience." This course seems to be a mix of Paragon and Renegade choices although I don't know if it's been exhaustively tested to date.