You are trying to connect with 2 different versions of the game.
Minecraft - Windows 10 Edition, the game you're playing, is part of the universal Minecraft game, written in C++, compatible with phones, tablets, game consoles, and Windows 10 PCs.
Minecraft - Java Edition is the other edition of Minecraft. This is most likely the one your friend is playing on his Mac, and more likely the one his server is compatible with.
Minecraft Java Edition and Windows 10 Edition are NOT compatible with each other. In order to play on your friend's server, you will have to download the java edition of the game here.
If you don't own a copy of that version you may have to buy it.
Good luck and have fun!