Mario - Well he's Mario, what else is there to say.
Luigi - He jumps higher, but he also moves slower as he jumps (both up and down) for better control. He also has a bit less traction on the ground.
Peach - Can float in the air by holding jump (happens before any suit's similar ability). Makes most jumps easier, but complicates timed bounces off enemies; this is irrelevant in most levels, but makes a choice couple levels very difficult. Moves slower (both walking and running), but takes less time to reach top speed.
Toad - Can't jump as high. Walks and runs faster; might take slightly more time to reach top speed, not totally sure.
Rosalina - Has the same higher/slower jump as Luigi; moves a bit slower than Peach (also reaches top speed faster). Can also use the Cosmic Spin when without a power-up by pressing the run button, which is a weak double jump and damages enemies equivalent to a paw attack. Much more effective for horizontal distance gain than vertical.
All characters seem to have equal jump height when bouncing off an enemy, wall, or trampoline. All characters have equal speed when boosted by a Boost Panel; not sure if invincible characters also run at the same speed. Movement speed affects horizontal swim speed, but jumping ability seems to have no effect on vertical swim speed.
All characters can complete all courses. Not all characters can get all collectibles, though that's mostly because some are accessed by character-specific switches; I'm not sure if it's otherwise possible for everyone to get everything.