Most of the momentos, when shown to The Stranger or other characters, will trigger some specific dialogue revealing more about the backstory of the game. Some of them, however, create monster pets that you can interact with at the Bastion. These monster pets are useful much later on in the game, when...

>! ...The Bastion comes under attack. There, the monster pets will help you to take out some of the enemies and make the fight much easier.

Other than these, I believe the only other noteworthy momento is the `Ceramic Pot`, which allows you to access one of the special fragment-farming dungeons (Who Knows Where).

As Shaun notes, you also get a vigil at the Memorial for collecting all 15 momentos, worth 1500 fragments. You don't really need to worry about having to look around for them, as all of them are usually in plain view or down a short sidepath not very far from the main road. If you do miss them, you can buy them later at the Lost & Found.