The jump between realities gives people a type of amnesia the first time they go through.
Refer to the dialog in this scene:
Booker: and wipe away the debt...bring us the girl, and wipe away the debt...
Robert Lutece: See? He's starting to put his story together.
Rosalind Lutece: Hm. You're quite fond of this theory of yours.
Robert Lutece: He's manufacturing new memories from his old ones.
Rosalind Lutece: Well...the brain adapts.
Robert Lutece: I should know. I lived it.
The Lutece 'brother' experiences it the first time he goes through, and so does Booker. In both cases, rather than realizing they have amnesia, their mind simply fills the gaps by creating new, false memories in their place as a coping mechanism.
In Booker's case, when the twins pull him into Columbia and his memory is affected, his mind makes the connection with the Lutece brother (and the fact that he gave the brother Anna to wipe his debt), replaces Anna with Elizabeth in his mind, forgets who the request came from (attributing it to a shadowy figure), and simply goes along for the boat ride, where he's dropped off to go hunt down Elizabeth.