Booker's baptism event is the point where Booker and Comstock split or diverge from each other. One gets baptized (Comstock), the other refuses (Booker). You can say that the baptism event is how Comstock was created. So the only way to kill Comstock (all of his variations in different universes or timelines) is to kill Booker at the point right before he makes a choice to go or not to go into baptism (before he diverges into unbaptized Booker and Comstock [baptized Booker]).
"One man goes into the waters of baptism. A different man comes out, born again. But who is that man who lies submerged? Perhaps that swimmer is both sinner and saint, until he is revealed unto the eyes of man."
– Zachary Hale Comstock, "Everyman, All at Once"
...is it just Comstock that dies, or does Booker die, too?
The Comstock(s) and Booker(s) that made a decision in the baptism event all die (including the player character, and other variations in different universes or timelines), since their "source", Booker (at the point right before he makes a choice whether to go or not to go into baptism) is killed.
Further reading:
"A young Booker participates in the massacre at Wounded Knee and is tramautized by the experience. Booker considers baptism as a means of escaping his sin. In one universe ... Booker refuses, believing that a baptism cannot erase sin. In another universe ... Booker accepts the baptism and is 'born again' as Comstock. The crucial split occurs here. Comstock and Booker both reflect on their actions at Wounded Knee (though nobody remembers Comstock being there since he assumed a new identity, Comstock reflects on his time at Wounded Knee in one of his Voxophone voice records)."
Source: "My detailed ending explanation. My attempt at the most accurate ending explanation/discussion." Reddit /r/Bioshock post by user Jusicarchon
Why does Booker have to die?
"Booker, are you sure you want to go through with this?"
– Elizabeth
The idea behind the multiverse is the idea that for every decision that is made, an infinite number of universes will come into existence that explores every possible outcome of that decision. In this game, that crucial decision is the point where Booker decides to meet Father Witting to be baptized of his sins in the war. Here, Booker decides to either refuse the baptism, upon which he remains Booker DeWitt, or accepts it, upon which he becomes Zachary Hale Comstock. Now, while it is clear that there are two distinct characters that emerge from this event, what about the Booker right before the choice is made? This Booker can be considered to be simultaneously Booker and Comstock until his decision is made. This idea stems from the principle of the Shrodinger's Cat thought experiment.
Booker, right at this moment, is about to make the decision on whether to continue on with the baptism or not. The decision is right before him, and there are two possible outcomes: one in which he refuses the baptism and remains Booker DeWitt, and the other in which he accepts the baptism and becomes Zachary Hale Comstock.
KILLING THE BOOKER WHO ACCEPTS THE BAPTISM WILL NOT END THE PROBLEM. As soon as a decision is made, an infinite number of possibility spaces come into exist. The only way to truly stop this space from forming is to kill Booker while the possibility of Comstock exists, but he has not made a decision yet. This is the state in which Booker is simultaneously Booker and Comstock, and this is where the multiverse Elizabeths drown him. This can only be possible when Booker realizes the existence of this loop and accepts his death.
But the Booker we played was the one that was drowned...right?
"The mind of the subject will desperately struggle to create memories where none exist..."
– R. Lutece, Barriers to Trans-Dimensional Travel
So, throughout the game, we see instances of Booker's memories getting rewritten, as well as some other people. Whenever this happens, it is indicated by dizziness and bleeding from the nose...
Whenever Elizabeth opens a tear, she is moving to a universe that contains the conditions that she wants or needs at the moment. However, her ability also has a small area of effect, bringing with her people around her in a small radius. We can see this shown when Fink's Head of Security is unaffected, despite being dead in the first timeline. Whenever a tear is opened and the people step through, the mind will attempt to settle paradoxes by rewriting memories...
... Elizabeth has the ability to open any tear she wants and can access the infinite doors that lead to different timelines. So, when Booker steps through, he is essentially going back in time, his own self is reverted to that time period's Booker. This can be seen when you accept Preacher Witting's hand for baptism, Booker's own hand is young and has no AD or bandage wrapping around it. This signifies that Booker has merged with his younger self, starting again at the baptism scene. This points to the fact that the Booker that we play through and the Booker that is drowned by the Elizabeths are one and the same person.
Excerpts from: "Bioshock Infinite: The Ultimate Spoiler FAQ" Reddit /r/Bioshock post by Reddit user awchern. See the original post for more details.
