After some short testing, and with this UESP article, the conclusion is simply:
It increases your max. Health and your current Health for the duration of the potion. Once the potion's effects fade, you lose the same value in max. Health and current Health. That is, unless you would die just from this - you will always have at least 1 Health remaining from the potion ending:
Alchemy description: Health is increased by <mag> points for <dur> seconds.
Enchanting description: Increases your health by <mag> points.
Fortify Health boosts the target's Health by <mag> points. When the effect expires, the target loses points of health unless that would reduce the target's health to 0 or less; i.e. the target is left with at least 1 health point when the effect expires.
Note in particular the last sentence.
(The same obviously applies to the other Fortify potions as well.)