While @ModDL gave a brief answer, I'd like to give an extended explanation of what exactly the issue is.
Normally, to power a redstone active block, such as a dispenser or a piston, you need to either power it directly, or power one of the 6 adjacent blocks.
For pistons, there is another rule: They also activate if the block directly above them (whether there is anything there or not) would get activated. This is called "Quasi-connectivity" (QC). While it most likely started out as a bug, it officially "works as intended" according to the bugtracker. This is because it is useful in building complex redstone machinery as a block update detector (BUD)1).
While this rule itself is fairly straightforward, and can be worked around if needed, it actually gets very strange when you look at powered blocks2).
QC and block update detectors
Most redstone components update every block two blocks away. Button-like components update 1 block from the block they are in, and from the block they are on. This means that, normally, QC is just another activation rule to consider, since there is no way to place these components such that they do not update pistons they are quasi-connected to.
Now, powered blocks only update adjacent blocks, which leads directly to your problem. The pistons still follow QC activation rules, i.e. if the powered block powers the block above the piston, the piston is powered. However, since the powered block only updates adjacent blocks, the piston does not know it's being powered, because nothing told it to check if that is the case.
This makes QC pistons ideal for use in BUD switches.
QC in your example
In your examples, the redstone wire powers the granite block below it, which powers the air block above the piston, causing it to extend. Placing the piston causes it to check for power, which is why it extends in the first place. However, removing that power does not update the piston.
In the final setup, if there is a block on the sticky piston, the repeater powers it, making it power the block below it, which activates the piston.
1) There are other methods of creating BUDs, but afaik, piston QC was used in the early days of BUD systems, and is presumably kept for legacy reasons/diversity.
2) A powered block is a solid block that receives redstone power from somewhere, such as active redstone wire pointing into it (or on top of it), levers or buttons on the block, comparators or repeaters pointing into it, a torch below it, etc. Basically, if you place a repeater pointing out of the block an it lights up, that block is powered.