In the original X-COM, your units had to have a clear line of sight to the enemy. This meant that if you wanted to stay safe(r), you had to walk into line of sight, fire, and then return to your cover.
Because of XCOM's reinterpreation of TUs, this previous method is no longer possible (you'd end your turn out of cover, immediately after firing), so cover had to be changed up a little.
I find it helps to think of it like this - when in cover, your soldier occupies multiple squares at once - both the square they're currently in, and all the squares adjacent. This is why, when firing from cover, your Soldier will step out, fire, and then hunker back down.
But to make this fair, it needed to work both ways - units needed to be able to shoot at those in cover too (else, strict lines of sight would prevent both parties from firing at each other in the majority of the situations). So, again, consider your soldiers as occupying all squares adjacent to them too - able to be fired upon if an enemy has line of sight to an adjacent square.
But when you can fire at a square adjacent to the enemy, "from" a square adjacent to yourself, "cover" no longer blocks Line of Sight. Instead, it reduces the enemies chance to hit. I'm not able to confirm the actual numbers at this moment, but I believe that low cover is 20% reduction, and high cover is 40%. Using "Hunker Down" doubles the cover bonus.
Think of it as soldiers and aliens alike peering around the corner, not enough to expose their whole body, but still enough to get shot at if they're not hunkered down (aka, not peering out) .