The other answers have very accurately described what a chunk is; a 16x16 column that represents a unit of terrain. Terrain is seeded, loaded and unloaded in these units. I will attempt to answer why they exist at all; as opposed to an amorphous continuum of blocks; loaded and unloaded as necessary.
Why not every block as its own object?
Each independent object that is passed to the graphics card has considerable overhead, its own connection to textures, a decision to render it or not has to be made etc etc. As such its implausible to have each block as its own independent object (I briefly programmed something like this; it was bad). This discounts the everything separate part of the spectrum.
Why not all the blocks together?
The other end of the spectrum is to have all the blocks grouped together as a single object (this would actually feel very similar to every block being separate). This would give the lowest overhead, but has several fatal drawbacks;
- When you're not looking at something the graphics card can cull it saving resources, if its all one big object then this isn't possible; its all rendered or none of it is
- Whenever you wanted to change a block (which happens a lot in minecraft) the whole thing would have to be rebuilt in less than one 60th of a second to look convincing and this just isn't possible.
Summary
Chunks are the intermediate position between these two extremes, you cut your number of independent objects down from hundreds of thousands of blocks down to a few hundred chunks but you can still cull parts of the scene you aren't looking at and if a block is added/removed you only need to rebuilt 1% of the scene instead of 100%