You can leave the Kerbin solar system in the sense you can go forward to infinity, but there isn't anything else past Eelo. The only celestial bodies in Kerbin Space Program right now are the ones in the wiki.
There are discussions regarding adding other solar systems
The forum lost everything since october 2012, so, i refer to this from the wiki.
The Kerbol System is at the moment the only available solar system.