You can leave the Kerbin solar system in the sense you can go forward to infinity, but there isn't anything else past Eelo. The only celestial bodies in Kerbin Space Program right now are the ones in the wiki.

http://wiki.kerbalspaceprogram.com/wiki/Celestials

There are discussions regarding adding other solar systems

https://kerbalspaceprogram.com/forum/showthread.php/16824-KSP-Only-the-one-Solar-System-Multiple-Solar-systems-as-future-expansions (forum is down at the moment).

You can leave the Kerbin solar system in the sense you can go forward to infinity, but there isn't anything else past Eelo. The only celestial bodies in Kerbin Space Program right now are the ones in the wiki.

Celestials

There are discussions regarding adding other solar systems The forum lost everything since october 2012, so, i refer to this from the wiki.

The Kerbol System is at the moment the only available solar system.

Kerbol System