You can leave the Kerbin solar system in the sense you can go forward to infinity, but there isn't anything else past Eelo. The only celestial bodies in Kerbin Space Program right now are the ones in the wiki.
http://wiki.kerbalspaceprogram.com/wiki/Celestials
There are discussions regarding adding other solar systems
https://kerbalspaceprogram.com/forum/showthread.php/16824-KSP-Only-the-one-Solar-System-Multiple-Solar-systems-as-future-expansions (forum is down at the moment).