I finally got into orbit and I let my kerbal out for a second. Now he's floating away. Is there a way to get him back?
This is a symptom of the current (EDIT: previous 0.25) version of KSP. This version (and perhaps previous versions) have had kerbals nerfed, making more and more "mistakes" this is because the next version will have the new feature kerbal "experience" where I assume kebals will learn to EVA without shooting off into space.
I've also heard that there may be training available from the astronaught complex, but I can't recall where I heard that - it may have been a dream.
Other than that, like Shadur said, use the "R" key to turn on your RCS, "ASDF, shift and ctrl" to control your kerbal. You might find it useful to turn on your lights ("L" key). Then the "F" key will grab doors, ladders and if you have the mod, handrails. You might also find it helpful to use the "V" key to change to view.