Solution: provided by theChrisKenttheChrisKent, taken from the question:
I have used a combination of the answers below to make this game even more awesome. First I setup a Dedicated server as mentioned by OrigamiRobotOrigamiRobot. This was super simple and meets my requirements perfectly. I don't mind leaving my computer on all the time with this running. To do this I also had to do a couple of other things unique to my situation:
- Setup a static route in my router for port 7777 to the machine where I am running the dedicated server
- Registered with DynDNS.org to get a domain name to be used with my dynamic IP Address
- I setup the DynDNS account in my router and I was good to go (This is needed so that my friend can ping the domain name and get my current ip address should it change in the future).
Doing that took care of my need to share the world file without us having to sync it between run times.
Looking at the other answers, KissakiKissaki provided me with the folders I needed, and I used flying sheepflying sheep's answer about symlinks along with Tom WijsmanTom Wijsman's suggestion of using DropBox to share my player folder. I use the dedicated server to hold onto the world, but putting my player folder in dropbox allows me to play as the same character regardless of which of my machines I am using.