Now that this question is answered, I'd like to go back to that cave. I already have a ship (just set my foot on the third island, after you join the pirates. Can I go back somehow? If so, how, and if I can't do it yet, when?
You will be able to travel between islands once you become a captain. That happens soon after you deal with Crow on the Sword Coast.
To travel simply talk to your helmsman at the ship's wheel and choose "It's time to set a new course" dialogue option.