You didn't mention the platform you're playing on.
If you're using a console, other answers have already suggested solutions for that:
by System Folder
If you're playing in the PC, just use these console commands to teleport your follower to you:
Console commands to teleport your follower to the player's current location:
prid targets an entity - your follower in this case. Search for your follower's RefID here: http://uesp.net/wiki/Skyrim:Followers
(Another method to target instead of using
prid <refID> is to open up the console and then click on an entity, this will automatically do
prid <RefID> for the clicked entity. However, this method won't be useful in your situation as the follower is not near your vicinity, and can't be clicked on.)
moveto player moves that targeted entity near the vicinity of the player.
Your follower should now be near you after doing this. If for some weird reason, he/she is dead (happens sometimes), use the console command
resurrect 1. Your follower should go back to life.
If for another weird reason, your follower is still not near your vicinity after doing this, he/she may have been 'disabled' from the game (might happen with NPCs that are long dead), making him/her not appear in the game. Type
enable in the console to remedy this. If he/she still isn't up, type
resurrect 1 again.
More info on console commands here: http://uesp.net/wiki/Skyrim:Console
Downvoter(s): Please explain the reason behind your downvote(s), so I could address them and/or edit this answer to make it better, if necessary.