I think you're mistaken about the Master Sword - in Ocarina of Time, the Master Sword is a time travelling device in general. The first time Link tries to wield it, sure, it doesn't actually time travel - it simply puts him to sleep for seven years. But subsequently, Link does use it to travel backwards and forwards through time. Sheik tells us:
Past, present, future...
The Master Sword is a ship with
which you can sail upstream and
downstream through time's river...
The port for that ship is in the
Temple of Time...
So, the sword itself allows one to travel through time, though only when placed in/pulled from the Pedestal of Time behind the Door of Time (opened by playing the Song of Time on the Ocarina of Time) in the Temple of Time by the Hero of Time. Presumably all the "of Time" naming came about since everything is in effect related to the time travelling sword.
The sword was used as the lock on the Sacred Realm. Rauru tells us
The Master Sword--the evil-
destroying sword that you pulled
out of the Pedestal of Time--was
the final key to the Sacred Realm.
The Master Sword is a sacred
blade which evil ones may never
Only one worthy of the title of
"Hero of Time" can pull it from
the Pedestal of Time....
However, you were too young to
be the Hero of Time....
Therefore, your spirit was
sealed here for seven years.
And now that you are old enough,
the time has come for you to
awaken as the Hero of Time!
So, the hero who can wield the sword is called the "Hero of Time". This must stem from the sword's time travelling properties - because the sword can travel through time, the one who wields it is called the Hero of Time. The Temple that was built to house it and seal the Sacred Realm is called the Temple of Time, the door it is hidden behind is the Door of Time, the pedestal that activates it is the Pedestal of Time, the song that opens the door is the Song of Time and the instrument used to play the song is the Ocarina of Time. (Presumably they would have called it the Sword of Time if they weren't trying to tie things in to Link to the Past!)