I'm on top of the mountain, the sign says it's below me, but there's nothing but snow on either side of the mountain. A cave perhaps, but where?
It is not quite clear where you are but I think that is one of the locations you will get to during the 'The Sunstone' quest late in the main story. There is a entrance to a big cave system on the coast but it is sealed until the quest starts.
Alternatively it might be the marker for an underground location in the 'King's Gambit' quest line (this marker is just north of Kaer Trolde).
There are several markers in Skellige you can only reach as part of quests.