As you noted, the map will show an indicator that notifies you when you collected all the skulltulas in the area.
For world map areas, the skulltula icon will be directly left of the name of the map region, as seen in the below image. Skulltulas within dungeons of these regions do not count towards this total. Any area that has its own dungeon map would have its own section of skulltulas to collect that are separate from these 'overworld' skulltulas.
For dungeons, it will do the same indicator, where the skulltula icon will be next to the name of the dungeon on the dungeon map. However, unlike the world map, you'll need to actually go into the dungeon to see the dungeon map and confirm you've collected them all.
Unfortunately, there's no way to know within the map section which skulltulas you've already collected, so there may be some backtracking and rechecking until you found all within a region.