In GTA 5 (PC), my framerate will remain at 60 fps with V-Sync, then will dip to about 20-30 fps with no explanation. My GeForce GTX 970, Intel i7-3820, 16 GB of RAM, and a 7200 RPM harddrive can handle the game just fine.
I have a very similar PC to you and I have had situations like that as well. I found out the biggest killer of framerate in GTA V happens to be grass. Depending on the area of the map you are in (the Vinewood hills are a prime example.) and quality you set your foliage to these areas can tank your framerate a lot. Try lowering your foliage quality a little bit and you should be able to get more steady FPS.