I read the moth priest would remove my dread elder scrolls, however he left me with dragon scrolls. Is there a way to get rid of them?
The "dragon" scroll is part of the base game, and is used in the main quest line.
The DawnGuard expansion gives us two more scrolls, "Blood" and "Sun", and makes use of all 3 scrolls. The Moth Priest relieves the dragonborn of these two extra scrolls.
When you are truly done with the "Dragon" scroll, look to the college to unload it.