One could say they were evil, as they've launched a seemingly unprovoked attack on the nord city of Saarthal, known as the Night of Tears, slaying everybody in the city, after which they were soundly crushed by Ysgramor and his Companions, whom he brought from Atmora. They then were poisoned by the Dwemer, who offered them refuge underground. They were made to eat poisonous mushrooms that made them blind and became a necessary part of their diet, ensuring they would forever remain so.
Of course, history is written by the victor, and the attack on Saarthal may not have been as unprovoked as the books suggest. Not to mention the fact that Nords are a rather eccentric bunch who came to settle Tamriel uninvited by its inhabitants.