The earliest non-chance way to get a corrosive weapon is to do the side mission "Rock, Paper, Genocide: Corrosive Weapons!" which becomes available from Marcus shortly after you get to Sanctuary. (You'll need to complete his incendiary and shock weapon missions first.) He'll give you a corrosive weapon (invariably a pistol, it seems, but the type varies) in order to shoot the target dummy. At this point you can leave the mission be (do not shoot the target dummy!) and you will have that pistol from that point on, though the type will change randomly if you reload from a save. (If you shoot the dummy you'll have the pistol for the rest of that game session, but it will not reappear if you reload from a save.)