I've noticed that the raiders are getting better and better weaponry each time they raid? Will this eventually stop? Is there some upper limit or will I eventually need the best weaponry in the game just to fend them off?
No, they won't stop coming. First they all start carrying machetes, then pistols, later on hunting rifles. Then a few of them start carrying Laser Pistols. Either Laser Rifles or Plasma Pistols come next, followed by the other. Then it was the Plasma Rifles. And now it is the Railway Guns.
Don't know what will come next.
The door will only hold them for a few seconds, so you can prepare and get your best soldiers to the vault door and the next rooms.
My experience has been different than what feco describe.
My older vault still only get swords and pistols holding raiders attacks. There's at least one that had a laser pistol but the last 4 were a mix of bullet pistols and swords, one actually was swords only as you can see below!
The best weapon that vault has is the Magnetron 4000.
I got a Gaitlin laser (it might be one of the better version, I can't remember exactly, will update asap) from a lunchbox in my new vault and the raiders never used swords, they are attacking with bullet pistols already (the 2 attacks so far, don't play it much compared to my oldest).
I suspect that it is either based on how much damage your best weapon does or its type (see this list of weapons and note that the Gaitlin is heavy while the Magnetron is a rifle) that makes the raiders become more difficult.
At least is seems more important than the age of the vault.