This post will be inaccurate when Mayhem 2.0 comes out
JorRaptor states (3:46) that when the Mayhem 2.0 rework comes out, higher Mayhem levels will drop better versions of the same gear.
The legendary gun found on Mayhem 4 will NOT have better stats than Mayhem 1. The stat ranges are determined by the item level, which is more or less based on your character level.
However, you'll find much more loot on M4 which greatly increases your chance of finding a better roll for the item.
Additionally, M4 increases the chances of finding anointment on guns. Anointments are extremely powerful endgame, and for a M4 endgame farmer a non anointed item is essentially worthless. In this way, farming on M4 does make the dropped items better (but again, it's just more likely to be better).
Lastly, there are some items that only drop in Mayhem 4. Especially the Class Mods, these are generally very good an are unavailable at Mayhem 3 or lower.