Should I go through 1 to 5 or just have my fun with 4 and 5 only? Would I miss anything?
Persona is like the Final Fantasy series.
Mechanics are the same, but except that, the story is completely different from game to game (except for P5: The Strikers, that is a follow up to Persona 5).
The most you will get is a quick reference to an older persona, like the protagonists of P4 going on a student visit to the school of P3.
Except if you look really deep in the lore there is not even an overarching story linking the games together. You can start with whatever Persona game you want, and you will not be lost with them referring to what happened in the previous game. I personally started with 5 with no problems. Played 4 later.
Another thing to note about the Persona Series, they seem to love to make a remake a few years after the original game, that is basically a retelling of the same story with some added components, an additional character and some QoL adjustments. So there is P3 and P3 FES, P4 and P4 Golden, P5 and P5 The Royale, that basically tell the same story just with some differences in gameplay and a few minor differences in how the story goes.
The basic things that link the Persona games together: You play a highschooler in Japan, that usually just transfered to a new school. You and the friends you make will become in contact with another world where you can use the power of personas to fight monsters. Thats pretty much it.
Unless you are willing to be surprised at the jump in graphics from ps4. The older versions have completely different characters in different worlds, while the premise remains the same. There are some different quirks and lore but the older versions are limited due to their console capacity.
However if you are itching for more franchise like story then go ahead.
there's also a continuation of ps5 coming as a small fighter game, however the release date is a mess