Is there a difference between unlocking the potential of a character at level 20 or at level 40? Do I lose any stats if I do it at level 20?
Oddly enough, the other answer suggests raising potential as soon as possible, which is counter to what would be the easiest growth path for a unit. Regardless of when you promote a unit, their stats will always end up the same. The SP they have when they finally reach max level, however, will not.
The main reason to keep a unit un-promoted until level 40 regardless of rating (unless we're talking 1-2 stars...) is that you get more SP for every 10 levels. 6 per level from 1->10, 12 per level from 11->20, 18 from 21->30, and 24 from 31->40. This is all on top of the SP you'd earn simply for defeating enemies to hit those levels. For instance, if you promote a level 30 unit to the next highest tier, you miss out on all of the freebie SP you'd have earned from getting your hero to level 40. Not counting the SP from defeated enemies, that's 240 SP, which is 2/3 of a final tier passive skill or enough to buy a defensive skill like Reposition.
For units who will be needing a lot of inherited skills (read: all heroes in the competitive scene these days), you want to prolong the leveling process as much as possible without skipping any levels by using shards in order to expedite the skill learning process. Shards are mostly useful for giving less battle-oriented units that little push to their max level, but you miss out on a ton of SP if you're boosting heroes from 1 to 40 with shards.
It may feel like a drag to spend time leveling a hero you're going to promote and send back to level 1, but it doesn't compare to the drag of redoing Training Tower 10 repeatedly for 3 SP an enemy after you hit level 40 as a 5☆ hero.
Raising potential will increase the star rating of your hero, at the expense of resetting level to 1 and losing the effect of Merges. Basically starting the Hero over.
It is probably not recommended to raise the potential on a Hero you tend to use/need often unless you have enough shards to bring them back up to strength.
The benefit of raising potential is just that, the Hero will have better growth. As well as unlock new skills to use.
For example, (I'm just making up numbers) a 3 Star Corrin will have an HP of 25 at lvl 15, but Corrin raised to 4 star will get 30hp.
as to answer your question. Raise potential early if you can, that way you don't have as much work to do to bring them back up.