Actually the size has alot to do witht the game. I leveled up two pidgeys until they were both on the same point on the cp meter. One was an xl and the other regular. The xl's cp was 79 while the others was at 73. So imagine the margin between an xl and xs Pokémon. It could be avert big cp difference.
And to the above theory that an xl pokemon requires more stardust and candy is simply not true. I have an xl and regulat vennonat at the same spot on the cp meter and they both take 400 starfust and one candy to upgrade.