Regarding the new Skyrim Special Edition, I heard it will be a 64-bit version. How/what does this improve?
One of the prime advantages of a 64-bit executable over a 32-bit one is the fact that the 64-bit one can address a significantly larger address space (that is, it can use more RAM).
A 32-bit executable can address 4GB of RAM, in theory. On Windows, in practice, that's limited to 3GB if the executable as the "large address aware" switch on, and only 2GB if it doesn't (because of what Windows reserves for itself).
A 64-bit executable, on the other hand, can address significantly more; the upper theoretical limit is measured in exabytes. In practice some architectures will only support 48 addressing pins, but even with 48 bits of physical address space, you're looking at a RAM ceiling that's much higher than what you can realistically put in a consumer desktop anyhow, and you're not confined to the arbitrary 2-or-3 GB limitations Windows enforces on 32-bit applications.
The 64-bit instruction set also affords some performance-improving options to the compiler that builds the executable. It's likely that Skyrim benefits somewhat from these. But I would expect that it's the ability to use more of your RAM, and thus fit more and bigger things into that RAM, that is going to be the biggest improvement afforded to the game by having a 64-bit executable.