UPDATE: This all changed in a release a few months ago.
In outfitting, there's an option to 'transfer' or 'store' any module. When you buy another module for a slot, there's an option to store the one you have equipped or to sell it. Simply store a module, switch ships, and transfer the module from module storage. If you try to store a required module (powerplant, fsd, etc), it will put an 'E' rated version on (which you'll have to pay the cost of).
Additionally, you can move these modules without having to fly them in a ship. It costs credits and takes time, both of which increase the further away it is.
Old outdated info that is now wrong:
Yes, it's possible to move them from one ship to another, however, you can't store them.
If you're in a station with outfitting and a shipyard (with the ship you want to buy), you can sell the utilities and guns on your current ship in outfitting, switch ships in the shipyard, then go back into outfitting and you'll have an option to "rebuy" the stuff you just sold at the same price you just sold it at. If you undock and come back, that rebuy option will be gone. I'm not sure they'll still be there if you quit and restart, either.
Make sure you have enough money to buy the new ship and to buy back your parts.