How can I get rid of a building I no longer want? For example, if I wall myself in as Protoss and later need to get a probe out to expand.
Destroy it. Unless it is a Terran bunker (which you can salvage) your only recourse is to destroy the building.
Edit: And by 'destroy' I mean to forcefully attack it with your own offensive units or workers.
You have to push A for attack, and then left click the building.
You can only destroy structures/units that you no longer want, you can always feed your enemy said units as well. To destroy buildings/units yourself you have to select structures or units that are able to attack then press the "A" key to issue a force attack order. If you hold down shift you only have to press the "A" key once and you can select all the structures/units you want to destroy.
Destroy in this context means to issue an attack order from your structures/units that are able to deal damage against units/structures you no longer need. Keep in mind you can salvage a Terran Bunker, the only structure you can get a refund from after it is built.