To block/defend/parry, you have press any direction button to block damage. Using "forward" can be helpful as your finger is probably already on the button, and up can be helpful to bound straight up from aerial parries. Block slightly before the hit lands. I first learned this from Quadrapod's Bunny Must Die walkthrough on GameFAQs which has a little info (search for Defend).
After a successful block on the ground, Bunny will stop immediately and raise her fist during the parry. After a successful block in midair Bunny will somersault straight up (maybe half normal jump height?) and you can control her during the jump. Since she does a somersault you can't stomp on enemies from a parry. You can parry again even in midair. A surprisingly satisfying ching! sound also plays with a successful parry.
The window for parrying is pretty short, even I can't do it flawlessly. Factor in your reaction time and press the button just a little before you think Bunny will take the hit. Enemies that shoot fireballs are a good place to practice this, once you have that down, try it on spikes.
You can block anything that causes damage in the game; enemy projectiles, enemy's bodies, fire, and yes, spikes. You'll really want to learn how to block spikes, as they deal extreme damage and are pretty predictable.
Enemy projectiles (except thick lasers and certain boss attacks) will be erased after a successful block (just as if you had taken the hit) and spikes will be destroyed. You can even, amusingly, walk into and block spikes to just walk through whole pits of spikes if you're good/lucky. Spikes regenerate, so keep moving.