As having recent experience in this field, I can tell you exactly how it goes!
If you turn apostate, the god gets angry and punishes you, which is a 10000 turn hex that reduces your Speed by 20 and your PV by 20%. It can't be dispelled and it sticks even after death, because you made a god angry geez. That's the whole of it though.
This isn't a strong penalty and it goes away soon enough. I spent the entire punishment period fishing in my basement to curry favor to my new god (plus a day travelling to town to buy more lures). Switching gods is actually rather recommendable - the only things you retain for following a god are the favor-based bonuses to certain skills/stats, and the innate blessing they give (like Yacatect's 'All Collection'). The gifts they give you for prayer at certain levels of favor (a servant, an item, and a weapon) all are retained even if you switch gods, so you can always switch to get more of these from more gods. Note though that servants are capped at one servant for every 10 levels in the Faith skill you have (in original, it was capped at 2). Just make sure you stop at whichever god you want to retain the bonuses of.
Of note, the Festival of Jure that occurs in Noyel in the last month of each year, you can actually convert to following Jure of Healing without incurring punishment from your current god.