1 - What is the formula for calculating a hit?
3 - What is the formula for calculating an evasion?
In addition to the guide Kevin linked in his comment, GameFAQs also has a stat mechanics guide and an enemy database that's helpful for this question.
Basically, Accuracy and Evasion are used to calculate the accuracy of the attack. To calculate the hit percentage from that, you add the attacker's luck and subtract the defender's luck - hence, the importance of Luck; it's added directly to (or subtracted from) the hit percentage, so every additional point of Luck is an additional increase / decrease.
In the end, the answer is that, even if your own Accuracy and Evasion are low, if your Luck is high enough, you should hit the enemy (and they should miss you) nearly all the time.
Two caveats in the equation:
1) When you're facing an enemy with similar Luck to your own, the original accuracy calculation value would dominate the final answer. A quick glance through the enemy database, however, shows only a few enemies with high Luck (mostly the Dark Aeons, especially Dark Bahamut, Dark Yojimbo, and Dark Mindy, who are each 100+). Even in those cases, if you max out Luck, the Hit% should be greater than 100 (attacking) or less than -100 (defending).
2) According to the enemy database, there are also plenty of enemy attacks that will apparently hit regardless of the Accuracy calculation.
2 - What is the formula for calculating a critical hit?
According to a post on the GameFAQs forums, the Critical Hit formula is similar to the hit percentage:
Critical+ Value for attack + Attacker Luck - Target Luck
Again, if you max out Luck, then your Critical Hit % should rise to nearly 100%.
4 - Which enemy has the maximum stat contribution to each of these formulas? (presumably Penance)
According to the enemy database, it appears that Dark Mindy, with her 130 Luck, would be the enemy whose Luck stat would make it most difficult to hit and evade.