If anyone kills Cow King for FIRST time (except, a person who has NOT killed Baal/Diablo, see below), then everyone INSIDE the Cow Level will get credit for the (hidden) Cow King quest and will NOT make the red portal anymore. If you don't want to complete the Cow King quest or are unsure, the safest place to be is in town (use Town Portal scroll!). Unpartying will do you no good. Watch your Mercenary and Summons as they can "accidentally" kill him, too.
However, if you do not kill Baal (or Diablo, if you don't have the expansion) on that difficulty you will be unable to make the red portal, but you will never get credit for the quest to kill the Cow King (because you need to have killed Baal/Diablo to be able to start the quest). If this character kills the Cow King, everyone in the Cow Level who could make the red portal will still be able to open the portal.
Any character who has previously killed the Cow King (can't open the red portal) can kill him again without affecting people who have not yet killed him (can open red portal).
Alternatively, you can have one character (who's killed Baal/Diablo on that difficulty) make the portal, log out and then log in onto another character (who has NOT killed Baal/Diablo on that difficulty) you want to farm the Cows on.
Side note: The Cow King is Lightning Immune on ALL difficulties So if you are Lightning Fury Amazon or a Nova Sorceress, you can avoid killing the Cow King by using Lightning skills (if you don't use a Mercenary or Valkyrie). The Cow King will always drop 8 stamina potions when he dies, you can (usually) use this as an indicator of whether or not it's safe to (blindly) murder all the other cows.
Here's the official source. MOO.