Most mobs swim to the surface, even if you're submerged beneath them.
The exceptions are:
Most mobs will not spawn in or on water. Again, this comes with the exceptions:
- Squids can spawn in any body of water, as far as 18 blocks below sea level
- Dolphins spawn in non-frozen ocean biomes
- Fish spawn in various places underwater
- Guardians/Elder Guardians spawn in and near Ocean Monuments
- Drowned mobs spawn at the bottom of oceans, rivers or swamps
Since most mobs swim to the surface of standing water, they can get to the top of a sheer edge and exit from there. No mobs can swim up waterfalls though, so you can keep them all down using running water. Non-Drowned undead mobs will stay at the bottom of water and will be unable to escape.
A long Soul Sand gate can be built along the shoreline, which will prevent mobs from crossing it. These gates can be traversed by players using minecarts on tracks placed over the Soul Sand. From the wiki:
When placed on the ground in front of
a two-block high doorway, soul sand
prevents players from going through it
from that direction, making it useful
for building one-way passages or
invisible barriers. This affects mobs
too as they have to jump to get out of
the slowing effects of the block, thus
are unable to get through the door.
Because minecart tracks can be laid on
Soul Sand like most other block
materials, this quirk can be exploited
to create minecart-only gates (by
placing soul sand on both sides of the
doorway). This is one of the few ways
to allow minecart tracks to go
directly inside a secure area without
risking wandering mobs getting in.
Here's a snapshot of a small one I built beside my house, with a crossing point for players.