If your main purpose is to redirect spawns away from the surface, you don't actually need to kill the mobs. All you would need to do would be to fill the space with spawning pads every three blocks vertically, i.e. divide the cavern vertically into 15 separate spawning rooms (3 blocks for air, 1 block for pad, repeat). This will allow all hostile mobs to spawn, including Endermen.
If you do actually want a system that will also kill the mobs and collect the drops, you will be hard pressed to avoid both water and redstone - you'll need one or the other, or likely some of both, to funnel the mobs to a central killing point. One simple method is to make large pads (10x3 or something like that) and in between them make channels 3 blocks deep and 2-3 blocks wide (at least 2 blocks for spiders). Make water streams through the channels, using the sign and stair trick to achieve constant flow, and lead them to a central point where you kill the mobs automatically by drowning or lava or dropping them or whatever. See picture below.
The slabs in my system prevent spiders spawning, and obviously you'll want a 3 block high ceiling, but otherwise this shows a good method to use. The stairs allow the mobs to be pushed up a block into the higher water stream. In this picture the water flows from left to right in the large channel, and in the small channels it flows towards the big channel. In the central collection point, the tube has alternating water and air blocks, using signs to create the air blocks, allowing the mobs to swim up while not drowning. If you want to move spiders you'll need to use a 2x2 tube and place water in two opposing corners of the tube at one level and the other two corners on the next level, then repeat the pattern.