Yes, it's possible to kill all the Yiga after they spot you.
I managed to do this, although I didn't venture into the Yiga fortress until relatively late in the game. The gear I had was:
- Upgraded 'Stasis' rune
- Savage Lynel Sword (72 damage, shortsword)
- Savage Lynel Bow (32x5 damage)
- Bomb Arrows (had 100+, but probably only used around 10)
None of those are essential, though the rune upgrade is extremely useful; especially if you don't have a massive stash of bomb arrows.
Anyhow, the basic strategy is to keep moving and don't panic. The big guys with the swords will teleport around, and one hit from them and it's Game Over. The archers are more of an annoyance, so don't worry about them initially (though feel free to take a swipe if you get a clear shot at one).
What tends to happen, as long as you keep moving, is that the sword-wielding enemies teleport to the same spot and form a big clump behind you. If you keep a medium-range distance (just slightly beyond the blast radius of a bomb arrow) from this clump, they'll slowly plod along in your general direction instead of teleporting. That pretty much allows you to hang back and rain bomb arrows on them with impunity.
If you don't have bomb arrows, another viable (though higher-risk) strategy is to use the obstacles around the room to peel one enemy off from the clump. Hit him with stasis, run in for some cheap hits, and then move back out of range before he's able to get a swing of his own in. Takes awhile, but as long as you don't allow yourself to get hit you'll eventually win.
Stasis is also useful for quickly shutting down the archers, after you've safely clumpified the swordsmen.
The ladder up to the catwalk is generally safe (although the catwalk itself is not). A good opening move is to quickly move to the upper level and run around for a bit until all the enemies have teleported above, and then transfer back to ground level. Makes it less likely to accidentally bump into a swordsman at the start of the fight.