Yes you can avoid arrest to complete this questline- or at least I did. I held off completing this quest because I did not want to go to prison when I first tried to return to Elryth around the level I would assume is expected. I RP avoiding as much "crime entanglement" as possible...the only groups I have joined are the companions, bard college, and mage college (I am arch-mage). And I have often killed forsworn. So, I went back to previous save, I became thane in Makarth first and continued helping the citizens there. Finally, at lvl 35 with high speech, I returned and saved before I went in.
I was able to talk to the guard. I responded that I am a Thane, and the guards sheathed their swords and warned me that my status only goes so far. but about 2 seconds later inside the shrine, they tried to arrest me again. I offered to pay my bounty. They again sheathed their swords, called me smart, took my money (I had nothing on me that was stolen) and I was able to go on my merry way. The guards outside treat me with the same respect as always...no change from before. They even let me go back in and loot Elryth right in front of them. (I saved before doing that as well because I had hoped he would have some sort of note or continuing quest item- he didn't and neither did his room.) The quest line now shows fully completed.
Please note that NONE of my 3 saves show anymore, which must be a glitch and one I have experienced before, but all the auto saves after the fact are fine. :( (This quest line is known to be very buggy, so who knows if what happened to me is nothing more than a glitch.)
I have been looking this up trying to decide whether or not I want to go back and complete this quest line the "normal" way and proceed to the NEW quest line No One Escapes Cidhna Mine. I can't find anyone who was able to complete Forsworn Conspiracy the way I did, so I thought it might be helpful to mention it and maybe someone else will be able to confirm if this is just a glitch or a possible viable alternative.