I rather suspect it's my ISP filtering it but I haven't figured out a way to prove this and they categorically deny filtering.
It's random, occasionally it suddenly starts working for a while, anywhere from seconds to hours, but it never lasts. This started early this year when the Pirate Cove special showed up--at first it failed only in the wilderness part of it but soon it was at least 95% of the time everywhere in the game.
Turbine's tech support suggested the filtering bit as a possibility after exhausting configuration ideas, my ISP lied about it (I know they do traffic shaping) and I'm stumped.
Note that when people talk my screen does show that they are talking, I just don't hear anything.