Pols Voice are susceptible to a number of non-sword attacks. Bombs, arrows, bomb arrows, and my personal favorite tactic of throwing pots at them are all functional for damage. If you're in Bottle Grotto, you're probably going to need to throw a pot.
In the instruction booklet for the original Legend of Zelda, it was mentioned that Pols Voice have a weakness to sound. While this never manifested in the original game for US audiences (as it required the microphone in the original Famicom's controllers), it manifests in Link's Awakening as instant death when hearing the Ocarina (specifically, play the 1st song).