What sort of audio plug are you using? Check this:
I wouldn’t lower the input impedance - you will lose signal voltage by that.