I think you can also map “mute HDMI audio output” to one of buttons on your remote as I did in the past. To be honest, it has some bugs. I think it stopped working when you paused video, so I basically stopped using this enhancement (as I could turn speakers on my TV off), but it might help in your particular case.
Maybe there is smoother solution or maybe somebody has fixed the bugs meanwhile.