Airplay delay ~40s

I’m running osmc on raspberry pi and trying to stream audio from my desktop.

On the desktop I’m running debian stable and pulseaudio is set to recognize airplay devices over the network via paprefs whereas osmc shows an airplay screen, nothing wrong.
The problem is that there is a delay of about 40s (!!!) between the play on the desktop and the start of the audio from the raspberry. Interestingly, if I change the volume on the desktop the strem over the network changes instantaneously.

I’ve searched a lot about that but cannot change this delay. Does anyone understand what’s happening?

Sounds like a big buffer between Pulse, libshairplay and ALSA.

Also may depend if you’re using Wired or Wireless

Wireless connection on the desktop but wired on raspberry

Is there a way to configure and reduce the buffer? Thank you