Normally I would not expect it to be related to buffering. But your setting surely is not ideal. Just reposting the recommended one.
We recommend people with 4K TV’s set their user interface (UI) to 1080p. Kodi’s UI is not optimized for 4K yet and this can put unnecessary demands on your device and can lead to a suboptimal picture quality, as well as potentially cause other issues.
The settings we recommend are as follows…
Settings>System>Display>Whitelist> (empty) *
Settings>Player>Videos>Adjust display refresh rate> On start/stop
Some televisions may also need, or benefit from, the following being set…
System>Display>Force 4:2:2 colour subsampling> (enable)
With the above settings your UI will be output in Full HD and your 4K content will be output in 4K. *Information regarding the whitelist can be found here. If you have any doubt, feel free to upload some logs so we can verify that your settings are indeed correct.