Given that mkv is a container, maybe it’s a matter of OSMC (or the hardware) being unable to decompress the video stream. A couple of days ago I tried to open an mkv with the video file encoded in h265 and my ipad would do the same, flickering video, mostly green, a good frame every 2 seconds or so, blocky most of the time and unwatchable. That was using VLC on my iPad mini, which uses the hardware to decode and process the video. I doubt the iPad has h265 support. The video played smoothly on my MacBook and my Kodi box (the one sans OSMC). So check the video using your PC just to make sure.