Problems with LiveTV stuttering on Vero 4k+


#103

Yes, I’m not sure whether packet drops are the problem…

I’ve installed the new kernel and have reproduced the problem with dropwatch running.

Here is dropwatch output: https://paste.osmc.tv/iduhoxagoj

During this capture, the stuttering occurred at the following times:
08:31:49
08:32:46
08:36:36
08:38:12
08:39:20

This is the kind of output I see around the stutters e.g. at 08:31:49

08:31:40 20 drops at skb_release_data+d4 (0x17aa60c)
08:31:40 1 drops at skb_release_data+d4 (0x17aa60c)
08:31:41 16 drops at skb_release_data+d4 (0x17aa60c)
08:31:42 17 drops at skb_release_data+d4 (0x17aa60c)
08:31:43 3 drops at skb_release_data+d4 (0x17aa60c)
08:31:44 25 drops at skb_release_data+d4 (0x17aa60c)
08:31:45 15 drops at skb_release_data+d4 (0x17aa60c)
08:31:45 2 drops at __udp4_lib_mcast_deliver.isra.9+254 (0x181d654)
08:31:45 10 drops at skb_release_data+d4 (0x17aa60c)
08:31:45 8 drops at skb_release_data+d4 (0x17aa60c)
08:31:45 4 drops at skb_release_data+d4 (0x17aa60c)
08:31:46 3 drops at skb_release_data+d4 (0x17aa60c)
08:31:48 2 drops at skb_release_data+d4 (0x17aa60c)
08:31:49 8 drops at skb_release_data+d4 (0x17aa60c)
08:31:50 5 drops at unix_dgram_sendmsg+29c (0x184432c)

skb_release_data+d4 (0x17aa60c)
This seems to occur all the time whilst streaming, and relates to this line of code according to addr2line:
/mnt/package/kernel-osmc/src/vero3-linux-master/net/core/skbuff.c:632

unix_dgram_sendmsg+29c (0x184432c)
This seems to occur fairly frequently, and always seems to occur immediately after the stutter
/mnt/package/kernel-osmc/src/vero3-linux-master/net/unix/af_unix.c:1613

__udp4_lib_mcast_deliver.isra.9+254 (0x181d654)
This seems to occur every now and again, even when the machine is Idle.

For this latest capture I also have posted grab-logs -A
https://paste.osmc.tv/bepiguceva

And here is the output from dmesg
https://paste.osmc.tv/eponimetev

I’m happy to be guided by your wisdom and expertise on where to go next with this.

Is there anything in these logs that helps to move us in the right direction?


#104

FTR: Last night, to rule out issues with the on-board Ethernet, I tried hooking up a USB Ethernet dongle and re-ran the tests… there was no change.


#105

That’s an interesting data-point, and suggests a possible issue in Kodi v17 rather than the kernel. I’m still looking in to the dropped packets however.

Are you able to try a Kodi v18 test build? There are a number of improvements there; particularly regarding LiveTV.

Sam


#106

Sure no problems… had started to think that would be worth trying myself.

I was previously reluctant to try the Kodi v18 test builds, but that’s not an issue now that I’ve gone back to my 10yr old HTPC for daily use.

I’ll keep you posted.


#107

Upgraded to Leia last night, and started with a fresh .kodi profile.

My first problem was that every few minutes it switched between 50fps and 25fps so the screen flashed black briefly as it switched resolution. I edited the new whitelist within kodi, and added all resolutions apart from 1080p@25fps and that seemed to do the trick.

I’ll continue testing, but so far (approx 3 hours of testing) I’ve been unable to reproduce the stuttering :grinning:

I did notice there are loads of “write codec data failed” messages during LiveTV playback. I realise these are test builds which probably need some tweaking, but thought I would flag it:

10:02:34.444 T:3367125744   DEBUG: write codec data failed, write_bytes(-1), errno(11), size(34142)
10:02:34.464 T:3367125744   DEBUG: usleep(RW_WAIT_TIME), len(0)
10:02:34.464 T:3367125744   DEBUG: CAMLCodec::Decode: write_av_packet looping
10:02:50.444 T:3367125744   DEBUG: write codec data failed, write_bytes(-1), errno(11), size(11512)
10:02:50.464 T:3367125744   DEBUG: usleep(RW_WAIT_TIME), len(0)
10:02:50.464 T:3367125744   DEBUG: CAMLCodec::Decode: write_av_packet looping
10:03:00.244 T:3367125744   DEBUG: write codec data failed, write_bytes(-1), errno(11), size(173869)
10:03:00.264 T:3367125744   DEBUG: usleep(RW_WAIT_TIME), len(65536)
10:03:00.264 T:3367125744   DEBUG: CAMLCodec::Decode: write_av_packet looping
10:03:11.764 T:3367125744   DEBUG: write codec data failed, write_bytes(-1), errno(11), size(11919)
10:03:11.784 T:3367125744   DEBUG: usleep(RW_WAIT_TIME), len(0)
10:03:11.784 T:3367125744   DEBUG: CAMLCodec::Decode: write_av_packet looping
10:03:21.964 T:3367125744   DEBUG: write codec data failed, write_bytes(-1), errno(11), size(38911)
10:03:21.984 T:3367125744   DEBUG: usleep(RW_WAIT_TIME), len(0)
10:03:21.984 T:3367125744   DEBUG: CAMLCodec::Decode: write_av_packet looping
10:04:54.444 T:3367125744   DEBUG: write codec data failed, write_bytes(-1), errno(11), size(45575)
10:04:54.464 T:3367125744   DEBUG: usleep(RW_WAIT_TIME), len(0)
10:04:54.464 T:3367125744   DEBUG: CAMLCodec::Decode: write_av_packet looping
10:05:00.364 T:3367125744   DEBUG: write codec data failed, write_bytes(-1), errno(11), size(2814)
10:05:00.384 T:3367125744   DEBUG: usleep(RW_WAIT_TIME), len(0)
10:05:00.384 T:3367125744   DEBUG: CAMLCodec::Decode: write_av_packet looping
10:05:02.324 T:3367125744   DEBUG: write codec data failed, write_bytes(-1), errno(11), size(17811)
10:05:02.344 T:3367125744   DEBUG: usleep(RW_WAIT_TIME), len(0)
10:05:02.344 T:3367125744   DEBUG: CAMLCodec::Decode: write_av_packet looping

Playback seemed unaffected.

I’m currently on the 17.8-345 build.


#108

Ive continued to test against the leia test builds and still have had no livetv stuttering, so looks like this is the solution. Looking forwards to Leia being released :slight_smile:


#109

Hi,

Thanks for the details. This is quite helpful. From some reading, one possibility is that the fragment size is too large and is being dropped. I’m not sure why this would only happen occasionally, unless there’s some broadcast packet throwing things off.

From experimenting, the external PHY supports a maximum frame size of about 3500 bytes; so Jumbo Frames (9000) would cause a problem. I don’t believe this is enabled on your network however.

I’ve been looking at changes to the networking stack and can see there have been a number of improvements in the core. I’ve backported these to 3.14 and it will be interesting to see if this improves things. It at least shouldn’t hurt. I’ll make this available soon on the network side regardless.

With that said: when streaming Live TV via TCP; you shouldn’t have stuttering. I believe that the issues you have experienced are due to the implementation of how the render thread functions in v17. This will indeed be resolved in v18; but I want to get rid of those occasional drops.

Sam


#110

Great. Let me know if there is any further testing you’d like me to do to help test the changes you’re working on.

I’ve decided to hold off moving over to the vero4k+ for daily use until leia is released. So happy to experiment with some builds or even move back to krypton in the meantime.


#111

Have reported this via the kodi forums, as i found a post where others were reporting similar behaviour on other platforms…

After further testing i seem to have a new problem whereby occasionally the picture ‘tears’ briefly like it isnt deinterlacing when playing livetv, then goes back to normal. Its only for a split second but fairly frequent and definately noticable. When testing today i played the same tvheadend channel on my vero4k+ and on my htpc running krypton, at the same time, and the difference in the tearing and smoothness of picture was definately noticable. :frowning:


#112

Does it only happen on HD channels?
Can you try an SD channel with hardware acceleration disabled?

Sam


#113

Done further testing on kodi leia this morning. I tried on sd channels (mpeg2) but didnt have the same problem neither with h/w acceleration enabled or disabled. With acceleration disabled it did drag at times, but then again, it was maxing the cpu.

With hd streams i can see the picture is occassionally moving left to right, probably only by a pixel or two. Its likely that this is causing the ‘tearing’ effect ive seen. I saw it worse whilst watching snooker yesterday where it was more noticable especially if it happened whilst a ball was in motion.

Ive just been watching good morning britain on ITV HD and could see the picture shifting left to right, its clear to see by focusing on the banner at the bottom of the screen.

Unlike my previous issue, this also affects recordings. Whilst watching good morning britain, I recorded it at the same time for a minute or so. When i played back the recording I could see the picture shifting left to right in the same places. The problem also occurs if I playback the raw .ts file via a samba share.

This didnt occur on krypton as far as i could tell, and doesnt occur if i play back the same short recording on my htpc (krypton). The recording is 98mb so could upload it somewhere if you’d like.


#114

A recording would be great, but just before that, two possible things spring to mind:

Try disable deinterlacing (persistent until reboot): echo 1 > /sys/module/di/parameters/bypass_all

(You will need to be root; so run sudo -s first).

If no joy, try turnign on noise reduction. IIRC; I turned it off because of ghosting issues; but it may be useul for Live TV:

echo 1 > /sys/module/di/parameters/nr2_en


#115

This stopped the problem i was seeing with the picture often shifting a pixel or two from left to right and back again. The tearing i was seeing also seemed to have gone away. However, without de-interlacing the picture drags in motion scenes so this isnt really a solution.

I tried this (with de-interlacing switched back on) but it didnt help with the tearing, and still the picture keeps shifting left to right and back again.

Another thing i observed is that in leia settings>system>display you can now choose between full screen and windowed. I remember in krypton this was greyed out and defaulted to windowed. In leia it’s set to full screen, and if i try change to windowed then the picture goes tiny.


#116

A further finding is that if i add a small amount of overscan compensation (1,1), (1,1), then the issue of the picture moving left to right and back again goes away. But I still i see the tearing, which as mentioned i did not see in krypton. I think something is not quite right in the Leia test builds.


#117

Tonight i looked at my tv picture settings, and have found the problem.

I had tuned these settings previously for my htpc on hdmi port 1. I then got the vero 4k+ to replace my htpc, and plugged it into hdmi1 in its place. However, when i went back to the htpc for daily use, and started trying the leia builds, i put the htpc back in hdmi1, and plugged the vero4k+ into hdmi2.

Looks like these settings on my tv are per device and/or hdmi port, so my vero4k+ (now on hdmi2) must have inherited the defaults when i switched ports.

Here are the settings which make a difference on my samsung series 6 tv:

  • Sharpness

The picture moving left to right and back again was caused by having the sharpness set to 40. If i set this back to 0 then that does not happen even with overscan set back to (0,0)

  • Digital Clean View

The ‘tearing’ was caused by having ‘digital clean view’ enabled. Disabling this setting had a positive effect on the smoothness of the picture. From what i read, this setting is only recommended for older devices.

Indeed, if i fiddle with these settings for my htpc (hdmi1) then i can reproduce the issue though didnt seem as noticable for some reason.

I also hate the ‘auto-motion plus’ (soap opera effect) setting. Indeed when i moved to hdmi2 i noticed this was re-enabled, but didnt think at the time to check the other picture settings (above) to make sure those were the same as per hdmi1.

So seems that the vero/kodi leia wasnt the problem afterall… but just (IMO) poor factory default settings applied by my tv to a previously unused hdmi port.

Sorry for the noise…


#118

@grahamh: symptom of our introduction of ‘desktop resolution’?

This is why I suggested setting nr2_en value (Digital Noise Reduction) on Vero. I actually thought this was be an issue on our side :slight_smile:

Glad you solved this.

Sam


#119

IIRC there’s a line which hard-codes non-windowed mode. If it’s in one of our patches, @gmc might like to check it’s in Leia.


#120

Hi @sam_nazarko,
I’m pretty sure I’m experiencing the same issue with live TV, watching HD channels (h264) results in stuttering, I’m using DVBLink rather than TVheadend though. The Vero4k+ is the only device in the house that’s affected, the other Kodi devices (various Intel NUC’s) play back HD TV perfectly.

Is the only fix for this to wait for Kodi 18 or is there anything I can do in the meantime? Wife and kids are going mad lol

Thanks

Karl


#121

I need more information from you.

Are recordings affected as well?
Can you provide a log?

Sam


#122

Hi @sam_nazarko
Recordings seem to be OK, I haven’t noticed any stuttering with them. I’ve just enabled debug logging and reproduced the issue but when I try to use the OSMC log uploaded i just get a pop opp saying the following:

OSMC Log Uploader
URL: https://paste.osmc.tv/

which isn’t a proper URL for a log. I tried saving to SD card and that says its successful even though i don’t have an SD card in the Vero, I presume this has saved it to the internal storage but I cant see where?

Thanks

Karl