Problems detecting and handling deinterlacing and pulldown

Two things have happened recently here. Vero V is finally out and I’ve got a new TV in the lounge so the 4k Panny that was there is now in the man-cave (replacing a 1366x768 job). That means I might be able to devote some time to this, with @tanio99’s help, and it makes seeing these effects easier, and easier to compare between devices.

But first I want to address the whitelisting issues - running out of time before Kodi Omega goes beta.

Why is it a low priority?

The technical specifications page of the Vero V describes the pulldown handling and deinterlacing features, but in reality the pulldown handling is not working properly and the deinterlacing is of poor quality.

If the function described is not functioning properly or the quality is poor, I think it needs to be dealt with first.

It will get looked at, but there are more urgent things to look at first.

Well, I’m all in favour of that. :slight_smile:

Thread.

Other thread. (This is definitely a whitelist-related problem).

That second one is really a puzzle. Team Kodi dodge the issue by preferring double framerate to single framerate. We’re trying find out why Kodi likes switching to 50/60Hz even for progressive 25/30Hz streams. It’s not a white list problem per se - their whitelist logic is a cludge to cover an issue that shouldn’t logically be there. In checking this yesterday, I found 30Hz material behaves better than 25 or 29.97Hz. Is that something you can confirm? Also, I don’t think 4k video has this problem, yet TK are forcing 50/60Hz 4k output for 25/30Hz streams.

Back on topic, could someone explain to this amateur what is expected for ‘pulldown’. AIUI, it concerns 24Hz material that’s recoded as 60Hz, so 3 frames then 2 frames. A player can only play those 5 frames as-is. Are we saying some smarts are expected to smooth the jitters or is it expected the player switches back to 24Hz when it detects this 3:2 pattern? Switching framerates is Kodi’s job - I don’t think we can expect our hardware to do this independently. Do other platforms running Kodi handle it better?

If you mean the one about jerky 576i playback, I’m pretty sure that’s basically the same issue that you already worked around for 1080i/50 material some time ago. Frame-interlaced 1080i/50 stuff used to have a problem where it would start playing with 1080p/25Hz output, lurch on for a few seconds, then switch to 1080p/50Hz output, but never properly recover and play jerkily thereafter. You worked around that one a couple of years ago by making 1080i/50 always default to 1080p/50Hz right from the start.

What’s happening in my (admittedly rather obscure) case is that the whitelist is active, but all resolutions below 1080p are disabled. So, while 1080i/50 still defaults to 1080p/50, frame-interlaced 576i/50 still behaves the way 1080i used to: it defaults to 1080p/25Hz, staggers a bit, then switches to 1080p/50Hz, but doesn’t recover.

This doesn’t normally happen, because with the whitelist disabled, 1080p/25Hz is vetoed anyway; and if the whitelist is active, normally people would have either 576p/50 or 720p/50 whitelisted, and it would default to one of those rather than trying to use a 25Hz mode.

If I’m right (and I’m 90% sure I am) then you can work around it simply by making sure 576i/50 material is never output at 25Hz (as is generally already the case for 1080i/50).

“Pull-down”, to use your preferred terminology, means taking something progressive and making it frame-interlaced. The term can actually be applied to 50Hz material (25fps stored as 50i), or to 60Hz material if it’s 30fps progressive stored as 60i: you can talk about “2:2 pulldown” as opposed to “3:2 pulldown”. In the 2:2 case you’ve got each progressive frame split into a pair of interlaced fields. In the 60Hz 3:2 case, as you say, not only is each frame split into fields, but some fields are repeated in an alternating sequence.

To see what happens in an ideal world, check out my 1080i/60 wedge pattern (which is called 1080i_60_deinterlace_test.mkv - I think it’s actually the same pattern as one @Scottosan 's [pulldown/FHD/00712.m2ts] but repackaged as mkv). This a 1080p/24fps video stored as 1080i/60 (so 3:2). It’s VC-1: leave deinterlacing set to Auto-select. When it plays, my Vero 4K+ not only recognises that the video is frame-interlaced, it also actually switches output mode to 1080p/24Hz.

But switching the output mode like this is obviously not something you can do unless the entire video is consistent; if it switches from frame- to field-interlaced on the fly, or if it switches from 3:2 to 2:2 on the fly (less common) then, during the frame-interlaced 3:2 sections, probably the best you can hope for is to reconstruct the original frames and output those in a 3:2:3:2 pattern with 60Hz output. (And if you’re dealing with 480i/60 there isn’t even a 480p/24Hz mode to switch to).

You do, however, need to make sure you don’t output any frames that consist of fields from two different original frames woven together. The pattern of repeated fields looks like this:

Frame 1, odd field
Frame 1, even field
Frame 2, odd field
Frame 2, even field
Frame 2, odd field again
Frame 3, even field
Frame 3, odd field
Frame 4, even field
Frame 4, odd field
Frame 4 even field again.

So, if this is misdiagnosed as 2:2 instead of 3:2, then you will end up with five frames instead of four, and the third and fourth output frame in each set of 5 will consist of fields from two different frames combined.

So, for 60Hz material, it’s not enough to determine if it’s frame-interlaced or field-interlaced; if it’s frame-interlaced you also need to check if it’s 3:2 or 2:2, because that affects which fields are assembled into how many frames.

In terms of the expected output, if it’s 60Hz 2:2 you would probably expect either 30Hz output or 60Hz output with each frame repeated twice. If it’s 60Hz 3:2 you would expect 60Hz output with frames repeated in a 3:2 pattern, so:

Frame 1
Frame 1
Frame 2
Frame 2
Frame 2
Frame 3
Frame 3
Frame 4
Frame 4
Frame 4.

Or, if the video is frame-interlaced and 3:2 all the way through, switching the output mode to 24Hz should work correctly!

The exception to this, of course, is if you have 120Hz output. Then you can actually switch between 30fps with each frame shown 4 times, 24fps with each frame shown 5 times or 60fps generated from field-interlaced 60i with each frame shown twice, and do all that on the fly, without changing output mode. But plenty of TVs don’t support 120Hz input, so that’s not something to worry about as a first pass.

The second thread concerned playback of 1080p25. IIRC we did make it prefer double framerates even when a whitelist was used. Back then, Kodi forced double framerates only if there wasn’t a whitelist. So for a typical 1080p GUI there was no effect on 576i video. After we tried to upstream our code, TK introduced the switchable ‘3:2 pulldown’ and double refresh rate options. Note this is mis-use of the term pulldown - all it means is looking for an output mode that’s 2.5 x the framerate of the video stream. Otherwise, I don’t know what you mean by ‘your preferred terminology’. Are we talking about the same thing per the title of this thread?

I can’t reproduce that here. It just plays at 60Hz if I whitelist evrything. Besides, if I play it frame by frame I get 4 different frames then one frame repeated and no sign of a made-up frame of two fields with different timestamps. That doesn’t tie up with the description above.

@Scottosan’s 00712.m2ts plays the same as yours. 00713.m2ts is a car-crash. Plays too slow and then I can’t pause and nudge it. What’s the difference and what does PF-T mean?

That’s a little surprising. Anything in my logs to suggest why it plays at 24Hz for me?

https://paste.osmc.tv/ziliqebeve

(Probably not worth spending too much time chasing this, as VC-1 is always going to be a special case. Obviously make sure you have hardware decoding turned on and 'Deinterlace method" set to either Auto-select or Off.)

You don’t have Deinterlace method set to Deinterlace, do you? I can’t see any other way to get 4 different frames in a row with 60Hz output unless it’s misreading it as field-interlaced. If it’s deinterlacing correctly, you should be seeing frames repeating in a 2:3:2:3 pattern.

No idea. :slight_smile:

If we’re now talking about the whitelist, then I guess not. :slight_smile: At the risk of dragging the thread even further off-topic (cough) - as far as whitelist logic is concerned I am still convinced there are two changes needed: 1) Interlaced material (both HD and SD) should always default to double frame-rate even if the double frame rate checkbox is not checked, and 2) for a progressive video, the double frame rate checkbox should be changed to mean “allow double frame rates”, as opposed to its current meaning of “prefer double frame rates” - so 1080p/25fps defaults to 1080p/25Hz even if the box is checked.

(This would also fix my issue with DVD remux playback).

The best behaviour when the whitelist is not active is a bit less obvious. I’d say that a 1080p/25fps video should be output at 1080p/25Hz if that mode is available; but if it’s not available, whether you should default to 1080p/50 or 2160p/25 I wouldn’t like to say.

Obviously, if you could figure out why outputting a 25fps video at 50Hz causes stutter and fix that, then much of this would become academic…

My bad. I didn’t have all the modes whitelisted. We actually get this:

2023-10-06 22:29:49.249 T:3973    debug <general>: AMLInsecureVideoCodec::setVideoRect: decoder fps has changed, video_rate adjusted from 3310 to 4174

So kernel is feeding back to Kodi and Kodi then switches to 24p. So that’s working as expected, isn’t it?

Then if I go to @Scottosan’s 00008.m2ts, there’s a white square in the middle that is either on or top field, bottom field or off. This clip is being played as 60Hz progressive so that’s behaving exactly as I would expect as well. I can’t see any stripes on my TV - it’s just looks like 3 ons and 2 offs or 3 offs and 2 ons in a pattern that again you can see by stepping through the video one frame at a time. You do know how to do that? Pause the video, hit OK then UP and then RIGHT advances one frame. Your clip advances 1/30s with every click before pulldown is detected and 1/24s after it switches. @Scottosan’s advances 1/60s and the two on/off patterns are clear with the ball on the right or left side of the screen. 00006.m2ts is the same - it’s played as progressive. The lack of stripes I suspect is down to some processing in the TV but I can see the square ‘moving up and down’ as it’s rendered in the two fields.

Deinterlacing is indeed an issue but I can’t see anything wrong with pulldown from these clips.

Indeed. But only you would notice a difference :stuck_out_tongue:

Apologies if I’m way off target here, but would what is being discussed here regarding interlacing also apply to broadcast TV? I watch TV via TVHeadend (on a separate server) and on my new Vero V SD channels broadcasting in 576i (very common in the UK) show up as 576p in the codec information dialogue with the deinterlacing option set to off and greyed out, and look terrible. Turning MPEG2 hw decoding off enables the deinterlacing option and the video looks better (but not great), and codec info now reports the correct 576i. It’s not a big deal to turn off hardware decoding for this, software works, but curious if this was the same issue and would potentially improve in the future.

If the TV stream is either MPEG2 or h.264 codec (which it almost certainly will be for sub-4K resolutions), and it is 480i, 576i or 1080i resolution, then yes, definitely.

It also affects DVD remuxes, and blu ray remuxes of quite a number of British/European TV shows.

Yep, MPEG2 576i, which a good number (most?) UK SD broadcasts seem to be in.

Thanks

There are still deinterlacing issues!? Family Guy has an interesting mix of episodes that work and those that don’t, the episode ‘And Then There Were Fewer’ in PAL is absolutely awful for jaggies, like “it hurts my eyes” bad. Maybe one day, before I die, deinterlacing will “just work” on every device :laughing: but I doubt it.

Not mathematically possible - not for 50Hz material, anyway. If everything were flagged correctly in the video metadata, then it might be possible; but since disc authoring errors happen, and since material can switch from field- to frame-interlaced and back multiple times within one video, you have to have the ability to “guess” whether things are field- or frame-interlaced on the fly, and there will always be some cases where the picture is unusual enough that it trips up the algorithm. Dealing with cases where the image is actually frame-interlaced but where the field-order has been reversed, for example, is hard.

For 60Hz material it’s potentially much easier, because most stuff that is frame-interlaced is also using 3:2 pulldown; so, if you have enough computing power available, you can check for repeated fields. But devices like a Vero V quite likely won’t have enough CPU power to be able to do that, and even if it did, bad edits can still mess things up for individual frames.

This stuff didn’t cause visible problems on CRT televisions, so material dating back to the CRT era is often very sloppily authored.

I completely get it, just one of those hangovers we have to live with unfortunately.