Deinterlacing / extraneous scaling issues

[angry.sardine]

This is an approximate repost of some stuff that I already raised in the v19 testing thread. I’m reposting it partly because that thread has now been locked, and I think the issues deserve an active thread, but mostly because, at the time of the original post, I was under the impression that these were longstanding issues, and didn’t represent a major regression. It turns out I was wrong about that: these are longstanding issues for people like me who have been testing the 4.9 kernel for a long time, but some are major regressions relative to the November 2020 release.

First a link to some of my deinterlacing test clips:

Please click here

And now the issues…

---

Things that have got worse since the November release:

• The Little Dorrit clip is 1080i/50 VC-1, captured as progressive frames. Watch for shots where you can see a gold box design on the side of the piano, and look at the top and bottom edges of the gold box and the reflections along the bottom edge of the piano. Artefacts can be fixed by setting deinterlacing to Off, but this is quite fiddly, because you not only have to manually set the deinterlacing mode, you also have to have set up a whitelist, and then manually switch the output resolution from 1080p/50 to 1080p/25 (otherwise you get jerky playback) - see also this thread.

Screenshots…

Default deinterlacing:

Correct deinterlacing:

---

• The Sherlock clip is 1080i/50 h.264, captured as progressive. Watch the pattern on Mycroft’s grey suit, especially between 00:30 and 00:47 - you get a shimmering effect because of deinterlacing errors. There is no workaround for this. (This issue is actually also visible in the November release, but it was somehow less “obvious”, in a way I can’t quite put my finger on).

What it should look like (Kodi screenshot):

What you see on screen (close-up):

---

• The clip from Time Flight is 576i/50, MPEG2, and video-mode (i.e. natively captured as interlaced fields). It highlights the absence of diagonal filtering when decoded in hardware. Watch the control console between 00:10 and 00:15, and between 00:34 and 00:39 - note the jagged edges and “crawling” effect along the edges of the metallic strips. Note also the constant flickering at the top left corner of the screen.

Here are some screenshots, although the effect is far more visible in motion. (Ignore the moire-like effect, that’s a camera artefact).

With filtering:

No filtering (note jagged edges on metal strips):

---

• The 480i wedge pattern decoded in hardware gives you a big moiré pattern in the horizontal wedge, centered on the 240 mark. This indicates that it’s being deinterlaced as video instead of film.

What horizontal wedge should look like:

What you see on screen:

---

Potentially significant, but not regressions from November behaviour:

• There is a weird scaling issue that affects the 480i wedge clip if it’s decoded in software (with 480p whitelisted). Set scaling to Nearest Neighbour and watch the vertical wedge on the left - every now and again you see a column of distortion sweeping across it from side to side. (You can somewhat mask the effect but setting scaling to Bilinear. But if the video is 480i and the output is 480p, why is there any scaling going on?) This is a longstanding issue, not a regression since November; but it is more relevant now than it used to be, because it means you cannot work around the bugs in hardware deinterlacing of MPEG2 simply by switching to software decoding.

What vertical wedge should look like:

What you see on screen (with distortion moving across pattern):

---

• Decoded in hardware, with deinterlacing set to Deinterlace or Auto-Select, the 1080i/60 wedge clip is deinterlaced as video rather than film. (Big moiré pattern around the 540 mark). With Deinterlacing set to Off it’s deinterlaced correctly. (It’s also handled correctly in software, but this isn’t practical for real-world VC-1 videos).

What it should look like:

What you see:

---

Other issues that already have their own dedicated threads:

• See here and here.

[joakim_s]

I understand that having this issue open with a thread feels good, that a locked thread doesn’t leave room for discussion. But just because the thread is locked doesn’t mean it isn’t on the todo-list and if further discussion space is needed to solve or discuss the issue, perhaps it is better to let the developers take the initiative?

Multipost about the same subject is frowned upon, as you know. And posting multiple issues in one post, is a “forum-rookie” mistake, that I wouldn’t expect from a veteran as yourself.

Why?

Multi-thread is not only forum etiquette, but lots of the issues posted will have multiple users sharing their experience, which is a goldmine when tracking down a bug. And since this is a small dev-team, with limited resources, we do leave stuff on the forum, to reference in the bug/todo-list. If all info on ONE issue is spread to 3 overlapping threads, there is a bigger risk of info getting lost. EG. longer dev/bug hunting time.

Conclusion, NO NEED for ‘reminder posts to Dev’s and keep it to one “NEW” issue per thread.’

There are a lot of other things needed to be taken care of before, what I feel is, corner cases. When there is time, there will be work on these issues.

[Chillbo]

@joakim_s

EG. longer dev/bug hunting time.

Or to put it in another way: less motivation to look at an issue in the first place as it’s become too confusing before any fixing has even been done.

[angry.sardine]

I hear what you’re saying. I felt it was worth bumping the topic because (as I mentioned before) when I previously raised all this, I was under the impression that a lot of these issues were long-standing things, and consequently they were not something it made sense to focus on when trying to get a release out; and I said as much at the time.

It turns out I was wrong about that - I installed the November 2020 release for a while yesterday, and was surprised to find that quite a lot of this stuff was actually working correctly back then. So I wanted to correct my mistake. Obviously I would normally add a new post to the existing thread saying “turns out I was wrong about that, and these are actually regressions.” But with the previous thread locked, starting a new thread was the only way I could get that across.

@joakim_s

corner cases

With the greatest respect, “deinterlacing not working as well as it used to” isn’t exactly a corner case. I can’t be the only user who watches DVD remuxes.

[joakim_s]

@joakim_s

corner cases

Since I’m read rather specific conditions, 1080i/50 and 2 decoders + 576i/50 MPEG2 I just assumed it was a minor issue, comapred to Vero4 not waking up after long time idle or newly solved back-up functionality. Have more impact on the masses, that doesn’t mean your issues is less important, but perhaps not as “General”, ergo corner case.

[ac16161]

just a word of support for @angry.sardine , while there seem to be few people on the forum who draw attention to the Vero’s deinterlacing problems, the amount of content that his impacts is huge (large amounts of my TV mkv collection is SD or HD progressive content that has been encoded at 50i). I never use a Vero for reference PQ playback of my HD 50i progressive content, as other devices can output a pure “weave” of the fields, though there was some success with VC-1 a while back, but it sounds like that might have regressed, and meanwhile h.264 50i now dominates.

To be fair to the Vero it’s not alone, but if this is fixable that would be pretty cool.

[joakim_s]

@ac16161

if this is fixable that would be pretty cool

Agreed, but my main point was: Reposting the info across multiple threads and combining 3 issues, which all have a common nominator, interlaced, is something I understand, But I tried to explain, that it is just one issue, that is rather specific, among a myriad of more generic core stuff, which often takes the upper hand.

I mean the backup issue for one was fixed, but stole time. Dev-time that got lost, because of a sync problem, where users reported correctly that it wasn’t solved, but we knew it actually was programmatically, Same dev-time gets “wasted” hunting all information concerning an issue when there is multiple threads, and interlinking between them.

Since I personally am rather blind when it comes to “fine details” like macro blocking, I value this issue a lot lower than getting a 24/7-on device to not hang when idling a long time. Or an advertised function like built-in back-up working. Or making sure that the newest version of OSMC actually can use modern encryption library’s. I can go on with a lot of stuff that, in my mind should and often has priority over fine tuning a working, but nor perfect playback of a certain video format.

[grahamh]

Meanwhile, on the topic at hand:

@ac16161

SD or HD progressive content that has been encoded at 50i

This is one of things I’m struggling with. Sorry for my ignorance, but can someone explain why progressive content would be encoded as interlaced? In fact what does ‘50i’ mean in this context? Mediainfo says the VC-1 is ‘interlaced’ but Sherlock is ‘Interleaved’. Does each field have a different timestamp (dictionary definition of interlaced) or are we talking about essentially 25p with each frame transmitted as two fields for some reason? What exactly is the process these videos have gone through and are we sure the metadata correctly reflects that? I would suppose the BBC material (and all of this stuff seems to be from UK TV) from 30 years ago would be taped as 50i and later converted to 50p for Blu-Ray. More recent episodes would have gone direct to disk, maybe as 50p. Perhaps @Steve_Neal can advise.

Then I have one Dr Who clip (I think from @ac16161) that’s 60i. What parentage would that have?

@ac16161

To be fair to the Vero it’s not alone

Quite. If a clip is tagged as interlaced when in fact it’s progressive frames formed of two fields I think chipmakers can be forgiven for being confused.

I hope someone can shine a light on this for us.

[angry.sardine]

@grahamh

can someone explain why progressive content would be encoded as interlaced?

I think the short answer to that is that, if you go back a few years, most TVs and blu ray players didn’t recognise 1080p/25Hz as a valid HDMI mode. The standard modes in those days were 1080p/24 (or 23.976), 1080i/50 and 1080i/60. So, if you had something that was intended for broadcast in 50Hz countries, it was broadcast as 1080i/50; and quite a number of blu ray releases of HD shows made by the BBC, etc. were also released as 1080i/50 discs. I’m not sure the blu ray standard even supports 1080p/25 discs - I’ve certainly never seen one, as far as I can recall.

You actually still have the same problem with broadcasts on (say) Sky or Freeview - HD broadcasts are still in 1080i/50 format rather than 1080p/25.

In terms of what it means, that depends on the material. Just as with DVDs, sometimes the material was captured as 1080p/25 and then stored on the disk as 1080i/50 by splitting each frame into two fields; sometimes it was actually captured as 50 native interlaced fields per second. My 1080i/50 test clips are all cases where the material was originally captured as progressive. But if you were dealing with something like a live TV broadcast of a talk show (recorded in studio) that would quite likely be interlaced at source.

@grahamh

Then I have one Dr Who clip (I think from @ac16161) that’s 60i. What parentage would that have?

This would only be a guess, but if it’s Doctor Who it was most likely originally 24p and then telecined with a 3:2 pattern of repeated fields. It’s the same process as creating an NTSC DVD based on a cinema movie. My 1080i wedge clip is in that format.

@grahamh

If a clip is tagged as interlaced when in fact it’s progressive frames formed of two fields I think chipmakers can be forgiven for being confused.

In an ideal world that’s why it would be nice to be able to control the deinterlacing strategy manually, as you can on the Pi 2 and 3 under Leia. But that’s a nice-to-have for the future; I imagine the priority now - if you can spare any time for it at all! - would be making sure everything works as well as it did in the November release.

[ac16161]

@grahamh

Then I have one Dr Who clip (I think from @ac16161) that’s 60i. What parentage would that have?

This would be the extract from a David Tennant special that the publisher released as a butchered 60i conversion on bluray (after the negative feedback, they released in 50i for most blurays after that). On the Vero, this 60i conversion (VC-1) plays with intermittent blocking artefacts that essentially make it unwatchable, whereas a Pi 3/Leia with VC-1 key plays it cleanly. This is a real corner case though I’d suggest.

[Chillbo]

@angry.sardine

I’m not sure the blu ray standard even supports 1080p/25 discs - I’ve certainly never seen one, as far as I can recall.

Nope, it doesn’t. Probably one of the only reasons this kind of material still gets released. Even the newer BBC Earth releases, produced at 50p, have to be put on BD disc at 50i, simply because there’s no alternative.

They’re being put on UHD BD as 24p wich supports 50p, but that’s a whole other topic. What it shows though: Things can also be messy, even if standards would allow for a reasonable approach.

[Steve_Neal]

@grahamh

This is one of things I’m struggling with. Sorry for my ignorance, but can someone explain why progressive content would be encoded as interlaced?

In this context it means that the material was shot natively progressive - say at p25 - but that it is being carried in an interlaced - say i25 - signal.

Broadcast video in Europe is usually either interlaced 576i25 or 1080i25, or progressive 720p50 (with some 1080p50 and 2160p50 knocking around).

UK broadcasters master their HD p25 and i25 productions to 1080i25 files for delivery (replacing 1080i25 HD tape).

Native i25 interlaced content and progressive p25 content both retain their distinctive 50Hz and 25Hz looks in the i25 signal - it just means that the transmission chain operates in a single standard. In fact there is actually a broadcast production standard to carry 1080p25 in a 1080i25 signal called 1080psf (progressive segmented frame) - which was a way of allowing 1080i25 production gear to work in 1080p (and for things like monitors to display pictures without requiring compatibility with another signal standard)

Most broadcasters require rolling or crawling credits on otherwise p25 shows to be rendered in the more fluid i25 domain to avoid judder on faster motion. These days rolls and crawls on high-end p25 drama and documentary have often been replaced by static cards that cut or fade - so that is a more moot point.

HD Blu-ray - annoyingly - didn’t include 1080p25 as a mastering format - for HD you have a choice of 720p50 or 1080i25 for HD in 50Hz-land. No 1080p50, no 1080p25. The only 1080p formats supported on Blu-ray are either 1080p24 or 1080p23.976 (essentially the same standard).

This means that if you have 1080p25 European TV drama edited 1080i25 - you release it on Blu-ray as 1080i25. Even if it was edited 1080p25 (or can be) - you can only release it in 1080p if you slow it down to 1080p23.976/24 (which some releases have done - like Downton Abbey S1) If you want to keep it at the same speed, you have to convert from 1080p25 to 1080i25.

In fact what does ‘50i’ mean in this context?

It means the video that has been mastered in VC-1, h.264 or MPEG-2 for Blu-ray was fed into the encoder in 1080i25 format.

Mediainfo says the VC-1 is ‘interlaced’ but Sherlock is ‘Interleaved’.

Interleaved usually refers to the processing of the interlaced video as a video frame (i.e. taking both fields into account when encoding the frame) Some codecs, like h.264, also have a ‘separate field’ encoding option which encodes each field independently and separately (almost treating the signal like 540p50). Encoding with interleaved fields usually offers some compression efficiency as interlaced compatible codecs (like h.264, MPEG2 and possibly VC-1) will look at both fields in each frame and in some cases (MBAFF in h.264 for instance) will analyse each macro block in the frame for motion, and if they see none, they will encode that block as a progressive block rather than as two separate fields. This improves quality and efficiency. It can mean that 1080p25 carried in 1080i25 then is encoded with all the Macroblocks in ‘progressive’ mode - until say a 1080i25 rolling caption appears.

Does each field have a different timestamp (dictionary definition of interlaced) or are we talking about essentially 25p with each frame transmitted as two fields for some reason?

Each field will still potentially be capable of being signalled as being 1/50th second apart - though with interleaved encoding it maybe taken as implicit, whereas with separate field encoding that may be explicit.

This situation is almost certainly p25 video being carried as i25 video - with no motion between the two fields in each frame (and encoder compression exploiting this) as they are sourced from the same p25 source frame.

What exactly is the process these videos have gone through and are we sure the metadata correctly reflects that?

Depending on the production process.

1. Shoot p25 (recorded either p25 native or as i25 with a 25psf signal within that i25 signal)
2. Edit p25 or i25
3. Master to i25 for delivery to most UK broadcasters - but potentially archive to p25 and slow-down to p23.976 for US broadcast delivery (or deliver p25 for them to slow down)

Blu-rays could either be mastered from an i25 master, or they would take a p25 master and convert it to i25 during mastering (or just before)

Sherlock will have been broadcast on BBC HD outlets in 1080i25 (*), it will have been released on Blu-ray in 1080i25 (probably from the same master as broadcast - or very similar). It was shot p25, and may have been partially edited p25. It will have been delivered to the BBC in i25.

I would suppose the BBC material (and all of this stuff seems to be from UK TV) from 30 years ago would be taped as 50i and later converted to 50p for Blu-Ray. More recent episodes would have gone direct to disk, maybe as 50p. Perhaps @Steve_Neal can advise.

Taping native interlaced content at i25 will have 50Hz motion. Film content transferred to i25 tape will have 25Hz motion. BBC TV productions have used i25 since 1936

All BBC HD deliveries are in i25 - with native i25 stuff (or p50 stuff converted to i25) having 50Hz motion, and native p25 stuff having 25Hz motion. The look is preserved.

BBC UHD deliveries currently can be both 2160p25 and 2160p50 I believe (the former is used for drama and documentary, the latter for live sport)

UHD Blu-rays include support for 1080p50, 2160p25 and 2160p50 I believe - so BBC UHD releases don’t have the i25 issue. (Not that it really IS an issue)

(*) BBC HD on Freeview HD uses some clever optimised encoding that dynamically switches the actual h.264 encoders from 1080i25 (with MBAFF progressive macro block encoding) to native 1080p25 (it looks for any motion between the two fields - and if it sees none, it encodes at p25) p25 encoding ekes out a bit more efficiency in the encode.

Also for the absence of confusion :
1080i25 = 1080/50i
1080p25 = 1080/25p

[Chillbo]

@Steve_Neal

UHD Blu-rays include support for 1080p50, 2160p25 and 2160p50 I believe - so BBC UHD releases don’t have the i25 issue. (Not that it really IS an issue)

But BBC nevertheless doesn’t to use the capabilities of the UHD BD format all the time… E.g. BBC Earth productions are mostly released as 24p on UHD BD despite being native 50Hz productions AFAIK. Go figure

[grahamh]

Well, that should cover it thank you! And for giving us the proper vocabulary.

What I understand from that is that in converting from p25 to i25 there is no temporal interpolation involved. That’s seems to the case with the VC-1 clips we’ve been working with. But there could be ‘progressive’ (p25) and ‘interlaced’ (50 fields with 50 timestamps) mixed in the same stream (eg for rolling credits).

[Steve_Neal]

US TVs don’t handle 50Hz. The market for UHD Blu-rays is getting smaller and smaller - I suspect the cost of mastering a p25 title for 50Hz markets and a separate p23.976 title for 59.94Hz markets isn’t economically justifiable, whereas a single p23.976 release will work globally.

Don’t ask me why US mainstream TVs still don’t handle p25 or p50 inputs… Reality is most mainstream TVs (Sony, Panasonic etc.) don’t…

[Steve_Neal]

Yep - p25 to i25 involves no alteration of the look - though there may be some vertical filtering to avoid interlace twitter (due to the V/T spectrum folding of interlacing)

V/T = Vertical/Temporal. Interlacing effectively trades off vertical resolution for temporal resolution - and there is some content in the vertical and temporal domains that can thus be identical (fine vertical detail vs slow vertical motion)

[Chillbo]

Yup, that’s what I thought… Interestingly, it’s not at all a hardware limitation or anything close to it. The exact same TV models, or at least differently named ones extremely similiar or the same on a hardware level here in Europe or anywhere else in the world do support those modes. It’s just a strange software choice for no apparent reason (at least not one I see or understand).

What’s making it even more strange: The BD discs in those very UHD BD releases are mostly mastered at 1080i50 - even in the US. Explain that to me

It’s really sad as the standard now supports this and it could easily be released at a much better quality - it would also deliver a much better experience, especially for nature documentaries.

[grahamh]

@Chillbo

Explain that to me

Isn’t this connected with the crumbling region system? Besides, I think the newer US TV models are supporting 50 and 25Hz at least at 2160p. That’s why we patch Kodi’s whitelist code to use those if available (but couldn’t convince Team Kodi it was a good idea).

[Steve_Neal]

No - it’s totally separate from the regional system. There is no regional restriction at all on UHD BD.

In Europe ‘HDTV’, ‘HD Ready’ and ‘HD Full 1080p’ logos and licences mandate compatibility for both 50 and 59.94/60Hz to meet the licensing criteria to carry that logo and be licensed. There is no such equivalent in the US.

The reality is that most TVs since the 80s sold in Europe would lock to both 50 and 59.94Hz sources - and VCRs with NTSC playback (often using PAL 60 to increase the chances of colour replay) were in widespread use - in part to allow home movies shot on rented NTSC camcorders in the US to playback when you returned home. There seems to have been less of a reverse market for this.

Region 2 for DVDs contained both 50Hz (Europe) and 59.94Hz (Japan) - but often Japanese players from mainstream manufacturers wouldn’t play 50Hz discs properly, and TVs wouldn’t accept them. For this region a lot of Region 0 (i.e. not regionally locked) DVD music videos are 59.94Hz releases internationally (even if the material on them was shot 50Hz)

It does just seem to be that 50Hz compatibility for North American mainstream TVs is not a major issue - whereas lack of 59.94Hz in Europe would be. (PCs and Laptops output 60Hz VGA by default, as they do over HDMI/DVI, now most streaming boxes default to 59.94Hz output etc. etc.)

[Steve_Neal]

I don’t think MANY people would notice the difference between a p25 and a p24 slow-down (particularly if it’s been properly pitch corrected).

None of this material is shot for p50/i25 native 50Hz motion broadcast - all the mainstream BBC Natural History stuff is shot for p25 (anything shot at higher than p25 frame rates is for over-cranked slow mo - not for p50 broadcast)