Update Library crashes with "Sad face"

Hi,

I like to turn on all the information and art scraping I can get to have a better choice for my shows and movies.
I have been experiencing crashes on Kodi 19.3 that did not happen with Kodi 18.x and MariaDB 10.5: turns out upon scraping:

  • c06 and maybe c11 columns of the tvshow table, as well as
  • c08 and maybe c20 columns of the movie table

have to be modified from TEXT to MEDIUMTEXT type, similar to this:
ALTER TABLE MyVideos116.tvshow MODIFY COLUMN c06 mediumtext DEFAULT NULL NULL;
ALTER TABLE MyVideos116.tvshow MODIFY COLUMN c11 mediumtext DEFAULT NULL NULL;
ALTER TABLE MyVideos116.movie MODIFY COLUMN c08 mediumtext DEFAULT NULL NULL;
ALTER TABLE MyVideos116.movie MODIFY COLUMN c20 mediumtext DEFAULT NULL NULL;
(I have actually used HeidiSQL for this)

First BUG: even if some or many SQL UPDATEs fail OSMC should not crash with a sad face, but handle the errors gracefully. This is a MAJOR stability issue, my updated Vero 4k+ crashes multiple time, used to be much more stable on the older Kodi version.

Second BUG:

  1. At a bare minimum above differences and how to hand-patch them should be part of the OSMC knowledge base ([Wiki] Install and configure mysql server and clients)
  2. Quick and dirty: OSMC could carry a patch after DB creation if DB is MySQL (MariaDB). This way I would not have to re-patch these tables every time there is a DB version upgrade (which I assume also truncates these fields upon upgrade. Which may cause problems because URLs are parsed out of these i believe… :man_facepalming:
  3. One better would be to handle these differences in the back-end as some Kodi forum posts suggest it, and try to upstream this change.

Even after the fixes I see significant CPU and memory use during scanning and eventually get a “Sad face” during library update (maybe some other errors are also not handled gracefuly?):


I have turned on debug logging and waited till the crash, here are the logs: https://paste.osmc.tv/atuqapiwej

This is all dependent on Kodi’s code responsible for the database. OSMC is not at is not responsible for it’s function or any bugs. I’d you wish to request the issues be addressed by the maintainers of the code responsible then you should report your issues and findings on the Kodi forums.

We believe this is a Kodi issue or a specific Kodi function and as such you should look at addressing the issue there. If you do believe that this is an OSMC specific issue, please let us know.

Excuse me?
1: this is not (AFAIK) an open source project where you can just point me at upstream: you did take an open source project for free (AFAIK) but you sold me your hardware-software combined multimedia player product for profit! (actually two of them)
2: you have modified Kodi from the gui to the deepest layers, how shoud I know where is it wrong? I just see the complete box freezing up rather often. Go, figure, if it’s any of the components you chose to incorporate - fix it or use something else.
3: I have actually provided a fix for the database related problem. You shoud carry it take it for integration to your vendor. Not that you say your product does not work with mySQL / MariaDB: currently the opposite is documented.
4: as I (a) descrcribed (b) screenshot and (c) provided debug logs for: even after the DB updates are fixed something else is still going wrong when updating the library, taking kswapd to 90%+ CPU and then the complete system ends up sad face. Please advise how to try to fix it or what additional information do you need?

Thanks, Sape

Hello,

MySQL support in Kodi after a decade is still considered to be experimental and is not used by default out of the box.

Which scraper are you using?

Do you have a link to such a post so we can check this?

Regarding the crash from your logs. The device seems to be running out of memory and Kodi is terminated due to OOM

Feb 04 10:24:47 osmc2 kernel: Out of memory: Kill process 3522 (kodi.bin) score 594 or sacrifice child
Feb 04 10:24:47 osmc2 kernel: Killed process 3522 (kodi.bin) total-vm:1850464kB, anon-rss:1208340kB, file-rss:0kB, shmem-rss:0kB
Feb 04 10:24:47 osmc2 kernel: oom_reaper: reaped process 3522 (kodi.bin), now anon-rss:4kB, file-rss:0kB, shmem-rss:0kB

You have a number of autofs shares. I’m wondering if that could be a cause.

Let’s focus on the OOM, I will dig up the DB fix suggestion tomorrow.
Yes, I have many file systems and the NFS paths are substituted with local paths mounted by autofs.

But how could this accumulate memory used in kodi.bin, growing up to use 1.2-1.4GB RAM?

What kind of memory leak in kodi.bongets heavily activated during library updates but otherwise let the player play videos all day?

Any chance there is no proper free-up after the many failed curl requests? Or the failed thumbnail generations?

Which scraper are you using?

I am surprised this is not visible from the logs. The UHDmovies-J source that I scanned for new content to reproduce sad face by OOM uses Universal Movie Scraper.
It always scrapes a few more movies then dies again.

If your using a MySQL db then checking Kodi’s forum and github should be your first stop when your getting db related error messages.

I don’t understand the relevance of this link @darwindesign: even in my first message I coped the alter table fixes frome this link, except I also added two other columns that may overrun.
But at least you dug up the sentence I referred to and @sam_nazarko asked about:

@yol yol commented on 19 Mar 2019

You can rewrite the SQL in the DB backend. This is already done for other small differences between MySQL and SQLite.

We were already past this with @sam_nazarko yesterday, but let me state it for the third time: the SQL UPDATEs are fixed by the table alterations I have already provided, but the OSMC box STILL goes OOM and kills the runaway process, that is kodi.bin, even after the DB is fixed.

If you had read through the entire bug report you should have seen that the Kodi team are aware of the issue and have decided to NOT alter the table sizes but instead address it on the scraper side such that they don’t try to dump an excessive amount of data in a field in the first place. You should have also seen that this was addressed with the TVDB and TMDB python scrapers only so if you have an issue with a single show/movie that is causing a scraping issue the fix would be to…

  1. Use the TVDB or TMDB python scraper for at least the problematic show/movie

  2. Use some program like Tiny Media Manager to make a nfo file that doesn’t have the excessive amount of links causing a problem.

  3. Take the issue up with the maintainers of the Universal Movie Scraper add-on since that is where the Kodi team decided this issue needed to be addressed.

What you shouldn’t do is press a downstream distro to make a change to an upstream project that runs counter to how the upstream project has already decided to address the issue.

It may also be possible that if you scrape on a PC this may get around your problematic show and the memory issue since you have already altered the table size.

2 Likes

There are two fundamental misunderstandings here:

  1. The DB column lengths are no longer causing any errors. And update library still runs into Sad Face. That is the unsolved issue to focus on. Do I really need to create a new topic for you to stop dwelling on the resolved errors? Any idea what may still be causing the OOM, after there are no more SQL errors? Is there any way to profile the memory usage of kodi.bin to pinpoint which component is ballooning? What is not releasing the memory after each movie scraped?
  2. OSMC sold me two Vero 4k player products (hardware with software) it is not an opensource distribution that can just say go and bother the upstream projects (BTW including choosing the hardware memory to accommodate running the htpc software of your choice). Or do you mean to say they purchased product has no support and OSMC is an independent project?
  1. You identified the issue as you scanning in content that has more information that what Kodi was able to handle without issue. It seems that now now assume that because you altered your db to handle more info that this information will not cause any issues with any other part of the scraping process. Change your scraper to one of the python ones and see if the out of memory issue goes away. If it does then you have your answer.

  2. Obviously OSMC provides support for things that are under their purview. The Vero is a piece of hardware that runs an open source OS designed mainly to run a particular piece of open source software (Kodi). Like any other project of this type there are different groups of people who maintain different pieces of the system. In the case of OSMC the parts of Kodi that are tweaked are mostly to do with audio and video playback as they are platform specific. The parts of Kodi that deal with the database and scraping are not being modified by OSMC at all and are the exact same code that is being used by Kodi on every other platform.

  1. That is incorrect: it is not Kodi that had issue with “content that has more information”. Instead it is MariaDB that was configured to not be able to fit into the designated column. Kodi, with the same column type (TEXT) on SQLite handles the same amount of information just fine (See Implementation Limits For SQLite → 10^9 bytes). So can we rather focus on the Kodi OOM please?
    Is the Universal Movie Scraper (by Team Kodi BTW) not working / not supported in OSMC? If so why not disable / blacklist / warn about it at least?
    BTW is this scraper not using python? What is the difference, can I read up on why others are preferred by OSMC? (My current preference is just based on which scrapes the most information, actors, images, fanart, ratings etc. for a given movie / TV show.)
  2. If OSMC is an opensource project, can you please point me at your open source (githut etc.)?
    I still am reasonably sure I did purchase my Vero from OSMC: Store - OSMC - if this is not the forum to ask for support for it (because this is only for the opensource OSMC project…) then please point me to the Vero4k+ support!
    What exactly do you mean by the “purview” of OSMC? Kodi is one of the many (although central) component provided in every version of the OSMC download packages. Why any part of that package is not in your “purview”?
    Also, OSMC does not seem to be limited to be tweaking “mostly audio and video playback” as it has a complete customer UI, config, update, log-grab etc. so I still think you do touch everything from the UI down to the drivers, but I will check once you point me to the OSMC code.
    This way I can also take a look at the DB and scraping code of OSMC, although even if you do not change it I assume there can be interactions with other parts of the code (say thumbnail generation) that can still OOM.
  3. Do you have a list of all the things that you feel like not supporting? MariaDB? Universal Movie Scraper? All Kodi issues? Anything else?
  1. Change the scraper to TMDB for the source and see if the issue goes away. If it does and you want to go back to using the Universal Movie Scraper then you would need to bring up the issue on Kodi’s forum or via a bug report on their Github. The “python” name is just how they call out the distinction between the old versions of the scrapers and the new ones. There is no scraper that is “preferred by OSMC” as that is not something that OSMC touches. I think that currently the Universal Movie Scraper is a bit more prone to problems than the others.

  2. OSMC is on Github just like Kodi is. OSMC is an OS that runs Kodi. Kodi is a separate entity. The custom UI is just a skin (add-on) and isn’t a modification to Kodi itself and the customer can use any skin they want that is available for Kodi. The only parts of Kodi’s config that are not using Kodi’s stock have to deal with platform specific audio and video. The Kodi part of updates, which are add-ons, are coming from OSMC but only as a mirror service to take the load off of Kodi’s servers (binary add-on’s excepted as they are platform specific).

  3. MariaDB is currently the recommended MySQL backend according to Kodi’s wiki but as already pointed out earlier in this thread Kodi has always considered MySQL to be experimental and it has never gotten much love. That being said I have been using Kodi this way for over a decade and issues have been few and far between. The OSMC devs didn’t write, and don’t maintain the Universal Movie Scraper so no, OSMC doesn’t provide support for it. There does exist the people who did, and do, and you are free to discuss your issues with them on Kodi’s forum. Generally speaking non platform specific Kodi issues should be taken up over at Kodi. The community on this forum will frequently try to assist people with Kodi specific issues on this forum anyway when it is feasible. This often is just a matter of providing workarounds or tweaks to configurations so there is no need for involvement of any programmers. When it comes to something like a particular scraper having an issue and the person wanting that particular scraper to stop having that issue then that is something that the person/s who are maintaining said scraper need to be contacted about.

OSMC stands for ‘Open Source Media Center’
You can learn more here: About - OSMC

and sources here: https://github.com/osmc

It is unchanged and comes entirely from upstream.