I am not doing anything manual during scan. Library Watchdog polls nfs shares for new files.
I always run clean library after deleting files.
To see if the DB is the issue, how can I clean the DB of deleted items? I thought the clean library function in Kodi would do that.
Exactly as you stated: Using Settings -> Media -> Library -> Video Library -> Clean Library should do the job.
Also interesting is that all these orphan db entries, the VideoInfoscanner is hunting, are folders ā¦ not video files. So it looks like you organize your movies/tv shows in separate folders for each item.
I have run clean library multiple times
The folders it is showing in logs do not exist in my nfs share.
So looks like clean library is not correctly removing entries that no longer exist.
MySQL works the same as SQLite in this regard (and pretty much every other). If you update a nfo file for a particular title you can just bring up the info window and click refresh. You can do this on an entire season or series as well if you like. The only exception Iām aware of to this is with movie sets as Kodi currently lacks a function to update a movie sets info once it has first been added. In this case the best bet is to highlight the movie set and remove it via the context menu and also say yes to removing the movies in the set. If you only remove the set then the movies become unassociated. By removing the movies with a library update they will get rescanned again with updated movie set information and the movies included in the set.
If the file path is not there they get cleaned from the library BUT for some reason these entries are ones that got an individual path entry in your database and Kodi does not clean that up. Those entries are not adding any notable time to your scan and pretty much are just a debug log noise. You can manually delete these entries from the database if it bothers you.
As for the issue at hand I have two thoughts . The first is are you updating the same database from more than one device? Iāve noticed that if I only update a database from one device then the scan time is normally fairly consistent, but if I update from a second device that scan takes significantly longer the first time. If I update next from the second device again then it is much shorter and will continue to be this way until I update from another device again, which will be slow the first time on that one. Iām guessing the multiple device thing screws up Kodiās mechanism where it can skip most folders it knows hasnāt changed.
My other thought is have you looked into optimizing MariaDB? Itās been a while since I set mine up, and Iām not particularly knowledgeable on the topic, but Iām pretty sure there are some tweaks to be had. Itās been years since Iāve touched this but maybe compare your my.ini to what I have to see what differences there are in the options. I have a vague recollection of the name resolve option helping with speedā¦
[mysqld]
datadir=C:/Program Files/MariaDB 10.4/data
port=3306
innodb_buffer_pool_size=2G
character-set-server=utf8
skip_name_resolve=1
optimizer_search_depth=0
#key_buffer_size=16k
#host_cache_size=16
# Query Cache Configuration
query_cache_limit=2M
query_cache_size=32M
# InnoDb optimizations
innodb_buffer_pool_chunk_size=1G
innodb_buffer_pool_instances=2
#innodb_adaptive_hash_index=OFF
innodb_buffer_pool_dump_at_shutdown=ON
innodb_buffer_pool_load_at_startup=ON
[client]
port=3306
plugin-dir=C:/Program Files/MariaDB 10.4/lib/plugin
Iāve tried various things to reproduce this scenario, but without success. The āClean Libraryā function deletes all entries for me when I organize movies in separate folders ā regardless of whether I delete the movie first and leave the folder untouched, or directly delete the folder containing the movie.
Ideas to get this cleaned:
Go to Settings -> Videos -> Files and Remove Source (and delete all related data in the db) followed by an Add Videos. This will re-scrape your whole file content on the provided NFS source. I think you have to do it for all three sources Downloads, Movies and TV. Make sure that the other OSMC devices are shut down while this cleanup/re-filling is running.
An even more drastic way is to drop the mysql/mariadb database:
2.a power down the OSMC devices
2.b login into your db system, there start a cli tool for the db like mysql -u root -p
2.c locate the database name using SHOW DATABASES; (typically it has the form MyVideosXYZ)
2.d DROP DATABASE <your found database name in 2.c> ; \q
2.e Restart one of your OSMC devices (it will automatically recreate a virgin video media db on the db system)
2.f Goto Videos -> Files and set again the right content in the context menu of the specific source and let re-scrape all the files.
2.g restart the other OSMC device(s)
Later you should follow @darwindesignās idea that toggling between the OSMC device could be the reason for the extended runtime using Update Library on your MySQL db. To test this, simply run Update Library twice in quick succession on the same OSMC device. If the second run is significantly faster, it confirms the theory.
I donāt even want to think about what would happen if multiple devices started making changes to the database simultaneously without any mutual exclusion mechanism in place. Itās really not a good idea to assign multiple devices to maintaining data other than the watched and position information. Only one device should handle the task of keeping the movie data uptodate or you have to be sure that there will be no overlapping runs of Update Library among the OSMC devices.
Nope only the Vero V is connected to dB. My pi is backup device now. So only one device updating library.
As other poster said itās not adding time to scan so for now I will leave it as it is and when I get time I can delete db and re create.
However as I mentioned above only one device connected to dB. Maybe the Vero not connecting at full gigabit speed on Ethernet.
Maybe something changed in the network, we suggest to check your network with iperf3. Please read this howto
Iām not really sure why file paths sometimes get left behind in the db, but it is a thing that happens. Iāve seen it in mine many times (Iāve been running Kodi with sql for decades at this point) and Iāve seen it in others. It really isnāt an issue that one needs for concern themselves with other than to cure an OCD problem.
My preference is to use HeidiSQL on a PC to remove just the offending bits. Itās free software and usually a lot faster than reconfiguring scrapers and doing a full library scan.
its running at gigabit so its not that,
so no one here knows why my scans are running slower then my Pi?
My V does library updates slower than my RPi 4 (connected to the same switch and MySQL library) so I would kind of expect itā¦ but only a fairly minor difference. I donāt remember the difference as itās been too long since I checked it, but I donāt remember it being any more than taking a number of seconds longer. It would not be expected for a library update without any new content to take ten minutes to run. I have a pretty decent sized video library and it usually clocks in around 90 seconds.
The only things coming to mind is maybe try disabling the watchdog and see if manually updating goes faster (just spitballing, Iāve no experience with this add-on) or maybe try some different settings for your network connection. You could Try different NFS versions in Kodiās settings or you could also try a system mountwhich should be more performative than having Kodi handle the networking interface.
