Video Scraping slow & hangs on Synology mounted shares

Help and Support
1

I seem to be having some long hangs and delays when scraping my Movie and TV directories. The scan will run quite quickly through Movies then experience a fairly long delay (maybe 2 minutes) before starting to scan the TV directories and then an even longer hang at the end of the TV directory scanning before it finishes up.

This happens even when there is nothing new to be scraped.

The shared folders are mounted via NFS mounts in my fstab and the shared folders are on a Synology NAS that also has Plex installed and scans the same directories/libraries. I use OSMC for viewing at home and Plex for viewing when I’m not at home.

I have a couple of suspicions as to why this is an issue, but I’m not sure reading the log files. One possibility is I see a ton of files that get created in an @eaDir path which seem to be either plex or synology. Maybe OSMC is choking on these files? I’m not sure if they can be deleted or if there is a way to get osmc to just avoid that directory entirely?

Here’s a log from a scrape that I just ran that took about 7 minutes to run, but there was no new content. The longest hang was at the end of the TV scrape before it finished up the process.

https://paste.osmc.tv/palofehoci

2

the @eaDir folder is created by the Synology. I’m not sure why the Synology does that or what it’s for, but there is no way I’ve found to keep the Synology from creating them. You can exclude them from the scanning, which may help. I exclude them and am running OSMC on a Vero4K+ with all my media on a Synology NAS using NFS with a mount point, and I’m not seeing times as bad as yours.

In advancedsettings.xml you need to add this and then restart Kodi:

       <video>
                <excludefromscan action="append">
                        <regexp>@eaDir</regexp>
                </excludefromscan>
                <excludetvshowsfromscan action="append">
                        <regexp>@eaDir</regexp>
                </excludetvshowsfromscan>
        </video>

If you don’t already have an advancedsettings.xml, you can create one in .kodi/userdata/ and add this:

<?xml version="1.0" encoding="utf-8"?>
<advancedsettings>
       <video>
                <excludefromscan action="append">
                        <regexp>@eaDir</regexp>
                </excludefromscan>
                <excludetvshowsfromscan action="append">
                        <regexp>@eaDir</regexp>
                </excludetvshowsfromscan>
        </video>
</advancedsettings>
3 Likes
3

thanks for the settings to add to the advancedsettings.xml. I did add it and restarted kodi but it didn’t work out at my system (synology and nfs) as it did with yours, as the @eaDir still showed up. I found an example on kodi How to hide/exclude the @eadir folders which i tried to move in between your suggestion and that worked. it even stopped showing the recycle bin. :slight_smile:

   <video>
            <excludefromscan action="append">
                    <regexp>\@eaDir</regexp>
                    <regexp>\@eadir</regexp>
                    <regexp>\@EADIR</regexp>
                    <regexp>\#recycle</regexp>
            </excludefromscan>
            <excludetvshowsfromscan action="append">
                    <regexp>\@eaDir</regexp>
                    <regexp>\@eadir</regexp>
                    <regexp>\@EADIR</regexp>
                    <regexp>\#recycle</regexp>
            </excludetvshowsfromscan>
            <excludefromlisting>
                    <regexp>\@eaDir</regexp>
                    <regexp>\@eadir</regexp>
                    <regexp>\@EADIR</regexp>
                    <regexp>\#recycle</regexp>
            </excludefromlisting>
    </video>
5

@mtv: The construct can be slimmed down a bit: By using (?i) you can make the regular expression case insensitive like

        <audio>
                <excludefromscan>
                        <regexp>(?i)(\@eaDir)</regexp>
                </excludefromscan>
                <excludefromlisting>
                        <regexp>(?i)(\@eaDir)</regexp>
                </excludefromlisting>
        </audio>```
2 Likes
6

Since there is a few derivations here that might prove confusing for some here is what the entire file would look like. Note that the “append” modifier (despite what Kodi’s wiki seems to state) is not needed as excluding it does not remove defaults.

<?xml version="1.0" encoding="utf-8"?>
<advancedsettings>
	<video>
		<excludefromscan>
			<regexp>(?i)(\@eaDir)</regexp>
		</excludefromscan>
		<excludetvshowsfromscan>
			<regexp>(?i)(\@eaDir)</regexp>
		</excludetvshowsfromscan>
		<excludefromlisting>
			<regexp>(?i)(\@eaDir)</regexp>
		</excludefromlisting>
	</video>
	<audio>
		<excludefromscan>
			<regexp>(?i)(\@eaDir)</regexp>
		</excludefromscan>
		<excludefromlisting>
			<regexp>(?i)(\@eaDir)</regexp>
		</excludefromlisting>
	</audio>
</advancedsettings>
2 Likes
7

Thanks @JimKnopf @darwindesign @pkscout

I just added the following below in every entry to remove #recycle from listing/scanning

<regexp>\#recycle</regexp>

8

You would avoid this issue by not using the root folder of drives to store your files but if you do you would probably want…

<excludefromlisting>
	<regexp>System Volume Information</regexp>
	<regexp>\$recycle</regexp>
</excludefromlisting>
9

Thanks for the tip. Each share on my synology gets an recycle bin and i only have 1 folder were i have a subfolder were the movies are stored. But i’ll add this as well and see how it works

10

Those two exclusions I posted were for drives that had been plugged into a Windows PC at some point and video files were being stored directly in the root folder. I spaced for a moment that we were talking about a NAS. :man_facepalming:

1 Like
12

Thanks for this and everyone else who jumped into this thread. This has significantly improved my scanning time. I still get a little bit of a hang at the end scanning the TV shows, but it’s looking to be less than a minute. Previously, I was seeing hangs at 6-8 minutes

1 Like
13

This might be something we want to get included upstream as a default to improve performance.

What do you think @darwindesign, @JimKnopf?

Sam

14

If you mean to ask the Kodi dev’s to add a hidden blacklist, I’m not so sure. Things can unintentionally get caught in those. If you mean to ask the scraper dev’s if they should be trying to scrape hidden folders, sure.

15

I’ll raise it in the Kodi Slack. I’m sure it’s worth a discussion.

16

I think it should not be hardcoded so this cannot be modified by users anymore.
Putting it in /usr/share/kodi/system/advancedsettings.xml at least for OSMC devices would be fine … and is another important key differentiator, then :upside_down_face: .

17

ok, I’ll look at it all
THANKS