[HowTo] Autoedit – Fully automated DVR with commercial detection

Hello,

first of all thank´s a lot for your tut.

I followed all the steps and the installation part seems to work.

If I try to record sth. nothing happens. The output file won´t be touched. The task goes into the autoeditqueue. But than nothing happens.

Then I removed the --wait option in Tvheadend and it ends with an error

(Error while transcoding. The original video file is kept.
02.10.2018 10:45:01 : Aborting…
02.10.2018 10:45:01 )

I didnt add the cron cause I want to get the movies transcoded after recording. But they only get registered in the queue. What then?

Would be great if you could help me.

Please be patient of my bad English.

Greetings.

Hi,

the problem you describe is a little thin to pinpoint exactly.

If you want to transcode it is preferable to transcode after searching for commercials and at a time your system is not used otherwise. That’s exactly the reason I added the --wait switch and cron job. What you described above is the expected behaviour for --wait: Queue the file and do nothing else. The actual processing is done when the cron job starts (or you run autoedit manually).

„Error while transcoding“ indicates either your command for ffmpeg is broken or the file is not accessible. If I had to guess I’d go for wrong file permissions. Keep in mind tvheadend runs on it’s own user so it may not be able to access directories you created with your username.

Please provide:

  • a full log showing the error
  • the full command from tvh
  • your settings file
  • what system you are running

to show more details. Also file info (ls -Alh) of the source’s directory, the source file itself and the target directory might help.

Cheers
Fretzke

Hello,

thanks for reply. Sorry for my late reaction. My system crashed and I had to set it up completly new.

Openmediavault Arrakis is running on a Microserver from HP N40L.

Tvheadend ist installed ontop of the system, Emby is also installed.

I now try to get away from MySql cause of the issues with the latest Libreelec Milhouse Build and SQL.

So now I reinstalled everything on a clean system and here is the following:

TVH default profile post processor command : /usr/bin/autoedit --input "%f" --title "%t" --comskip --transcode peg2_mmal h264_omx 2000k --rename --wait

TVH movies post processor command: /usr/bin/autoedit --input "%f" --title "%t" --comskip --transcode mpeg2_mmal h264_omx 2000k --rename --movie --wait

The logs and settings:

autoedit:

06.10.2018 13:30:00 : New video recorded. The file name is /sharedfolders/Raid/Aufnahmen/TV/2018-10-06/rbb Berlin HD/Verrueckt nach Meer (114)/_ts .
06.10.2018 13:30:00 : Appending video file to queue.
07.10.2018 02:00:01 :
07.10.2018 02:00:10 : Start processing...
07.10.2018 02:00:10 : Fetched next file from queue. Processing /sharedfolders/Raid/Aufnahmen/TV/2018-10-06/rbb Berlin HD/Verrueckt nach Meer (114)/_ts.
07.10.2018 02:00:10 : Cannot move video file to temporary place. Missing permission?
07.10.2018 02:00:10 : Aborting...

ls -Alh ´root´

root@openmediavault:~# ls -Alh
insgesamt 276M
-rw------- 1 root  root  11K Okt 14 09:24 .bash_history
-rw-r--r-- 1 root  root  570 Jan 31  2010 .bashrc
-rw-r--r-- 1 root  root 6,1K Okt  6 12:27 comskip.ini
-rw-r--r-- 1 root  root  50M Sep 20 04:55 emby-server-deb_3.5.3.0_amd64.deb
-rw-r--r-- 1 root  root  50M Sep 20 04:55 emby-server-deb_3.5.3.0_amd64.deb.1
-rw-r--r-- 1 root  root  134 Okt  5 11:22 .inputrc
-rw-r--r-- 1 root  root 178M Okt  6 12:15 jdk-8u181-linux-x64.tar.gz
-rw------- 1 root  root  606 Okt  7 15:11 .mysql_history
drwxr-xr-x 2 root  root 4,0K Okt  6 12:27 .nano
-rw-r--r-- 1 root  root    0 Okt  7 13:43 netstat
drwxr-xr-x 2 root  root 4,0K Okt  6 12:17 .oracle_jre_usage
-rw-r--r-- 1 root  root  148 Aug 17  2015 .profile
-rw-r--r-- 1 root  root   66 Okt  6 12:32 .selected_editor
drwxr-xr-x 2 root  root 4,0K Okt  6 12:26 sources
drwx------ 2 root  root 4,0K Okt  5 11:22 .ssh
drwxr-xr-x 2 Thilo 1000 4,0K Jul  7  2009 tevii_ds3000
-rw-r--r-- 1 root  root 322K Nov 20  2014 tevii_ds3000.tar.gz
-rw-r--r-- 1 root  root  205 Okt  6 12:20 .wget-hsts

ls -Alh Target

root@openmediavault:/sharedfolders/Raid/Aufnahmen# ls -Alh
insgesamt 36K
-rw-rw-r--+ 1 root root     0 Okt  7 02:00 autoedit.lockfile
-rwxrwxr-x+ 1 kodi users 2,8K Okt 14 02:00 autoedit.log
-rwxrwxr-x+ 1 kodi users  154 Okt  6 13:30 autoeditQueue.txt
drwxrwxr-x+ 2 kodi users 4,0K Okt  6 12:31 Movies
drwxrwxr-x+ 3 kodi users 4,0K Okt  6 12:40 TV

settings:

##Change personal settings here:
#
#Change the encoding table to be used
export LANG=de_DE.UTF-8

#Path where the processed files will be stored. Filebot will create corresponding subfolders here.
outfolder="/sharedfolders/Raid/Aufnahmen/"

#default ini file for comskip if parameter not givven
comskipINI="/home/comskip.ini"

##Filebot settings:
#language
language="de"

#Naming format for series
#Standard is "series_title (year)/season.episode_title.file_extension"
#e.g. "Breaking Bad (2008)/S04E01.Box Cutter.mkv"
#Including capital letters and spaces, file will be stored in the same directory
nameFormatSeries='/sharedfolders/Raid/Aufnahmen/TV/{n.ascii().replaceAll(/[?]+$/).tr("–","_").tr("*","_").tr("|",("_")} ({y})/{s00e00}.{t.ascii().replaceAll(/[?]+$/).tr(":","_").tr("*","_").tr("|","_")}'

#Naming format for movies
#Standard is "title (year)/title.file_extension"
#e.g. "Rogue One A Star Wars Story (2016)/Rogue One A Star Wars Story.mkv"
#Including capital letters and spaces, file will be stored in a separate movie directory
nameFormatMovie='/sharedfolders/Raid/Aufnahmen/Movies/{n.ascii().replaceAll(/[?]+$/).tr(":","_").tr("*","_").tr("|","_")} ({y})/{n.ascii().replaceAll(/[?]+$/).tr(":","_").tr("*","_").tr("|","_")}'


#General settings
#–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

#Log file
autoeditlog="${outfolder}"autoedit.log

#Queue file
queuefile="${outfolder}"autoeditQueue.txt

#–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

syslog:

syslog

Thanks in advance.

Hi No3light,

in the meantime you may have already figured out by yourself, but your problem’s cause lies with the target directory permissions.

In settings.ini you define /sharedfolders/Raid/Aufnahmen/ as working directory. You did not include the permissions for this dir, but from the file list I conclude you use Linux ACL there, too. This means, while your user has access to this directory, the user executing the script may not.

I personally had a similar problem once caused by a Windows machine over Samba.

You can get the permissions straight with the getfacl and setfacl commands, but I can offer little help there as I’m no expert on these advanced settings.

I personally prefer to simply removing ACL all together with setfacl -bn <directory> as I have no need for this in my private and closed home network.

Cheers
Fretzke

Hey,

thanks so much for this tutorial.
But I have the following issue:

I am using OSMC on my RasPi3 and I want to use hardware encoding with h264_omx.
However the aspect-ratio of the encoded h264-file is completely wrong after the encoding-process.

This is how it is supposed to look like:
https://imgur.com/a/2GtKLVg

and this is how it actually looks like, after the encoding:
https://imgur.com/a/gmwfe3A

Any idea on how to fix the aspect ratio for this in general?
I tried -aspect 16:9 but that did not really help.

This effect appears whenever the last show before your recording is in a different aspect ratio than the recording itself. The aired stream may change AR midstream. The .ts format also supports this feature (but has other caveats, like not containing metadata). The .mkv format I recommended has many upsides, but unfortunately doesn’t support changing AR. So even without transcoding .mkv files may show a distorted picture.

The script currently does not have any option to force AR and doesn’t handle files without metadata very well. For this reason I’m currently working on an update that offers an option to also record in .ts . I’m not quite done yet. If you hang on a week or two I’ll post the update here.

Also forcing AR is an interesting idea. I may include a switch for that in a future update.

Well, the show I recorded DID NOT change the AR. It was all in 16:9. The show before was, too.

I have noticed the following: If I use the hardware based h264 encoder (h264_omx), the AR is wrong. No matter what. Even if I start the recording in the middle of the show.
But if I use the software base h264 encoder, the AR is set correctly. So I guess there is a bug in h264_omx.

The only thing that helps is setting a fixed size for h264_omx - from 720x576 to 720x405. But that would only work for 16:9 shows and is not a good solution.

Any news on this? Especially on the wrong AR when using h264_omx (see comment above)?

Dear pixelquadrat,

comming back to your issue, can you please tell if you are using my script or directly calling ffmpeg? You got me a little confused with calling “-aspect 16:9” and “fixed size”. Both of these functions aren’t included in autoedit. :-?

I replaced my RPi with a more powerfull device long ago, but from the time I used it for transcoding I can tell you there is no general bug in OMX that always corrupts the aspect ratio. It can encode all kinds of resolutions (16:9 SD and above), even though nowadays you can get encoding with better quality AND higher efficiency in software at sufficient speed with more modern systems. I’d rather guess it’s some oddity with ffmpeg, but we need to examine the video files closer to identify that.

If you’d like to try that, please record some sample and get the stream info by calling ffmpeg -i video_file for the original and the converted (with wrong AR) file. Paste both info here and we might see who is the culprit…

Dear Fretzke,

I have an similar Problem with the autoedit script. Everything is up and ready to run, but the script fails with:

„Cannot move video file to temporary place. Missing permission?“

I Have absolutely no clue where the script put temp data into $RawFolder. How can I fix the permission issue? Everything is located in /home/osmc.

Thanks a lot.

Admittedly the script grew over time and is now a little intricate. I tried to add explanatory comments to all relevant commands, but still… :joy:
At line #604 you can see that $RawFolder is merely the original file’s folder. So if the video is located in /home/osmc, that is the folder causing problems. The error message indicates the script cannot write here.
Do you run the script manually or through cron? If the cronjob is not configured correctly that might be the reason.

Thank you for your quick response.

I have reinstalled the script and try a new run. My OSMC Home Directory with using user osmc:

drwxr-xr-x 11 osmc osmc 4096 Nov 19 13:21 .
drwxr-xr-x 3 root root 4096 Nov 18 11:56 …
-rw------- 1 osmc osmc 23338 Nov 18 15:37 .bash_history
-rw-r–r-- 1 osmc osmc 220 May 15 2017 .bash_logout
-rw-r–r-- 1 osmc osmc 3574 Nov 13 19:25 .bashrc
drwxr-xr-x 3 osmc video 4096 Nov 14 16:25 .hts
drwxr-xr-x 8 osmc osmc 4096 Nov 15 07:34 .kodi
drwxr-xr-x 2 osmc osmc 4096 Nov 14 17:09 .nano
-rw-r–r-- 1 osmc osmc 675 May 15 2017 .profile
-rw------- 1 osmc osmc 0 Nov 13 18:49 .python_history
-rw-r–r-- 1 osmc osmc 0 Nov 14 17:09 .selected_editor
-rw-r–r-- 1 osmc osmc 254 Nov 14 20:54 .wget-hsts
-rwxrwxrwx 1 osmc video 2627801208 Nov 17 16:44 Film1.ts
-rw-rw-r-- 1 osmc video 3007170544 Nov 17 19:54 Film2.ts
drwxr-xr-x 2 osmc osmc 4096 Nov 4 16:08 Movies
drwxr-xr-x 2 osmc osmc 4096 Nov 4 16:08 Music
drwxr-xr-x 2 osmc osmc 4096 Nov 4 16:08 Pictures
drwxr-xr-x 2 osmc osmc 4096 Nov 18 13:14 TV
-rw-r–r-- 1 root root 6167 Nov 19 13:19 comskip.ini
-rw-r–r-- 1 osmc osmc 368401 Nov 14 20:54 repository.kodinerds.zip
drwxr-xr-x 8 root root 4096 Nov 15 11:22 rtl8812au
drwxr-xr-x 2 osmc osmc 4096 Nov 14 18:14 sources

Settings:

osmc@megatron:~$ cat /opt/autoedit/settings.txt
##Change personal settings here:
#Change the encoding table to be used
#Path where the processed files will be stored. Filebot will create corresponding subfolders here.
outfolder=“/home/osmc/”
#default ini file for comskip if parameter not givven
comskipINI=“/home/osmc/comskip.ini”
##Filebot settings:
#language
language=“de”
#Naming format for series
#Standard is “series_title (year)/season.episode_title.file_extension”
#e.g. “Breaking Bad (2008)/S04E01.Box Cutter.mkv”
#Including capital letters and spaces, file will be stored in the same directory
nameFormatSeries=‘/home/osmc/{n.ascii().replaceAll(/[?]+$/).tr(“–”,““).tr(”*“,””).tr(“|”,““)} ({y})/{s00e00}.{t.ascii().replaceAll(/[?]+$/).tr(”:“,””).tr(““,”“).tr(”|“,”“)}’
#Naming format for movies
#Standard is “title (year)/title.file_extension”
#e.g. “Rogue One A Star Wars Story (2016)/Rogue One A Star Wars Story.mkv”
#Including capital letters and spaces, file will be stored in a separate movie directory
nameFormatMovie='/home/osmc/{n.ascii().replaceAll(/[?]+$/).tr(”:“,”_“).tr(”
”,““).tr(”|“,””)} ({y})/{n.ascii().replaceAll(/[?]+$/).tr(“:”,““).tr(”*“,””).tr(“|”,“_”)}’
General settings
#–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
#Log file
autoeditlog="${outfolder}“autoedit.log
#Queue file
queuefile=”${outfolder}"autoeditQueue.txt
#–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

osmc@megatron:~$ sudo /usr/bin/autoedit --input “Film1.ts” --transcode mpeg2_mmal h264_omx 2000k
osmc@megatron:~$ cat autoedit.lo
autoedit.lockfile autoedit.log
osmc@megatron:~$ cat autoedit.log
19.11.2019 13:33:26 : New video recorded. The file name is Film1.ts .
19.11.2019 13:33:26 : Appending video file to queue.
19.11.2019 13:33:26 :
19.11.2019 13:33:26 : Start processing…
19.11.2019 13:33:26 : Fetched next file from queue. Processing Film1.ts
19.11.2019 13:33:26 : Cannot move video file to temporary place. Missing permission?
19.11.2019 13:33:26 : Aborting…
19.11.2019 13:33:26 :


osmc@megatron:~$ sudo /usr/bin/autoedit --input “Film1.ts” --transcode mpeg2_mmal h264_omx 2000k --wait
osmc@megatron:~$ cat autoedit.log
19.11.2019 13:33:26 : New video recorded. The file name is Film1.ts .
19.11.2019 13:33:26 : Appending video file to queue.
19.11.2019 13:33:26 :
19.11.2019 13:33:26 : Start processing…
19.11.2019 13:33:26 : Fetched next file from queue. Processing Film1.ts
19.11.2019 13:33:26 : Cannot move video file to temporary place. Missing permission?
19.11.2019 13:33:26 : Aborting…
19.11.2019 13:33:26 :
19.11.2019 13:34:43 : New video recorded. The file name is Film1.ts .
19.11.2019 13:34:43 : Video file already queued, not adding duplicates.

osmc@megatron:~$ bash -l -c autoedit
/usr/bin/autoedit: line 7: /home/osmc/autoedit.log: Permission denied
/usr/bin/autoedit: line 7: /home/osmc/autoedit.log: Permission denied
/usr/bin/autoedit: line 7: /home/osmc/autoedit.log: Permission denied
/usr/bin/autoedit: line 7: /home/osmc/autoedit.log: Permission denied
/usr/bin/autoedit: line 7: /home/osmc/autoedit.log: Permission denied
/usr/bin/autoedit: line 7: /home/osmc/autoedit.log: Permission denied

osmc@megatron:~$ sudo bash -l -c autoedit
osmc@megatron:~$ cat autoedit.log

19.11.2019 13:36:08 : Start processing…
19.11.2019 13:36:08 : Fetched next file from queue. Processing Film1.ts
19.11.2019 13:36:08 : Previous run did not complete.
19.11.2019 13:36:08 : Exiting!

Even “force” did not work. I have a working directory with osmc user in osmc home directory. Playing on last OSM release:

osmc@megatron:~$ uname -a
Linux megatron 4.19.55-6-osmc #1 SMP PREEMPT Sun Nov 3 22:15:28 UTC 2019 armv7l GNU/Linux

Any Ideas? :pray:

I tried also removing the lockfile and run again. Still the same error message.
I tried run as root even as cronjob too. Still the same error message.

Maybe as workaround how can i change the $RawFolder directory? Maybe /tmp make sense.

Not sure if this is only an oddity of your ssh session, but your screenshot indicates you only have user write permission to the chosen directory, not even user group:

d r w x r (no w!) x r (no w!) x:

Moving the file to some place with ‘777’ permission and pointing the outfolder there might help. You don’t manually overwrite $RawFolder, it is just a snippet of the source’s path. Just move the video file somewhere else.
/tmp usually is a ramdisk (not sure about OSMC), so a video might well be too large to fit in there. But /var might do.

Strange thing. I set permissions to 777 and have tryed via root:

root@megatron:/home/osmc# sudo /usr/bin/autoedit --input “Film1.ts” --transcode mpeg2_mmal h264_omx 2000k
root@megatron:/home/osmc# ls -la
total 5503376
drwxrwxrwx 11 osmc osmc 4096 Nov 19 16:40 .
drwxrwxrwx 3 osmc osmc 4096 Nov 18 11:56 …
-rw------- 1 osmc osmc 23338 Nov 18 15:37 .bash_history
-rw-r–r-- 1 osmc osmc 220 May 15 2017 .bash_logout
-rw-r–r-- 1 osmc osmc 3574 Nov 13 19:25 .bashrc
drwxr-xr-x 3 osmc video 4096 Nov 14 16:25 .hts
drwxr-xr-x 8 osmc osmc 4096 Nov 15 07:34 .kodi
drwxr-xr-x 2 osmc osmc 4096 Nov 14 17:09 .nano
-rw-r–r-- 1 osmc osmc 675 May 15 2017 .profile
-rw------- 1 osmc osmc 0 Nov 13 18:49 .python_history
-rw-r–r-- 1 osmc osmc 0 Nov 14 17:09 .selected_editor
-rw-r–r-- 1 osmc osmc 254 Nov 14 20:54 .wget-hsts
-rwxrwxrwx 1 osmc video 2627801208 Nov 17 16:44 Film1.ts
-rw-rw-r-- 1 osmc video 3007170544 Nov 17 19:54 Film2.ts
drwxr-xr-x 2 osmc osmc 4096 Nov 4 16:08 Movies
drwxr-xr-x 2 osmc osmc 4096 Nov 4 16:08 Music
drwxr-xr-x 2 osmc osmc 4096 Nov 4 16:08 Pictures
drwxr-xr-x 2 osmc osmc 4096 Nov 18 13:14 TV
-rw-r–r-- 1 root root 0 Nov 19 16:40 autoedit.lockfile
-rw-r–r-- 1 root root 400 Nov 19 16:40 autoedit.log
-rw-r–r-- 1 root root 48 Nov 19 16:40 autoeditQueue.txt
-rw-r–r-- 1 root root 6167 Nov 19 13:19 comskip.ini
-rw-r–r-- 1 osmc osmc 368401 Nov 14 20:54 repository.kodinerds.zip
drwxr-xr-x 8 root root 4096 Nov 15 11:22 rtl8812au
drwxr-xr-x 2 osmc osmc 4096 Nov 14 18:14 sources
root@megatron:/home/osmc# cat autoedit.log
19.11.2019 16:40:29 : New video recorded. The file name is Film1.ts .
19.11.2019 16:40:30 : Appending video file to queue.
19.11.2019 16:40:30 :
19.11.2019 16:40:30 : Start processing…
19.11.2019 16:40:30 : Fetched next file from queue. Processing Film1.ts
19.11.2019 16:40:30 : Cannot move video file to temporary place. Missing permission?
19.11.2019 16:40:30 : Aborting…
19.11.2019 16:40:30 :
root@megatron:/home/osmc#

So, even root is not working. Have you created a script with SuperMuh Power? :wink:

Absolutely! Only (Kodi) Krypton can stop it. :laughing:

Duh! :confounded:

Was about to write a lengthy answer when I noticed you call the script with just “Film1.ts” as parameter. Try again with full path, please.

Uhhhh… New Challenge :face_with_hand_over_mouth:

osmc@megatron:~$ sudo /usr/bin/autoedit --input “/home/osmc/Film1.ts” --transcode mpeg2_mmal h264_omx 2000k
ffmpeg version 3.2.14-1~deb9u1 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
configuration: --prefix=/usr --extra-version=‘1~deb9u1’ --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
libavutil 55. 34.101 / 55. 34.101
libavcodec 57. 64.101 / 57. 64.101
libavformat 57. 56.101 / 57. 56.101
libavdevice 57. 1.100 / 57. 1.100
libavfilter 6. 65.100 / 6. 65.100
libavresample 3. 1. 0 / 3. 1. 0
libswscale 4. 2.100 / 4. 2.100
libswresample 2. 3.100 / 2. 3.100
libpostproc 54. 1.100 / 54. 1.100
Unknown decoder ‘mpeg2_mmal’
osmc@megatron:~$

So independently from the decoder issue. Should i modify the script in some way? Because from your fantastic Tutorial (really) it doesn´t work out of the box.

That’s not necessary. Call the script with the right params and it should work.
’ --input with full path, --transcode only with existing de- and encoders.
Try ffmpeg -decoders for a list of the ones installed on your system. mpeg2_mmal only works on the RPi and only if ffmpeg was compiled with the corresponding libraries and flags. Same with h264_omx encoder. You can also stick to software h264, which runs a little slower but produces better quality.