Updates not installing when using crontab

geedee · 9 December 2017 04:34

I run the apt-get script everyday to check for, download and install any new updates on a daily basis. The commands i use for this are:

osmc@lab1:~$ sudo apt-get update && sudo apt-get dist-upgrade -y && sudo apt-get autoremove && sudo apt-get autoclean

I want to use crontab to do this on a daily basis so I don’t have to manually do this. My config for crontab is:
osmc@lab1:~$ sudo apt-get install cron
osmc@lab1:~$ sudo crontab -e
_osmc@lab1:~$ _
15 03 * * * echo “Starting nightly upgrade check process $(date)” 2>&1 >> /home/osmc/update.txt
_15 03 * * * apt-get update 2>&1 >> /home/osmc/update.txt _
_17 03 * * * apt-get dist-upgrade -y 2>&1 >> /home/osmc/update.txt _
_28 03 * * * apt-get autoremove 2>&1 >> /home/osmc/update.txt _
_30 03 * * * apt-get autoclean 2>&1 >> /home/osmc/update.txt _
35 03 * * * echo “Ending nightly upgrade check process $(date)” 2>&1 >> /home/osmc/update.txt
35 03 * * * echo “Initiating system nightly reboot” 2>&1 >> /home/osmc/update.txt
35 03 * * * /sbin/reboot
osmc@lab1:~$ Ctrl-O & Ctrl-X to save the file and exit the file

When I manually run the command, if there are any updates the same get downloaded, installed and unneeded updates get removed.

When the crontab job runs, it downloads the updates but fails to install any. This is the error I see in the update.txt file

Starting nightly upgrade check process Tue Dec 5 03:15:01 EST 2017
Get:1 http://security.debian.org jessie/updates InRelease [63.1 kB]
Hit http://apt.osmc.tv jessie InRelease
Ign http://ftp.debian.org jessie InRelease
Get:2 http://ftp.debian.org jessie-updates InRelease [145 kB]
Get:3 http://apt.osmc.tv jessie/main armhf Packages/DiffIndex [2023 B]
Hit http://ftp.debian.org jessie Release.gpg
Hit http://security.debian.org jessie/updates/main armhf Packages
Hit http://ftp.debian.org jessie Release
Hit http://security.debian.org jessie/updates/contrib armhf Packages
Hit http://security.debian.org jessie/updates/non-free armhf Packages
Hit http://security.debian.org jessie/updates/contrib Translation-en
Hit http://security.debian.org jessie/updates/main Translation-en
Get:4 http://ftp.debian.org jessie-updates/main armhf Packages/DiffIndex [9376 B]
Hit http://security.debian.org jessie/updates/non-free Translation-en
Hit http://ftp.debian.org jessie-updates/contrib armhf Packages
Get:5 http://ftp.debian.org jessie-updates/non-free armhf Packages/DiffIndex [736 B]
Hit http://ftp.debian.org jessie-updates/contrib Translation-en
Get:6 http://ftp.debian.org jessie-updates/main Translation-en/DiffIndex [3688 B]
Get:7 http://ftp.debian.org jessie-updates/non-free Translation-en/DiffIndex [736 B]
Hit http://ftp.debian.org jessie/main armhf Packages
Ign http://apt.osmc.tv jessie/main Translation-en
Hit http://ftp.debian.org jessie/contrib armhf Packages
Hit http://ftp.debian.org jessie/non-free armhf Packages
Hit http://ftp.debian.org jessie/contrib Translation-en
Hit http://ftp.debian.org jessie/main Translation-en
Hit http://ftp.debian.org jessie/non-free Translation-en
Fetched 225 kB in 18s (12.0 kB/s)
Reading package lists…
Reading package lists…
Building dependency tree…
Reading state information…
The following packages will be upgraded:
_ curl libcurl3_
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/432 kB of archives.
After this operation, 0 B of additional disk space will be used.
**dpkg: warning: ‘ldconfig’ not found in PATH or not executable**
dpkg: warning: ‘start-stop-daemon’ not found in PATH or not executable
dpkg: error: 2 expected programs not found in PATH or not executable
_Note: root’s PATH should usually contain /usr/local/sbin, /usr/sbin and /sbin_
Reading package lists…
Building dependency tree…
Reading state information…
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
Reading package lists…
Building dependency tree…
Reading state information…
Initiating system nightly reboot
Ending nightly upgrade check process Tue Dec 5 03:35:01 EST 2017

I have verified that root’s path has /usr/local/sbin, /usr/sbin and /sbin

#
# See the man page for details on how to write a sudoers file.
#
Defaults env_reset
Defaults mail_badpass
Defaults secure_path=“/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin”

# Host alias specification

# User alias specification

# Cmnd alias specification

# User privilege specification
root ALL=(ALL:ALL) ALL

# Allow members of group sudo to execute any command
%sudo ALL=(ALL:ALL) ALL

I am kinda stuck and don’t know what path to go down to troubleshoot the error and fix it. Anyone have an idea or two as to where i should start looking?

Thanks,
G

dillthedog · 9 December 2017 08:23

It wasn’t difficult to find:

I’m also a little surprised that this is working correctly:

apt-get update 2>&1 >> /home/osmc/update.txt

since the 2>&1 bit is normally position-dependent. Eg:

osmc@osmc:~$ cat nofile.txt 2>&1 >>zz.out
cat: nofile.txt: No such file or directory
osmc@osmc:~$ cat zz.out
osmc@osmc:~$

So just to be on the safe side, I’d recommend you place 2>&1 after >> /home/osmc/update.txt.

Edit: In the future you should consider placing everything into a single shell script. For example, you’ve allowed 2 minutes between the apt-get update and apt-get dist-upgrade but that 2-minute delay is an arbitrary figure and might be insufficient in some circumstances. And this is Linux so reboots are rarely required after an update.

You do know that My OSMC has a scheduled update mechanism?

geedee · 9 December 2017 15:25

I found a few similar articles before and I created a script of sorts but I am not sure if my script is missing something or my crontab job is missing something … On my best day I am a noob at all things linux …

This is what I have in my script, which is located in /home/osmc/update.sh:

#!/bin/bash

PATH=’/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin’
sudo apt-get update && sudo apt-get dist-upgrade -y && sudo apt-get autoremove && sudo apt-get autoclean && sudo reboot
exit

My cron job for this was:

osmc@lab1:~$ sudo crontab -e
osmc@lab1:~$
15 03 * * * echo “Starting nightly upgrade check process $(date)” 2>&1 >> /home/osmc/update.txt
_17 03 * * * update.sh 2>&1 >> /home/osmc/update.txt _
35 03 * * * echo “Ending nightly upgrade check process $(date)” 2>&1 >> /home/osmc/update.txt
osmc@lab1:~$ Ctrl-O & Ctrl-X to save the file and exit the file

It never produced an output except for the info from the echo statements and since the system wasn’t rebooting i know the script wasn’t being run.

I think I am overlooking something basic in my script above which is why it wasn’t running but i can’t seem to find the obvious …

When you say place 2>&1 after >> /home/osmc/update.txt. that means my cron lines above should read something like this:

35 03 * * * echo “Ending nightly upgrade check process $(date)” >> /home/osmc/update.txt 2>&1

Sorry for all the dumb questions …

Thanks.
G

dillthedog · 9 December 2017 16:14

I’ll start with the last bit first:

Correct.

Returning to your cron job:

Include the start and end echo statements in the shell script, not as separate cron jobs.
If you create the cron table using sudo crontab -e (which is fine), it runs all its jobs as root. So, the shell script doesn’t need to contain sudo before its commands.
The update.sh script failed to run because (a) it was running as root, and (b) root doesn’t know that the script is in /home/osmc. So it needs to be a fully qualified name. Better still, you should place it in /usr/local/bin and have it owned by root, since it’s a job that runs under root.
I also suggest you (a) send your log output to /var/log/daily_update/some_name and (b) redirect your output on the crontab line with a single chevron (>) and use $(date +%Y%m%d) in the log file name or you’ll get one massive (and unmanageabl) log file. So that would be > /var/log/daily_update/log_$(date +%Y%m%d). You’ll need to create the daily_update directory just once using sudo.
My final suggestion is to place each command on a separate line for readability and remove && sudo apt-get autoremove && sudo apt-get autoclean && sudo reboot. Try to keep things simple; you can do those extra steps manually from time to time.

geedee · 9 December 2017 22:26

ughhh … path to script is osmc home and I didn’t indicate that in the script … simple things that bite us in the ass …

thanks a ton for helping me with this @dillthedog
I’ll work on redoing the script and adding in the various things you pointed out … I am sure it will now work as expected … unless I decide to miss something obvious once again …

geedee · 9 December 2017 22:52

I added the section to check for internet connectivity based on the linked article you posted earlier and removed the auto remove/clean sections. I think this is going to work now… hopefully

osmc@lab1:~$ sudo nano /usr/local/bin/update.sh
#!/bin/bash
PATH=’/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin’
if ping -q -c 1 -W 1 8.8.4.4 >/dev/null; then
echo “Starting nightly upgrade check process $(date)”
sudo apt-get update
sudo apt-get dist-upgrade -y
echo “Ending nightly upgrade check process $(date)”
exit
fi
osmc@lab1:~$ Ctrl-O & Ctrl-X to save the file and exit the file

Change owner and grp on script file to root
osmc@lab1:~$ sudo chown root /usr/local/bin/update.sh
osmc@lab1:~$ sudo chgrp root /usr/local/bin/update.sh

Create directory
osmc@lab1:~$ sudo mkdir /var/log/daily_update

Create new cronjob for root
osmc@lab1:~$ sudo crontab -e
osmc@lab1:~$
15 03 * * * update.sh > /var/log/daily_update/log$(date +%Y%m%d)_
osmc@lab1:~$ Ctrl-O & Ctrl-X to save the file and exit the file

dillthedog · 9 December 2017 23:13

PATH=’/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin’

Remove the single quotes.

if ping -q -c 1 -W 1 8.8.4.4 >/dev/null; then

Best to add 2>&1 just as good practice.

echo "Starting nightly upgrade check process $(date)"
sudo apt-get update
sudo apt-get dist-upgrade -y

Sudo! It’s running as root.

Being a bit pedantic but I’d move the exit to be outside of the if… fi block.

15 03 * * * update.sh > /var/log/daily_update/log$(date +%Y%m%d)_

Fully qualified name, please. You only add /usr/local/bin to the path inside the script. It might work but it’s always good discipline to use a fully qualified name in cron. And do you want the underscore at the end of the log file name? You should also add 2>&1 to the end of the line.

Finally, you’ve not shown me a chmod u+x on /usr/local/bin/update.sh.

geedee · 10 December 2017 19:21

fixed the single quotes,
added 2>&1
Forgot to add the line with chmod u+x in the previous post …
moved exit to outside of the block
updated cron to indicate the path.
not sure why i have the _ at the end in the previous post, it isn’t there in the cron job.

Need clarification on placement of 2>&1 in two spots:

//Create script for auto updates
osmc@lab1:~$ sudo nano /usr/local/bin/update.sh
#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
if ping -q -c 1 -W 1 8.8.4.4 2>&1/dev/null; then
echo “Starting nightly upgrade check process $(date)”
apt-get update
apt-get dist-upgrade -y
echo “Ending nightly upgrade check process $(date)” 2>&1
fi
exit
osmc@lab1:~$ Ctrl-O & Ctrl-X to save the file and exit the file

//Give root control on script file
osmc@lab1:~$ sudo chown root /usr/local/bin/update.sh
osmc@lab1:~$ sudo chgrp root /usr/local/bin/update.sh
osmc@lab1:~$ sudo chmod u+x /usr/local/bin/update.sh

//Create directory
osmc@lab1:~$ osmc@lab1:~$ sudo mkdir /var/log/daily_update
osmc@lab1:~$ osmc@lab1:~$ sudo chmod 777 /var/log/daily_update

//Create new cronjob for root
osmc@lab1:~$ sudo crontab -e
osmc@lab1:~$
15 03 * * * /usr/local/bin/update.sh > /var/log/daily_update/log$(date +%Y%m%d)
osmc@lab1:~$ Ctrl-O & Ctrl-X to save the file and exit the file

In my script is this the correct placement? if ping -q -c 1 -W 1 8.8.4.4 2>&1/dev/null; then
In my cron job should I replace the > with 2>&1 ?

dillthedog · 10 December 2017 20:09

1 Should be:

if ping -q -c 1 -W 1 8.8.4.4 >/dev/null 2>&1; then

2 No. cron line should be:

15 03 * * * /usr/local/bin/update.sh > /var/log/daily_update/log_$(date +%Y%m%d) 2>&1

(I’ve added an underscore to the log file name, simply to make it a bit more readable.)

geedee · 11 December 2017 13:29

Final final version! If this works for you, take a moment to say thanks to @dillthedog

//Create script for auto updates
osmc@lab1:~$ sudo nano /usr/local/bin/update.sh
#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
if ping -q -c 1 -W 1 8.8.4.4 >/dev/null 2>&1; then
echo “Starting nightly upgrade check process $(date)”
apt-get update
apt-get dist-upgrade -y
echo “Ending nightly upgrade check process $(date)” 2>&1
fi
exit
osmc@lab1:~$ Ctrl-O & Ctrl-X to save the file and exit the file

//Change owner and grp on script file to root
osmc@lab1:~$ sudo chown root /usr/local/bin/update.sh
osmc@lab1:~$ sudo chgrp root /usr/local/bin/update.sh
osmc@lab1:~$ sudo chmod u+x /usr/local/bin/update.sh

//Create directory
osmc@lab1:~$ osmc@lab1:~$ sudo mkdir /var/log/daily_update
osmc@lab1:~$ osmc@lab1:~$ sudo chmod 777 /var/log/daily_update

//Create new cronjob for root
osmc@lab1:~$ sudo crontab -e
osmc@lab1:~$
15 03 * * * /usr/local/bin/update.sh > /var/log/daily_update/log_$(date +%Y%m%d) 2>&1
osmc@lab1:~$ Ctrl-O & Ctrl-X to save the file and exit the file

Outside of the script, every now and then, run the following commands to remove older/unused/deprecated/outdated components

osmc@lab1:~$ sudo apt-get autoremove
osmc@lab1:~$ sudo apt-get autoclean