Docker no longer works in OSMC 2018.10-1

[swagner74]

Hi,

I’ve been using docker just fine up until a few days ago when I updated to the latest docker-ce.

/etc/apt/sources.list.d/docker.list:

deb [arch=armhf] Index of linux/debian/ stretch stable

Latest docker-ce package:

5:18.09.0~3-0~debian-stretch

service docker start

A dependency job for docker.service failed. See ‘journalctl -xe’ for details.

journalctl -xe

…
Nov 17 10:18:07 osmc modprobe[17551]: modprobe: FATAL: Module overlay not found in directory /lib/modules/3.14.29-126-osmc
…

So kernel 3.14.29-126-osmc which I assume shipped with OSMC 2018.10-1 is really missing the overlay module? Anything I can do on my end to make docker work again or do I have to wait for osmc devs to provide the overlay module for the current kernel?

[Tom_Doyle]

Hi,

I may be wrong but I don’t think the overlayFS module has ever been enabled in the vero4k kernel, which this suggests also:

Docker support

I got Vero 4K for some time now and its great. Altough there are some things that Im missing from OSMC on RPI2. The bigest one is docker support. Docker is not officially supported yet for aarch64 but it can be compiled from source. Whats blocking me are some missing kernel modules/config. Bellow Im pasting output of dockers check-config script. Is it possible in foreseeable future to add those missing parts? osmc@osmc:~/code$ sudo ./check-config.sh warning: /proc/config.gz does not exi…

Also I’ve checked a few of the previous kernels and I can’t see it either.

I would check nothing has changed the docker configuration when it updated to:

@swagner74

5:18.09.0~3-0~debian-stretch

Thanks Tom.

[swagner74]

Thanks for your feedback Tom.
I’ll dig into the docker configuration as you suggest and obviously report my findings if any.
But it’s gonna take some time since I’m no expert and have to read / understand some doc first.

Thanks again.

[swagner74]

1/ My system doesn’t have any /etc/docker/daemon.json
Maybe it is normal, I don’t know. But that gives me not much to work with…

2/ Surprisingly, starting dockerd manually actually works (i.e dockerd --debug) and I can indeed run all my containers just fine.

So at least, that is a decent workaround for me now.

So the problem might not be docker-ce but maybe related to systemd somehow. I was hoping to find a /etc/systemd/system/docker.service.d directory but that’s not the case.

[Tom_Doyle]

Hi,

@swagner74

was hoping to find a /etc/systemd/system/docker.service.d

its /lib/systemd/systemd/docker.service

edit:

I think

@swagner74

dockerd --debug)

Whats the output of running this command?

[swagner74]

@Tom_Doyle

@swagner74

dockerd --debug)

Whats the output of running this command?

http://paste.osmc.io/ifonibuviv.vhdl

[Tom_Doyle]

Hi,

INFO[2018-11-17T14:31:10.855767765+01:00] [graphdriver] using prior storage driver: aufs

Looks like it started with aufs, also if I’m reading it correctly it was successfully using aufs with systemd before the update.

Please provide the out of

ls -lah /etc/docker

Thanks Tom.

[swagner74]

total 12K
drwx------ 2 root root 4.0K Nov 17 12:18 .
drwxr-xr-x 80 root root 4.0K Nov 17 14:30 …
-rw------- 1 root root 244 Jun 7 18:43 key.json

[Tom_Doyle]

Hi,

Just found this:

18.09: containerd depends on overlay

Seems to be and issue with the latest docker. I just installed docker and it wouldn’t start, same issue as reported by yourself.

I Removed ExecStartPre=/sbin/modprobe overlay from containerd.service and sudo systemctl daemon-reload. Then rebooted (just restarting the services didn’t help, I’m guessing there is an order.)

Thanks Tom.

[swagner74]

I figured out why docker is broken.

Here’s containerd (a dependency of the docker daemon) systemd service configuration:
cat /lib/systemd/system/containerd.service

[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target

[Service]
ExecStartPre=/sbin/modprobe overlay
ExecStart=/usr/bin/containerd
KillMode=process
Delegate=yes
LimitNOFILE=1048576
…

Since the overlay module is not needed (and even non existing) on our system, it actually hurts to require it before starting containterd.

Commenting out the ExecStartPre directive fixes containerd and dockerd systemd services.

[swagner74]

Ohh… You’re faster than me…

[dillthedog]

Just a FYI. overlayfs is available in the Pi build of OSMC, though not on the Vero4K/+

osmc@osmc:~$ modinfo overlay
filename:       /lib/modules/4.14.78-2-osmc/kernel/fs/overlayfs/overlay.ko
alias:          fs-overlay
license:        GPL
description:    Overlay filesystem
author:         Miklos Szeredi <miklos@szeredi.hu>
srcversion:     8BB6A167AD7727D1C2B1A3C
depends:        
intree:         Y
name:           overlay
vermagic:       4.14.78-2-osmc SMP preempt mod_unload modversions ARMv7 p2v8 
parm:           check_copy_up:bool
parm:           ovl_check_copy_up:Warn on copy-up when causing process also has a R/O fd open
parm:           redirect_max:ushort
parm:           ovl_redirect_max:Maximum length of absolute redirect xattr value
parm:           redirect_dir:bool
parm:           ovl_redirect_dir_def:Default to on or off for the redirect_dir feature
parm:           index:bool
parm:           ovl_index_def:Default to on or off for the inodes index feature