NFS stopped working after update

Hey,

I just did the major update of KODI and OSMC and my NFS shares immediately stopped working. In the sense some data is read (2-3 sec from a video), then the whole file system locks up. KODI interface completely freezes and even via console commands like ls hang until I cancel them

Uploaded logs: https://paste.osmc.tv/ufocuvezot

This is my mount file (/etc/fstab)

192.168.7.46:/quantumsafe /home/osmc/QuantumSafe nfs vers=3,proto=tcp,noauto,x-systemd.automount 0 0
192.168.7.46:/criticalmusic /home/osmc/CriticalMusic nfs vers=3,proto=tcp,noatime,noauto,x-systemd.automount 0 0
192.168.7.46:/criticalvideo /home/osmc/CriticalVideo nfs vers=3,proto=tcp,noatime,noauto,x-systemd.automount 0 0

This looks relevant

Aug 24 17:14:10 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.automount: Got automount request for /home/osmc/QuantumSafe, triggered by 2708 (transmission-da)
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Mounting /home/osmc/QuantumSafe…
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: /lib/systemd/system/rpc-statd.service:13: PIDFile= references path below legacy directory /var/run/, updating /var/run/rpc.statd.pid → /run/rpc.statd.pid; please update the unit file accordingly.
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Reached target Host and Network Name Lookups.
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Starting Preprocess NFS configuration…
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: nfs-config.service: Succeeded.
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Started Preprocess NFS configuration.
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Starting Notify NFS peers of a restart…
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Starting NFS status monitor for NFSv2/3 locking…
Aug 24 17:14:10 osmc-QuantumRay sm-notify[2721]: Version 1.3.3 starting
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: rpc-statd-notify.service: Succeeded.
Aug 24 17:14:10 osmc-QuantumRay rpc.statd[2723]: Version 1.3.3 starting
Aug 24 17:14:10 osmc-QuantumRay rpc.statd[2723]: Flags: TI-RPC
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Started Notify NFS peers of a restart.
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Started NFS status monitor for NFSv2/3 locking…
Aug 24 17:14:10 osmc-QuantumRay mount[2714]: mount.nfs: Network is unreachable
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.mount: Mount process exited, code=exited, status=32/n/a
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.mount: Failed with result ‘exit-code’.
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Failed to mount /home/osmc/QuantumSafe.
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.automount: Got automount request for /home/osmc/QuantumSafe, triggered by 2708 (transmission-da)
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Mounting /home/osmc/QuantumSafe…
Aug 24 17:14:10 osmc-QuantumRay mount[2725]: mount.nfs: Network is unreachable
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.mount: Mount process exited, code=exited, status=32/n/a
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.mount: Failed with result ‘exit-code’.
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Failed to mount /home/osmc/QuantumSafe.
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.automount: Got automount request for /home/osmc/QuantumSafe, triggered by 2708 (transmission-da)
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Mounting /home/osmc/QuantumSafe…
Aug 24 17:14:10 osmc-QuantumRay mount[2727]: mount.nfs: Network is unreachable
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.mount: Mount process exited, code=exited, status=32/n/a
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.mount: Failed with result ‘exit-code’.
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Failed to mount /home/osmc/QuantumSafe.
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.automount: Got automount request for /home/osmc/QuantumSafe, triggered by 2708 (transmission-da)
Aug 24 17:14:10 osmc-QuantumRay systemd[1]: Mounting /home/osmc/QuantumSafe…
Aug 24 17:14:11 osmc-QuantumRay mount[2729]: mount.nfs: Network is unreachable
Aug 24 17:14:11 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.mount: Mount process exited, code=exited, status=32/n/a
Aug 24 17:14:11 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.mount: Failed with result ‘exit-code’.
Aug 24 17:14:11 osmc-QuantumRay systemd[1]: Failed to mount /home/osmc/QuantumSafe.
Aug 24 17:14:11 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.automount: Got automount request for /home/osmc/QuantumSafe, triggered by 2708 (transmission-da)
Aug 24 17:14:11 osmc-QuantumRay systemd[1]: Mounting /home/osmc/QuantumSafe…
Aug 24 17:14:11 osmc-QuantumRay mount[2731]: mount.nfs: Network is unreachable
Aug 24 17:14:11 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.mount: Mount process exited, code=exited, status=32/n/a
Aug 24 17:14:11 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.mount: Failed with result ‘exit-code’.
Aug 24 17:14:11 osmc-QuantumRay systemd[1]: Failed to mount /home/osmc/QuantumSafe.
Aug 24 17:14:11 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.automount: Got automount request for /home/osmc/QuantumSafe, triggered by 2708 (transmission-da)
Aug 24 17:14:11 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.mount: Start request repeated too quickly.
Aug 24 17:14:11 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.mount: Failed with result ‘exit-code’.
Aug 24 17:14:11 osmc-QuantumRay systemd[1]: Failed to mount /home/osmc/QuantumSafe.
Aug 24 17:14:11 osmc-QuantumRay systemd[1]: home-osmc-QuantumSafe.automount: Failed with result ‘mount-start-limit-hit’.

But it could explain only why the shared are not initially mounted. This network unreachable did not happen before the update.

Now after startup I remount them manually and then they work for the aforementioned 2-3 sec of video and then file system death.

Just to be sure: You activated the flag GUI → My OSMC → Network → Wired → Wait for Network ?

1 Like

Here’s the important line in the log:

Aug 24 17:14:13 osmc-QuantumRay avahi-daemon[2329]: Registering new address record for 192.168.7.28 on eth0.IPv4.

Note the time when you first got an IP address: 17:14:13.

Thank you guys, but I don’t really care about the mount on startup issues that much. I already have a cron script that checks the connection and remounts every few hours. What troubles me is the issue with the file system locking up that makes my library and the whole device unusable now. Please, help!

You’ve provided a log that shows Kodi starting too early in the startup sequence. If you apply the splendid solution from @JimKnopf, that issue should be resolved.

As for the lockups, it’s difficult to say without some hard data to work with. I can tell you, for example, that if you switch off an NFS server when a client is connected to it or the client loses contact with the server because of network problems, a kernel-based mount, which is what you are using, can cause a client system to freeze.

Thanks dillthedog!

I’d say you are right, there seems to be some kind of malfunction in the NFS client and that freezes the whole system when if fails, probably because of the mount. The question is why it fails after the update? And how to give you more logs? I tried to mount the NFS shares via the Files Add function where it has NFS, but it doesn’t do anything when I click. Is there any way I can use the shell to connect and debug NFS without mounting?

The last update to the nfs-common package on your system was on 5 January 2021, so any problem is likely to lie elsewhere.

IMO, the only useful information is going to come from the command line. You could try enabling permant journaling and see if it picks anything up.

(Don’t include -k in the journalctl command. That way we’ll see the whole log.)

Happy to report that the problem went away by itself, could have been due to a problem with my NFS server or the network, it resolved when I restarted the server host machine. I will keep an eye if it happens in the future

1 Like