Trouble with autofs on Vero4k+, hangs after server reboot

Hi, can someone help me figure out what’s going wrong with autofs? I set it up following the wiki guide here, and while it connects successfully, if I come back to kodi later after the computer the SMB share is on has restarted, it hangs whenever I try to access it.

I’ve enabled debug logging of autofs, and what I get from journalctl -l -u autofs is: autofs log starting - Pastebin.com
Nothing suspicious in there I can see.

After the server providing the share reboots, kodi hangs on trying to play a file. I can still SSH in, though, but ls /mnt also hangs. sudo umount /mnt/tv also hangs. The only thing which works is sudo umount -l /mnt/tv, this unfreezes everything. If I do that, the log from autofs shows: autofs log after umount - Pastebin.com

Any ideas, anyone?

Please don’t provide just log snippets. At least provide the full system journal and better still, the full OSMC log set (grab-logs -A).

Autofs should automatically unmount a share after a configurable timeout (default 10 mins, I think). AFAIK if the autofs mount is still up when the server restarts, then it’s probably going to hang, whether you’re using fstab or autofs. You don’t mention what timeout you’re using, though the wiki uses 15 seconds in its example code. It’s possible that Kodi keeps accessing the share, keeping the timeout from kicking in.

Clearly, one approach is not to restart the server…

Edit: typo

I’m not sure how helpful endless repetitions of st_ready st_expire are, but OK, here’s the journalctl log from today up to the point where I turned autofs verbose logging off again: journalctl -l - Pastebin.com

I expect not restarting the server would help, sure, as would copying all the files onto a USB attached hard disk, but the situation I’ve got is a server that is turned off overnight and I want a reliable connection to it. I thought autofs would do that.

According to the mount.cifs man page, we can have a hard mount or a soft mount.

hard
The program accessing a file on the cifs mounted file system will hang when the server crashes.

soft
(default) The program accessing a file on the cifs mounted file system will not hang when the server crashes and will return errors to the user application.

The default is “soft” but the behaviour is as if it were “hard”. The log doesn’t show any related information that I could find. You could try adding “soft” to your mount options in /etc/auto.smb.shares and see if that makes a difference.

Yeah, I thought soft was the default too. ls /mnt/tv still hangs if the server has rebooted, but now after a long wait it returns ls: cannot access ‘/mnt/tv’: Resource temporarily unavailable

If I then repeat ls /mnt/tv then it works. So we’re on to a good start, if I can figure out which timeout is affecting this then I can reduce that to 3 seconds or so and it will be fine.

You appear to have the watchdog addon monitoring /mnt/tv. This may be keeping the mount active, not allowing autofs to unmount after the inactivity period. Try turning off watchdog and see if that helps.

That doesn’t seem to have had any effect, I still can’t see anything unmounting in the logs, and there is still a long freeze when trying to access the mount point after the server has restarted. But at least it does eventually time out and recover now.

If autofs is not unmounting after the timeout period, then something is still accessing /mnt/tv.

You could try using lsof to try and find what’s accessing it.

sudo apt install lsof

And then when you think that there should be nothing accessing /mnt/tv

sudo lsof +D /mnt/tv

that will show what processes are accessing /mnt/tv

Thanks for the suggestion. lsof +D /mnt/tv shows no output at all, which I assume means nothing is accessing it.

On a similar note, sudo fuser -mva /mnt/tv shows:

                     USER        PID ACCESS COMMAND
/mnt/tv:             root     kernel mount /mnt/tv

The last entry in systemctl status autofs remains

osmc automount[438]: mounted /mnt/tv/

from when it was mounted, about half an hour ago.

I’m not sure why autofs isn’t auto unmounting it, but it doesn’t appear to be. In auto.master I have:

/- /etc/auto.smb.shares --timeout=15 --browse

which I believe is supposed make it unmount in 15 seconds, right?

Yes, unmount after 15 seconds of inactivity.

Try this

sudo lsof -u osmc | grep /mnt

Ah ha, I think I see the problem! you have --browse in your auto.master. It should just be browse, no ‘–’

sudo lsof -u osmc | grep /mnt

also returns no output. Just to sanity check I ran it without the grep too, and it does come back with a long list of entries, so it’s not that lsof isn’t working at all.

Changing browse didn’t make any difference, I’m afraid.

Please share what is in your auto.master and auto.smb.shares.

Did you reboot after making the change to auto.master?

Not a lot to them, but here is auto.master: auto.master - Pastebin.com
And auto.smb.shares: auto.smb.shares - Pastebin.com

I tried both just systemctl restart autofs and rebooting, as far as I can tell neither make it behave differently, I still see no evidence of unmounting after 15 seconds.

Thanks for looking into this, by the way, I appreciate the help.

Run

sudo systemctl stop autofs
sudo automount -f -v -d

then you should see the activity of autofs and maybe the reason why id doesn’t unmount.

Nothing interesting happens after 15 seconds: automount -f -v -d - Pastebin.com

In that log you can see that when I do an ls /mnt/tv in another SSH session it correctly auto-mounts, but then never unmounts.

Eventually I do a Ctrl+C and then I get: After Ctrl+C - Pastebin.com

Which seems to suggest it can unmount, it just doesn’t want to.

If you stop Kodi

sudo systemctl stop mediacenter

does it unmount properly? Also, you have the soft option in auto.smb.shares. That should not be needed.

This might be a lead: auto.master(5) - Linux manual page

NOTE: autofs currently does not collapse multiple slashes in paths,
so it is important to ensure paths used in maps are correct.
If unnecessary multiple slashes are present in a path it can
lead to unexpected failures such as an inability to expire au‐
tomounts. An exception to this is a trailing slash at the end
of the automount point path in the master map which will be
removed if present.

Your auto.smb.shares file – which isn’t the master map – contains this line:

/mnt/tv/ -fstype=cifs,vers=3.0,rw,credentials=/home/osmc/.smbcredentials,iocharset=utf8,uid=osmc,gid=osmc,soft ://192.168.1.203/.tv/

Perhaps it doesn’t like the trailing slash in /mnt/tv/ and/or ://192.168.1.203/.tv/ :slight_smile:

2 Likes

I put soft in auto.smb.share on dillthedog’s recommendation, but I can take it out again.

Stopping mediacenter doesn’t make any difference.

@dillthedog: Removing the slashes has done the job, you genius!

Sep 23 20:13:54 osmc systemd[1]: Started Automounts filesystems on demand.
Sep 23 20:14:08 osmc automount[19634]: attempting to mount entry /mnt/tv
Sep 23 20:14:09 osmc automount[19634]: mounted /mnt/tv
Sep 23 20:14:11 osmc automount[19634]: 1 remaining in /-
Sep 23 20:14:15 osmc automount[19634]: 1 remaining in /-
Sep 23 20:14:19 osmc automount[19634]: 1 remaining in /-
Sep 23 20:14:23 osmc automount[19634]: 1 remaining in /-
Sep 23 20:14:27 osmc automount[19634]: expiring path /mnt/tv
Sep 23 20:14:27 osmc automount[19634]: expired /mnt/tv

I’m assuming that this will now not hang on server reboot, going to test that tonight.

Whoever coded autofs to think that “unexpected failures” with no error message is an acceptable way to handle trailing slashes needs a swift kick up the rear, but nevermind.

Thanks again.

1 Like

I didn’t think it would make a difference, but this was getting strange so one of those ‘Try anything’ problems…

I will update the HowTo and add a warning about trailing slashes.