Networking slow/no NFS after a reboot

I have both a Vero 4K and a 4K plus. Today I rebooted the 4K, and after this it now no longer will mount my NFS drives. While trying to figure out why, I also noticed the ping is dramatically worse.

10 packets transmitted, 10 packets received, 0% packet loss
round-trip min/avg/max = 2.492/3.762/13.284 ms
osmc@osmc4plus

compared to the 4K

10 packets transmitted, 10 packets received, 0% packet loss
round-trip min/avg/max = 0.697/0.933/1.171 ms
osmc@osmc4

iperf results from the two machines are essentially the same.

Both are running the same kernel.

Would love some help figuring out what has happened. I’ve tried rebooting more times, unplugging and replugging into the switch, and that’s about it.

cheers,
Brett

Well I think many people would love to see logs to be able to help.

Ugh, sorry for not supplying those initially:

https://paste.osmc.tv/igazapedek

cheers

Thanks, maybe also for comparison provide the logs from the 4k

The 4K doesn’t automatically mount NFS like like the 4Kplus does, but I did that by hand after a reboot with debug mode to make sure it was still fine, and it was

https://paste.osmc.tv/odetapodoc

4Kplus fstab:

swamp:/tbla/video /tbla/video nfs defaults,noatime,timeo=14,auto,_netdev 0 0
swamp:/tbla/music /tbla/music nfs defaults,noatime,timeo=14,auto,_netdev 0 0
swamp:/tbla/backup /tbla/backup nfs defaults,noatime,timeo=14,auto,_netdev 0 0

4K fstab:

swamp:/tbla/video /tbla/video nfs defaults,noatime,timeo=14,noauto 0 0
swamp:/tbla/music /tbla/music nfs defaults,noatime,timeo=14,noauto 0 0
swamp:/tbla/backup /tbla/backup nfs defaults,noatime,timeo=14,noauto 0 0

:slight_smile:

So a bit confused here. How are you comparing the systems if they are configured differently?
Also for the Iperf3 test I assume you did it between each Vero and the NFS Server, or? Did you also tried the reverse direction with -R?

How are you comparing the systems if they are configured differently?

I’m happy to pretend the 4k doesn’t exist, if that helps.

Either way, at boot the 4Kplus now no longer mounts NFS (it times out).
It also has a lot of trouble talking to the mysql server (times out and hits connection limits)
It also has a very high ping.

I assume you did it between each Vero and the NFS Server

Correct.

Did you also tried the reverse direction

Here are the reverse results. Each from the vero unit to the NFS/Mysql server

[ 4] local 10.9.16.100 port 5001 connected with 10.9.16.111 port 33985
[ 4] 0.0-10.1 sec 113 MBytes 94.1 Mbits/sec
[ 5] local 10.9.16.100 port 5001 connected with 10.9.16.110 port 38139
[ 5] 0.0-10.1 sec 113 MBytes 94.1 Mbits/sec

That looks all fine, maybe install ethtool to see if you have a negotiation issue.

Also check with showmount -e swamp the exports.

One question (not directly related to your issue, any logic reason to use a 255.255.0.0 netmask?

Well …

osmc@osmc4plus:~$ sudo ethtool eth0
Settings for eth0:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Half 1000baseT/Full 
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Half 1000baseT/Full 
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Link partner advertised link modes:  10baseT/Half 10baseT/Full 
                                             100baseT/Half 100baseT/Full 
                                             1000baseT/Half 1000baseT/Full 
        Link partner advertised pause frame use: Symmetric Receive-only
        Link partner advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: on
        Supports Wake-on: ug
        Wake-on: d
        Current message level: 0x0000003d (61)
                               drv link timer ifdown ifup
        Link detected: yes

But what confuses me is:

osmc@osmc4:~$ sudo ethtool eth0
Settings for eth0:
        Supported ports: [ TP AUI BNC MII FIBRE ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Speed: 100Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 8
        Transceiver: external
        Auto-negotiation: on
        Supports Wake-on: ug
        Wake-on: d
        Current message level: 0x0000003d (61)
                               drv link timer ifdown ifup
        Link detected: yes

They’re plugged into different gigabit switches (happy to move them around to see if that helps diagnose anything)

Shrug.
Router at the time was configured to use a /16
Some overkill, but I never bothered adjusting

Oh, please ignore my confusion above. Of course the 4K is limited to 100M

Yes, thats correct.
But why does your iperf only shows 100Mbit? Is your NAS just support 100Mbit?
Are the NAS and Vero4k+ on same switch?

The NFS/MySQL server is a FreeBSD desktop, and has gigabit support.
Between it and the 4Kplus is Ethernet over Power. However, I’ve just unplugged the 4Kplus from there, taken it down and plugged in in place of the 4K, directly into the gigabit switch that the NFS server is plugged into. It’s still seeing the same symptoms.

I’m happy to upload logs again, now that they’re on the same switch, if you think that will help

edit: a slight correction. It’s still seeing the symptoms of not being able to mount NFS. Ping has now dropped to the same levels I saw for the 4K, and it may be that the EoP was always giving me horrendous ping, and that was a red herring. The NFS mounting issues are definitely new though

Well EoP is greatly unreliable and I would not chase down too much on that.
To be honest I am struggling to understand why you had iperf of 100 Mbit only if both devices are Gigabit.

Anyhow, either you can chase down your network issue, or alternatively try autofs instead of fstab mounts maybe that solves your issue.

I’m just struggling to see why this is not working. It has been operational like this, using NFS a year and a half now. First on the 4K and then on the 4Kplus. Then today I reboot, and it no longer works.

Your 4K has noauto in the fstab, so you have to manually mount those shaares as @dillthedog pointed out. You should probably leave the noauto, and change _netdev to x-systemd.automount

That’s by design. Different hosts for different purposes. The 4K is fine. It mounts NFS when I want to copy files to it, then it gets shut down and moved with us to watch episodes at a different house with no networking. If I had those automout, it’d try to do it at boot at the other house and take minutes failing.

You may consider using autofs in that use case.

Have you restarted you server recently?

This message might be relevant:

Jul 29 21:45:53 osmc4plus kernel: NFS: state manager: lease expired failed on NFSv4 server swamp with error 93

Error 93 means that the protocol isn’t supported – which I assume is referring to NFSv4.

Hmm, good point.

I just re-tested autofs once more (went down this path initially when I got the 4K).
It just doesn’t mount the damn filesystems for osmc. I follow the guide, and sure if I log in and ls it mounts them. But if I reboot and let kodi come up, nothing gets mounted and no files are available.

I’m just not an autofs fan