Video buffer not loading fast enough for playback

I’ve been experiencing an issue for the last month or so and am so far unable to track down my issue. I’m running OSMC on a Vero 4K+. I’ll start playing a file, and the playback buffer will begin to fill…only to slow and eventually stop, halting playback. If I try to resume the same file, I get either “can’t play back file, check log” or “file no longer exists, delete?” There doesn’t seem to be a discernable pattern in all this. The files are stored on a new 3.5" HDD that’s connected via USB2.0 to my RPi 3b, which in turn is connected to my LAN via a USB3–>Ethernet adapter. I know all the network ports on the 3B share the same bus, but I’ve never had this problem up until recently, so it’s either the new drive (which seems to test out OK via iperf, etc.), or there’s some other issue. Here’s my most recent log; sorry it’s so long, the relevant bits are from today.

Thanks for any tips you can share!

Had a quick look into your logs:

You’re using a Vero which opens the media files via NFS (mediacenter stack) located on a NAS.
You say this NAS is a RPi 3B which got a new HDD.

The stalled IO stream could be any issue in the network infrastructure or is even a problem within in the NAS. I suggest you first have a look at [How To] Check Network Performance with iperf3 and verify the possible bandwidth between the Vero and the NAS.
If the bandwidth is sufficient (you can share the outputs here for help) the issue can reside on the Vero side otherwise you know for sure it is something in your network infrastructure or the NAS device itself.

Hope this helps for the first step of troubleshooting.

Addition: How is the new hard drive supplied with power? Own power supply or by the USB port of the rbp? What brand and model?

1 Like

Thanks for the reply. Here’s the drive I got. It’s in a powered USB enclosure.

But now that you mention the power thing…I’m starting to wonder. About the same time I got this new drive, I also started using an SSD for my download folder for SABnzbd. It’s also, of course, connected to one of the Pi’s USB2 ports. I’d initially done this to improve my download speeds, which worked, but if that’s the issue, I’m not going to sacrifice playback speed for download speed.

I did do some testing with iperf3, but only between the Pi and the Vero. I did not use a file stored on the USB drive attached to the Pi as a test file, which probably would’ve given a more accurate reading. I should probably do that.

1 Like

FWIW, here’s the iperf3 output just between OSMC and the RPi:

osmc@osmc:~$ iperf3 -c 192.168.50.103
Connecting to host 192.168.50.103, port 5201
[  5] local 192.168.50.119 port 55822 connected to 192.168.50.103 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  23.3 MBytes   196 Mbits/sec  121   58.0 KBytes       
[  5]   1.00-2.00   sec  23.4 MBytes   196 Mbits/sec  107   50.9 KBytes       
[  5]   2.00-3.00   sec  22.5 MBytes   188 Mbits/sec  100   72.1 KBytes       
[  5]   3.00-4.00   sec  24.2 MBytes   203 Mbits/sec  111   39.6 KBytes       
[  5]   4.00-5.00   sec  27.8 MBytes   233 Mbits/sec  542   83.4 KBytes       
[  5]   5.00-6.00   sec  30.0 MBytes   252 Mbits/sec  661   70.7 KBytes       
[  5]   6.00-7.00   sec  29.5 MBytes   248 Mbits/sec  547   77.8 KBytes       
[  5]   7.00-8.00   sec  22.3 MBytes   187 Mbits/sec  332   56.6 KBytes       
[  5]   8.00-9.00   sec  21.4 MBytes   180 Mbits/sec  223   80.6 KBytes       
[  5]   9.00-10.00  sec  23.2 MBytes   194 Mbits/sec  218   86.3 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   248 MBytes   208 Mbits/sec  2962             sender
[  5]   0.00-10.04  sec   247 MBytes   206 Mbits/sec                  receiver

Using iperf3 -c ... just tests the direction osmc -> RPi 3B+… and it looks to be very bad since a lot of retries, small cwnd (congestion window) and low bandwidth. Here is an example in my environment where the iperf3 server runs on a RPi 4:

root@osmc-vero4k:~# iperf3 -c osmc-pi4
Connecting to host osmc-pi4, port 5201
[  5] local 10.10.10.57 port 39598 connected to 10.10.10.41 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.01   sec   101 MBytes   845 Mbits/sec    0    288 KBytes
[  5]   1.01-2.00   sec   111 MBytes   939 Mbits/sec    0    530 KBytes
[  5]   2.00-3.00   sec   112 MBytes   939 Mbits/sec    0    530 KBytes
[  5]   3.00-4.00   sec   111 MBytes   931 Mbits/sec    0    530 KBytes
[  5]   4.00-5.00   sec   112 MBytes   939 Mbits/sec    0    554 KBytes
[  5]   5.00-6.00   sec   110 MBytes   922 Mbits/sec    0    554 KBytes
[  5]   6.00-7.00   sec   112 MBytes   940 Mbits/sec    0    554 KBytes
[  5]   7.00-8.00   sec   111 MBytes   934 Mbits/sec    0    554 KBytes
[  5]   8.00-9.00   sec   111 MBytes   932 Mbits/sec    0    554 KBytes
[  5]   9.00-10.00  sec   111 MBytes   929 Mbits/sec    0    554 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.08 GBytes   925 Mbits/sec    0             sender
[  5]   0.00-10.03  sec  1.07 GBytes   921 Mbits/sec                  receiver

iperf Done.

So, your output first indicates to see some serious network issues. But testing the other direction RPi 3B+ -> osmc using iperf -R -c ... would better fit the scenario where the RPi sends video data to the mediaplayer while playback. Look at the retry count and cwnd value on the server side, then.

1 Like

ok, verified that my pi-holes here are RPi 3B+ with Raspbian (bullseye). As an example the output from the iperf3 server running on such RPi 3B+ when using iperf -R -c ... on the osmc vero4k+:

root@pi-hole-dns:~# iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.10.10.57, port 48690
[  5] local 10.10.10.201 port 5201 connected to 10.10.10.57 port 48692
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  33.8 MBytes   283 Mbits/sec    0    495 KBytes
[  5]   1.00-2.00   sec  33.1 MBytes   277 Mbits/sec    0    495 KBytes
[  5]   2.00-3.00   sec  33.0 MBytes   277 Mbits/sec    0    495 KBytes
[  5]   3.00-4.00   sec  32.6 MBytes   274 Mbits/sec    0    495 KBytes
[  5]   4.00-5.00   sec  33.2 MBytes   279 Mbits/sec    0    495 KBytes
[  5]   5.00-6.00   sec  32.6 MBytes   274 Mbits/sec    0    495 KBytes
[  5]   6.00-7.00   sec  33.0 MBytes   277 Mbits/sec    0    495 KBytes
[  5]   7.00-8.00   sec  32.7 MBytes   275 Mbits/sec    0    495 KBytes
[  5]   8.00-9.00   sec  33.0 MBytes   277 Mbits/sec    0    495 KBytes
[  5]   9.00-10.00  sec  33.1 MBytes   277 Mbits/sec    0    495 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec   330 MBytes   277 Mbits/sec    0             sender
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

So, seems you cannot get higher bandwidth than 277 Mbits/sec ≈ 33 MiB/sec with the SOC in the RPi 3B+ but this is fast enough to satisfy the need for all video streaming to the mediaplayer.

1 Like

I really appreciate you testing this out on your end. So my Vero should be getting the data fast enough between the Vero and the RPi, and indeed it appears to when a video starts playing (I’m judging this solely by the progress bar at the bottom of the screen when a video is playing). Since my media drive is connected to the RPi, too, that’s eating a portion of that bus as well, but even it were 50%, I should still be getting enough throughput to play these files without choking. Looking more and more like the (new) media drive may be the culprit. I do have another RPi 3 lying around…maybe I could install Openmediavault on that, connect my Media drive to it (and maybe that SSD drive, too) and get it off network bus, at least. Guess I should do some more testing between the RPi and the Media drive, first, to try to confirm whether the issue resides there. :cry:

The first thing you need it to get rid of those transfer errors. I think the most likely is a bad cable. The second is that your PSU isn’t adequate for how your using it. If memory serves that model would start getting network errors before the voltage dropped low enough to cause other issues. You could try running the speed test without anything else running or plugged in. Also you didn’t state if this SSD was being powered by the RPi. Additionally you stated that your torrent performance increased when you added the SSD. Note that when that increased that is at the expense of other devices sharing the same USB bus. Going from spinning rust that can do 200 io/s to tens of thousands is going to have an impact. Your probably in need of doing some testing after getting your network sorted in if this hardware is really enough to act as both a seedbox and NAS.

2 Likes

Thank you for the reply.

Sorry, yes, the SSD was drawing power from the RPi, and I know power is an issue there; the SSD has since been removed from the mix (no change to my issue). I was under the impression that my video playback over the LAN (between Vero and RPi) would only be affected by the SSD if SAB were writing a file there (thereby consuming bus) at the same time I was playing a file. I’ll see what I can find on the cables…

Now I realize that you’re also using a usb3-ethernet adapter, probably to achieve bandwidths beyond 100Mbit/sec. So you will probably have a PRi 3B and not a PRi 3B+ as I assumed. This adapter is also a possible cause of the error. Can you test the PRi without this thing? I think these adapters also draw significant power. If your PSU for the PRi starts to wear out such problems could be result.

Can you test the PRi without this thing?

That’s exactly what I plan to do for my next step.

Removed the USB3–>Ethernet dongle from the mix and am now here:

osmc@osmc:~$ **iperf3 -R -c 192.168.50.243**
Connecting to host 192.168.50.243, port 5201
Reverse mode, remote host 192.168.50.243 is sending
[  5] local 192.168.50.119 port 39890 connected to 192.168.50.243 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  11.3 MBytes  95.2 Mbits/sec                  
[  5]   1.00-2.00   sec  11.2 MBytes  94.2 Mbits/sec                  
[  5]   2.00-3.00   sec  11.2 MBytes  94.1 Mbits/sec                  
[  5]   3.00-4.00   sec  11.2 MBytes  94.1 Mbits/sec                  
[  5]   4.00-5.00   sec  11.2 MBytes  94.1 Mbits/sec                  
[  5]   5.00-6.00   sec  11.2 MBytes  94.1 Mbits/sec                  
[  5]   6.00-7.00   sec  11.2 MBytes  94.1 Mbits/sec                  
[  5]   7.00-8.00   sec  11.2 MBytes  94.1 Mbits/sec                  
[  5]   8.00-9.00   sec  11.2 MBytes  94.2 Mbits/sec                  
[  5]   9.00-10.00  sec  11.2 MBytes  94.1 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.04  sec   113 MBytes  94.3 Mbits/sec    0             sender
[  5]   0.00-10.00  sec   112 MBytes  94.2 Mbits/sec                  receiver
osmc@osmc:~$ iperf3 -c 192.168.50.243
Connecting to host 192.168.50.243, port 5201
[  5] local 192.168.50.119 port 39894 connected to 192.168.50.243 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  11.5 MBytes  96.5 Mbits/sec  326   28.3 KBytes       
[  5]   1.00-2.00   sec  11.2 MBytes  93.8 Mbits/sec  358   28.3 KBytes       
[  5]   2.00-3.00   sec  11.4 MBytes  95.4 Mbits/sec  319   33.9 KBytes       
[  5]   3.00-4.00   sec  11.2 MBytes  94.4 Mbits/sec  417   29.7 KBytes       
[  5]   4.00-5.00   sec  11.2 MBytes  93.8 Mbits/sec  350   31.1 KBytes       
[  5]   5.00-6.00   sec  11.2 MBytes  93.8 Mbits/sec  376   21.2 KBytes       
[  5]   6.00-7.00   sec  11.2 MBytes  93.8 Mbits/sec  325   28.3 KBytes       
[  5]   7.00-8.00   sec  11.2 MBytes  93.8 Mbits/sec  376   32.5 KBytes       
[  5]   8.00-9.00   sec  11.2 MBytes  94.4 Mbits/sec  354   33.9 KBytes       
[  5]   9.00-10.00  sec  11.2 MBytes  93.8 Mbits/sec  373   31.1 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   112 MBytes  94.4 Mbits/sec  3574             sender
[  5]   0.00-10.04  sec   112 MBytes  93.7 Mbits/sec                  receiver

The high retries and small cwnd still don’t look right. Please

  • describe the network infrastructure between the RPi and the Vero4k+
  • if you use iperf3 -R -c ... on the Vero4k+ you should show the output on the server side (RPi); always the output showing you the retries and cwnd

I wonder whether this is really the root cause or we see here multiple problems.

1 Like

Certainly could be multiple problems, I suppose. Network goes like this:

Vero to ASUS RT-92U router via Ethernet (they’re both in the A/V cabinet upstairs); ASUS to Ethernet switch via Ethernet; Ethernet switch to RPi via Ethernet (switch and RPi are both in basement).

Here’s iperf3 from RPi to the server running on the Vero:

dietpi@DietPi:~$ iperf3 -c 192.168.50.119
Connecting to host 192.168.50.119, port 5201
[  5] local 192.168.50.243 port 54466 connected to 192.168.50.119 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  11.9 MBytes  99.7 Mbits/sec    0    144 KBytes       
[  5]   1.00-2.00   sec  11.3 MBytes  94.8 Mbits/sec    0    144 KBytes       
[  5]   2.00-3.00   sec  11.2 MBytes  93.9 Mbits/sec    0    150 KBytes       
[  5]   3.00-4.00   sec  11.2 MBytes  94.4 Mbits/sec    0    150 KBytes       
[  5]   4.00-5.00   sec  11.2 MBytes  94.4 Mbits/sec    0    150 KBytes       
[  5]   5.00-6.00   sec  11.1 MBytes  92.8 Mbits/sec    0    157 KBytes       
[  5]   6.00-7.00   sec  11.6 MBytes  97.0 Mbits/sec    0    157 KBytes       
[  5]   7.00-8.00   sec  11.2 MBytes  93.8 Mbits/sec    0    157 KBytes       
[  5]   8.00-9.00   sec  11.1 MBytes  92.8 Mbits/sec    0    157 KBytes       
[  5]   9.00-10.00  sec  11.2 MBytes  93.8 Mbits/sec    0    157 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   113 MBytes  94.7 Mbits/sec    0             sender
[  5]   0.00-10.04  sec   112 MBytes  93.8 Mbits/sec                  receiver

Since you have a large number of retries in one direction and none in the other I strongly suspect you have a cable that has a single wire inside that is damaged or is making a bad connection on one of the RJ45 connectors. You need to do some cable swaps running iperf to see when it goes away. Unfortunately a cheap cable tester isn’t going to be able to diagnose a marginal connection so your going to have to play the guessing game. If these runs are ones you did up yourself and terminated with plugs then you might consider redoing them with punch down jacks and patch cables.

2 Likes

Thanks for this reply. I swapped out each cable involved (I didn’t realize I had so much cat5 sitting around!) and am still getting the same iperf3 output. I also changed the jack I was using on my Ethernet switch, just for kicks. So probably not my cables; not the Ethernet port on the RPi, since this was happening when I was using the USB3–>Ethernet dongle, too. :thinking:

If you not using the same cables or the same NIC’s then instead of checking from the Vero to the RPi instead check between a PC and the respective devices to try to narrow things down. You can get iperf for windows and run it from cmd. It is also possible for a switch to go bad. They sometimes don’t just fail outright but instead start giving transmission errors or slow down. This is much less common than a bad cable, jack, or plug, but I have run across it personally on more than one occasion over the years. I think the fact that the retransmits are in only one direction rules out the likelihood that it is an externally induced signal degradation (like laying it parallel to a power cable).

1 Like

I’m not sure if we’re hitting a dead end here. Might be the retries are just symptoms of a saturation of one of the component in the intranet including the NICs but the reported bandwidth should be high enough for any video stream.

I suggest to switch the NFS stack and use kernel NFS instead of the kodi’s NFS implementation.
@nicheplayer Have a look at Mounting network shares with autofs (alternative to fstab).
By that you mount your sources to a point in your local root file system but have to setup again the sources in the mediacenter accordingly (and remove the old one).
Another advantage would be that we could easily test the file transfer from the NFS server with Linux on-board sources and tools.

1 Like

There is a quicker and easier way then dumping your library and starting over…

2 Likes