Let me start by staying that OSMC is fantastic! Thank you for great software!
I am running on a Raspberry PI 3. I stream my videos from a local file server across WiFi. Throughput on the wifi network is typically 95 mbps, but there are flucuations. I am using the advancedsettings to enable chaching to disk, which in this case is on my micrSD card. This protects me from the flucuations and enables me to skip around commercials rapidly. The video stream is hidef MPEG2, 10-20 mbits/second. KODI is accessing the file through an NFS mount.
The problem: If I hit “stop” to stop the video the playback stops immediately, but there is sometimes a 5-10 second during which KODI is unresponsive / feels “hung”.
If I run while [ 1 ]; do cat /sys/block/mmcblk0/stat ; sleep 1; done
while the video is playing, I can see the there are times when there are a large number of “in flight” I/Os. If I hit stop when there are 100+ I/Os in flight, things are frozen until the inflight I/O count goes to 0 – about 5-10 seconds. If I hit stop while the inflight I/O count is very small (0 - 5), there is no noticable delay.
I believe that my microSD card is a good one – hdparm shows read speeds of 22MB/sec; dd test shows write speeds of 15MB/sec.
Is there any way to prevent this delay from happening? Maybe a way to flush the inflight I/O when I hit stop?
Thanks in advanced for your help!
Marc
<advancedsettings>
<cache>
<memorysize>0</memorysize> <!-- number of bytes used for buffering streams in memory
When set to 0 the cache will be written to disk instead of RAM -->
<buffermode>1</buffermode> <!-- Choose what to buffer:
0) Buffer all internet filesystems (like "2" but additionally also ftp, webdav, etc.) (default)
1) Buffer all filesystems (including local)
2) Only buffer true internet filesystems (streams) (http, etc.)
3) No buffer -->
<readfactor>4.0</readfactor> <!-- this factor determines the max readrate in terms of readbufferfactor * avg bitrate of a video file.
This can help on bad connections to keep the cache filled. It will also greatly speed up buffering. Default value 4.0. -->
</cache>
</advancedsettings>
Then don’t cache to disk. We default to 20mb but there is no reason why you can’t try 200mb. The root of the problem is the failure of the server to provide the data at a sustainable rate.
Hi ActionA, thank you for the quick response – very much appreciated.
To skip commercials efficiently I need a 5-minute buffer. The bit-rate ranges from 10-20 mbits/sec, depending on the show. I tried a 512MB cache, which gave me a bit over 5 minutes of buffer, but I still saw times when the inflight I/O count spiked – I suspect using that much memory might have driven more swap activity.
To be clear, I can get stable video playback with a much smaller cache. My goal is to buffer enough locally so that the 60-second forward-skip works quickly.
My guess is that the caching process is waiting for the in flight I/O to finish before it can exit.
I find that if I reduce “readfactor” to 3, the pending I/O doesn’t back up as much, but I can still experience up to a 5-second delay.
Is there some safe way to flush the pending writes from that process and/or limit the degree to which the writes are buffered?
A quick follow up. I found that lowering the readfactor to 3.0 reduce the degree to which writes fall behind – probably by reducing the write-load on the SD card. The worst-case “delay on stop” I’ve seen is 4-5 seconds, and it’s typically less.
Actually, this isn’t the issue.
The server is able to provide data at sustained gigabit speeds on wired ethernet. On the wifi bridge I am using to reach the PI3, I see average througput of 91mbits/second of data on the copy of a 1.1gbyte file from the server to the PI3. (Timed using “time cp” across an NFS mounted file system.) I ran the test twice and the screenshot from the Asus traffic monitor below shows that the wifi throughput remains stable across both tests. I’ve also confirmed this with iperf.
I am trying to use the cache to make the 60-second skip-forward / 10-second skip-backward less “laggy” on high bitrate files. When done against local storage, it’s fast. When don’t across a wifi link, the latency makes it a less desirable experience. Thus the reason I am trying to use the cache.
I believe that the core issue is using a low-cost device with limited memory and “slowish” storage. If the player were able to flush any inflight cache writes at the time it’s stopped, I suspect the problem would be solved.
Once again, thank you for all the time you put into producing OSMC and KODI – it’s a great product.
You don’t mention the brand, but SD card write performance can vary considerably depending on block size and the mode of writing.
Ultimately, it looks to me like you’re IO-limited and the SD card is probably not as fast as you believe it to be. Plus, the Pi 3’s SD card is on a relatively slow data bus:
Also, check out this (randomly selected) site for an idea of how markedly performance can vary and how poor SD cards’ write performance can be.
You might get better performance from using a USB3 thumbdrive for your video cache. USB3 can still provide performace benefits on a USB2 Pi 3 and they don’t cost a fortune. Always worth a try.
First,thank you for entering the thread – very much appreciated.
The card is a Sandisk Ultra micro SD card, but you are right that there are microSD cards that benchmark better than this one does on the PI.
I’ve tried various USB flash drives I have hanging around – one of them is doing a bit better than the others and better than the microSD card. None of them eliminates the problem.
I’ve been thinking of getting either a faster microSD card or a faster USB 3 drive – your fantastic point about the PI’s SD card being on a relatively slow bus makes the decision a bit of a no-brainer. I’ve been looking at the SanDisk Ultra Flair – benchmarks in some of the user reviews look good.
Ultimately, I do think that flushing the pending writes when the player is stopped, if possible, provides the most robust solution for prevening the long pause at the end. I’ve done some searching online – so far haven’t found any way to make this happen…
Very happy to have been of some help. I should also have added that using the SD card as a video cache is going to increase wear on the card significantly.
As far as I can see, the only tweakable parameter available to you is “readfactor”, which you’ve set = 4.0. Increasing this is likely to increase the unresponsiveness and decreasing it might not give you sufficient buffer to skip commercials. Since Kodi is the process writing to the system buffer, it’s difficult to see how you could cause it to be flushed when you stop a video.
It’s probably not ideal for your needs but while poking around the Kodi wiki I came across “edit decision lists” which, amongst other things, deal with detecting and skipping commercial breaks:
Ultimately, you might find you have to choose between either a big buffer or a temporarily unresponsive Kodi, though it’s always fun trying to “beat the system”.
PS You might want to check out the “watch” command, which can replicate the function of your while loop.
I’m a bit late replying here, but I think that using the SD card for buffering like this is a real bad idea. It will cause heavy wear on the SD card leading to premature failure.
I think that using a USB spinning drive would be a much better idea, and would probably perform better also.
I reformatted the USB 2 stick I had to ext4, mounted it via fstab with “noatime” set and sym-linked .kodi/temp to a directory on that USB drive. So far my longest delay on “stop” is under 3 seconds, and frequently closer to under a second. It works even with readfactor set to 100.0, which effectively means read as fast as the network will allow. The average network utilization was running around 70mbits/second.
Interestingly, the “I/O in flight” count will go high (130+) and stay high for as long as 10 seconds. But when I hit “stop”, it’s always able to flush all the buffers in under 3 seconds.
Much nicer.
I’m going to purchase that better USB stick and see if it improves things – I’ll let you know the outcome…
These points are all technically correct, but in practice the flash drive will work better for me due to the smaller footprint.
The lifespan of a typical consumer flash drive is 10K - 100K write cycles, which probably yields 40 years at the low end based on the following assumptions:
10,000 write cycles (Low end)
21 hours of playback/week (we actually use it less)
Average hour of playback is a 6GB file.
32GB flash drive with wear leveling (whcih ScanDisk claims it has).
This assumes that the deletion of the file doesn’t count as a write cycle – if it does we’re down to 20 years.
Given the performance I am seeing on my 5 year old USB flash drive, I suspect this one will meet my needs.
The actual life of the USB drive can vary enormously, depending on the quality of the memory cells and the sophistication of the device’s controller chip. You might only manage 1,000 writes on some drives with poor quality memory and little or no wear levelling. In this context, the USB drive should be viewed as a consumable item, to be thrown away once it wears out, and should have no important data stored on it.
This has also highlighted in my mind the issue of the Vero 4k’s eMMC memory, which is (AFAIK) soldered to the board and can’t realistically be changed. But that’s for another post…
While I don’t doubt you, do you have any figures you could offer regarding the eMMC’s performance and projected life. Would you say its internal data management regime matches SSD performance, for example?
Still tangentially relevant to this thread, what speed is the Vero 4K’s SD card bus?
Quick follow-up. I found a 32GB Lexar USB 3.0 drive that I missed during the first search of my desk. Formatted to ext4 and mounted it with “noatime” option. Readfactor set to 4.0. No lag on exit no matter how hard I try to time it to coincide with high “in flight” I/O counts!
We just watched Madam Secretary w/ readfactor set to 4.0 – 75% of the video was locally cached by the time I needed to do my first set of skip-forwards – about 15 minutes in the start!
In terms of lifespan – I’ll take the risk. It’s an $11 part and even if it only has a 1,000 write-cycle lifespan, it should last several years. If I’m wrong, the only thing on it is temporary files…
It’s been a while, but I tought I’d provide another udpate
I migrated my root partition to the USB 3.0 stick and performance improved dramatically – even with caching disabled.
Performance when skipping forward / backward during hidef mpeg2 video playback is way better in this configuration vs having the root partition on the SanDisk Micro SD card with caching to memory enabled. The difference is stunning.
In general, everything feels “zippier”.
Performance remains zippy all around even with the boot partition moved to an older, slower 2GB microSD card as long as I keep the root partition on the fast USB stick…
– about 15 minutes in the start!