Bandwidth and buffering issues on some scenes

Ah, ‘quake-cam’. I hate that.

The video cache uses 3x the value set (so 1.5GB).
That probably leaves you with enough resources; but check free -m during playback to be sure.

That’s good to hear. I’m going to see if I can PM you a build with some buffering improvements. It lets me play the 400Mbps Jellyfish clips over Fast Ethernet; but it can cause playback stalls.

1 Like

Haha Quake-Cam, made my day!

Looking forward to your PM, if there’s anything I can help with just shout :slight_smile:

So out of curiosity I ramped up the buffersize to a Gigabyte.
Now I know this is highly volatile and by all means should crash kodi.

The result was very unexpected. It seems that I can set the buffersize to any large amount (tested with 2G even) and I’d always end up having at exactly 64.5MByte free memory and between 300 and 500 Mbyte memory-cache.

On the playback side, It passed the magical 4 minute mark. It held up fine until 15 minute into play, there it has the first shutter. Pause, Skip 10s back/forth, Play didn’t help. It kept interrupting every 30-45s from that point on.

So… What’s up with that 3x Usage? Is there a technical documentation why it is like this?

At this point I’m very curious about it and I’ll poke around some more over the weekend

http://kodi.wiki/view/caches_explained

http://kodi.wiki/view/HOW-TO:Modify_the_video_cache#Cache_settings

Neither of those explains why kodi is duplicating memory allocations rather than using shared memory.

My specific question was, why kodi leaks memory at a factor of 3.

Which part forks instead of threads which caused the duplication of memory allocated?

In a normal threaded application, memory is usually shared which leverages the requirement that every thread needs a copy of the current allocated memory.

//EDIT:

I’ve compiled the last 3 stable versions on my gentoo workstation. I’m using the patches that fome with my default tree.
I’ll check with gdb which component leaks memory or which duplicates it

A memory leak normally manifests itself by a process using an ever-increasing amount of memory. I can’t see anything in this thread where that’s being suggested.

As to the 3-times allocation, I’ve not read the code and the documentation I know of simply states it as a fact. However, I can speculate that there are possibly 3 cache “buckets” - one currently being read, one still being filled up and one being erased - that are dealt with on a round-robin basis. It’s probably a lot easier to program it that way than having a single circular buffer.

As for now there really isn’t any document about how memory is being handled, I must asume the worst by default.

But regardless, running 3 buckets makes little sense because you wouldn’t need to actively free the erasing bucket. You would simply overwrite it with new data. From a computational complexity standpoint the erasing doubles the cycles involved in processing the data.

A single reserved memory area with two sliding windows pointers should be optimal.
PointerA moves at playback speed (bit rate).
PointerB moves at readfactor/linerate.
Constraints of Addr(PointerB)%CacheSize < Addr(PointerA)%CacheSize should suffice to avoid overlapping memory operations.

I’d like to see the actual code where the caching happens, going through the threads with gdb is a mess :wink:

Well, as you know, it was just a guess, but what’s completely clear is that the documentation says that the buffer uses 3 times the specified value. What’s your plausible theory?

Go for it! It’s all open source. If you figure out how it’s working, please let us know.

Or simply go ask Kodi themselves. They have a forum.

It actually is using a Circular Cache with a sliding window of 4th of the cachesize.

Now the operation used doesnt inherit a requirement of 3x free cache size.

Even when Multi-Stream is used, they divide the cacheSize by two before duplicating the CircularCache object.

Still digging though

I came across this post on the Kodi forum from 2012:

In the Wiki it states that the cachemembuffersize setting consumes 3X the amount of ram, howerver I was wondering if that is still the case or if it is out of date?

Looking at the source it appears that xbmc / xbmc / filesystem / MemBufferCache.cpp uses that amount however xbmc / xbmc / filesystem / FileCache.cpp doesn’t seem to use that caching strategy anymore (as of March 25th, 2011)

I can’t even find MemBufferCache.cpp in the git repo, so it’s probably been removed, suggesting that the 3x requirement was probably superseded but nobody bothered to tell us.

yeah that matches my observations and the current cache related code in place.

It also seems that at least the current OSMC build handles OOM situations with ease. Enabling a higher setting than physical memory available no longer crashes kodi, or at least I didnt had any crashes yet with a 1G-2G cache setting

//EDIT:
In all fairness, I did add an eMMC for additional storage and swap. Just in case kodi really was about to claim the full 2G. Nontheless it never really claimed that much and always kept precisely 64MByte free ram and 0 swap used.

I’ve taken a look at this, and there are some improvements that may improve the playback of very high bitrate material. However – for a good few seconds, you may find yourself seeing no picture, and it is subject to stalls.

I’ve PMed you a patch if you want to try; but it’s too experimental for me to include in OSMC by default.

Sam