I got a Vero 4K recently as an upgrade from a RPi3. Imported my settings from the Pi but it’s a pretty basic setup with samba dirs for TV and Movies.
I’m having a problem where the Vero will crash and freeze up while the system is not in use. When I turn it back on the HDMI output is just a solid color (the color seems to vary). It seems similar to the error on this thread, but my library scans happen fairly quickly and I couldn’t find any deep folders trees.
Similar to that thread description, the network is also down (no SSH or web UI). Have to power cycle the Vero to bring it back up.
As you might imagine, these are often difficult to solve.
How often has the crash occurred and what’s the approximate interval between crashes?
Is the position of your Vero4K likely to cause it to overheat?
First off, I’d suggest that you stop Kodi when you’re not using it. If the problem goes away, we at least have a possible candidate. From the command line, run the following:
sudo systemctl stop mediacenter
Whenever you want to use Kodi, the syntax to start it is:
sudo systemctl start mediacenter
(Just note that Kodi will automatically start if the Vero4K is rebooted.)
It might also be a faulty power supply, but one step at a time.
I’ve had the Vero about 2 weeks and it’s crashed this way 3 times so far: so every few days.
I don’t think that the Vero should be overheating as it’s got decent ventilation but I’ll reposition it so that it gets more ambient air. If I still get crashes I’ll see about starting/stopping Kodi.
As @dillthedog says, this can be a tricky one to diagnose as there could be a number of reasons that you are seeing this problem.
Indeed. In the thread @nickv2002 linked to, I believe the user’s issues were caused by potentially corrupt / problematic files that were causing a crash when library scanning was invoked. Do you have the Library Watchdog add-on enabled? This allows regular scans of the library without having to manually refresh them, but has sometimes caused problems in the past. If you do – I would disable that temporarily.
After several weeks of testing this seems to be a heat issue.
I spaced the Vero 4K out from my AppleTV where it was sitting on and the crashes stopped for several weeks. Then I put it back and saw the same type of color screen crash again after several days.
The AppleTV doesn’t get particularly hot but it seems like the Vero is particularly sensitive to localized heat.
In any case, easy fix. Thanks for all the help and great product.
We have been working on the thermal governor for the vero4k and this should prevent crashes caused by overheating when the device is used in hot locations with heavy cpu use.
My own vero4k on my main TV was freezing in the same way you describe - a solid colour when left “idle”, usually overnight.
Mine is inside a TV cabinet that gets quite warm due to other equipment and the device has many background services installed as well, which was sometimes pushing the CPU temperature to the point of instability.
Contrary to intuition Kodi uses a lot more CPU when left “idle” especially on the home screen or a screen with auto-scrolling text than it does when actually playing video, which is why the crash won’t typically occur when it is being used to watch video.
I’ve had no crashes at all since updating to the test kernel. You can try the test kernel early as described in that thread, or wait for the next monthly update due sometime in the next week or so which will include the fix.
The monthly update is out and the thermal governor changes are included.
I’d be interested to know whether your box is now stable in its original warmer location where it was previously crashing - it should be.
However for optimal performance in the long term I would leave it in a cooler location as if the thermal governor limit of 105C is hit it will start to gradually throttle the CPU speed back to reduce the temperature - the box will remain stable but run slightly slower than it would in the cooler location.
I installed the monthly update about 5 days ago, shortly after it went live. So far I’ve had the Vero 4K in a few positions and haven’t had any of the crashes described above where the screen will lock on random color.
Unfortunately, instead I’m getting some kind of crash that disables all video output and turns the LED on the front of the box red (in the previous color-lock crashes, the LED would remain blue). Again the solution here seems to be a power cycle.
I’m not sure if this new red LED crash is related to temperature. It’s happened to my Vero in two separate locations however which leads me to suspect that it’s not. Perhaps something else changed with the kernel update? (And, if it’s not related, I’m sure there’s another tread tracking the issue.)
The thermal limiter will throttle back the cpu speed from 1.5Ghz → 1.2Ghz → 1.0Ghz → 0.66Ghz as necessary to try to keep the temperature under 105C.
Normally under an extreme load it should stay below about 110C. At 114C it will log temperature warnings in dmesg, and at 118C it will shut down the system to prevent further overheating, and as sam mentions this will cause the LED to go red.
As this is a shutdown it requires a power cycle to boot again. The reason why it shuts down at 118C after all attempts to regulate the temperature have failed (cpu throttled back to 667 Mhz) is because the device will crash at about 122C, and this sometimes causes it to get extremely hot, while shutting down safely will let it cool right down, so this is a last line of defence to protect the device.
So if you are now seeing it shut down with a red led it is indeed overheating. What sort of environment is it exposed to ? I have mine inside a closed TV cabinet along with a TV, Stereo amplifier and Xbox one (inside cabinet about 26C) and mine doesn’t come anywhere near overheating even with cpuburn-a53 running. (A very stressful CPU load designed to heat up a cpu)
You could try running the following script via ssh:
Leave this running, keeping an eye on it - it will show the SoC temperature (in millidegrees, so divide by 1000 for degrees C) and cpu speed, updating twice a second.
Above 105C the CPU speed will start to throttle back and it should not get any hotter than about 110C. If you see it continue to rise and hit 118C with the cpu speed at 667 and then the light goes red and the device shuts down it has definitely overheated, presumably due to an excessively hot environment or lack of ventilation.
A picture of where the device sits to provide some context may be helpful.
Unfortunately we need kernel messages from previous boots which are disabled by default with OSMC. To activate and provide such information, please, follow the steps below:
login via SSH to the OSMC device, user osmc, password osmc
cd /var/log
sudo mkdir journal
(from now, kernel messages are written to new directories for every boot)
sudo shutdown -r now
now wait for the issue/event which is the problem of this topic
once it happens again and you are forced to reboot the OSMC device or it rebooted automatically, you’ve to identify the right kernel message log:
7.a) login via SSH and invoke sudo journalctl --list-boots --no-pager
7.b) the lines start with an index id like 0, -1, -2, etc. and contain the date and time when log was started
upload the appropriate kernel log using sudo journalctl -k -b <identified index> --no-pager|paste-log
(replace <identified index> with the real index id, see above)
also, upload the appropriate full log using sudo journalctl -b <identified index> --no-pager|paste-log
(replace <identified index> with the real index id, see above)
provide the returned URLs here
don’t forget to remove the created journal directory otherwise your system’s root file system gets filled
11.a) login via SSH
11.b )cd /var/log
11.c) sudo rm -R -f journal && sudo reboot
65C is fine for idle - anything from about 50C to about 105C is normal depending on how busy the cpu is. Over 110C is a concern. You say it crashes when “idle”, not when playing video ?
The way Kodi works the CPU use and SoC temperature is typically lowest during video playback, and highest when Kodi is on a menu that animates or has scrolling text.
Is it possible that the crashes have occurred when a Movie/TV show has finished and it has been left sitting on a listing screen where text is scrolling ?
Can you try leaving Kodi on a screen with scrolling text to see what temperature it gets up to ?
Are there any other background services installed such as Transmission/Deluge that may periodically start using a lot of CPU ?
As for the location of the device, I would be very unhappy with positioning mine like that from a heat perspective - you have it crowded in between three sources of heat - an Apple TV (latest model ?) an amplifier and a switch/router.
I presume that the amplifier has vents at the top which are blocked due to being squeezed into the cabinet ? If so that is going to force all the heat out the side vents. If the vero is resting against the side of the amplifier you may also get some heat from conduction.
I see you’ve put something under it to lift it up from the Apple TV - that may reduce the heat slightly but not by much in a confined space. In short your positioning is very problematic. Are there doors on the front of the cabinet that are usually closed or is it open ?
Without seeing a wider shot it’s hard to be sure but from what I can see above I would relocate it to the right of the white box with 2-3" gap between them. Heat travels up so it’s likely to be quite a bit cooler at the bottom than the top left.
Temperature when showing that screen (the episode plot summary on the left scrolls):
temp: 73000
cpu speed: 1512000
The AppleTV 4K below my Vero is a little warm: but I rarely use it so it’s asleep 99% of the time. The receiver next to it also puts out little heat. The tube space is a tube: so round. so doesn’t block the holes underneath the Vero. As you can see there’s no front cabinet doors.
Usually my Kodi box sits on the home screen where there is no scrolling, just a list of recent episodes. I can get notifications to update things from SickRage which would cause some UI animations and changes, but that only happens after a new episode shows up etc. I don’t run any torrent services (or much of anything other than Kodi) on the Vero: that’s what the large NAS below it is for.
I created the journal folder as described by @JimKnopf and will upload logs after future crashes. Thanks.
Looks like you don’t really have anywhere else cooler it could sit other than behind the TV.
For the moment I think you’ll just need to keep an eye on the temperature to see if you can capture the last measured temperature before it shuts down. It’s possible that a background service like Sickrage is contributing - although it spends a lot of time idle it will periodically wake up to perform tasks like scanning incoming downloads.
Are you able to leave the ssh session open running the temperature monitor script ? If so if it does crash or shut down with a red LED you should have a record of the last measured temperature.
I tweaked the temperature and frequency watch command listed above to write the current values to a file every 10 seconds and left that running via Byobu for the last few days. Got home from work today and found the Vero shut down with the red light on today. Here are the last values it recorded:
frequency: 667000
temperature: 117000
So it slowed down the CPU but still got up to about 117ºC while I was at work and it was set to do nothing but show the home screen. I’m not 100% sure there was no scrolling text but I doubt there was much.
The Vero’s position is still the same as pictured above. I live in the home probably got to about 26ºC inside today: but nothing too warm. Also to be 100% clear: SickBeard is running on my NAS not Vero, but it does occasionally add new items to the Kodi library.
None of this seems like it should cause the Vero to overheat in its current position when I’m not even at home using the TV though.
I’ll move the Vero around a bit and see if I can get it in a cooler spot though. Any other advise?
-N
PS @JimKnopf the TV is a LG OLED model C7P a bit pricy but I got it on sale recently as before the 2018 models (which change little) came out. I highly recommend them if you value picture quality and dark blacks.
