I can’t add any new music files to my Kodi library. Those that were there when I first set it up are working, but now when I add new music files and select “scan to library”, it doesn’t give an error message but the new music files can’t be played.
I tried asking this question on the Kodi forum and the only only reply I got was to tell me to go post it here:
https://forum.kodi.tv/showthread.php?tid=379427&pid=3216199#pid3216199
The Kodi log files are available in that posting. Someone please have a look at them and let me know what is wrong.
I’ve now descended to computing md5 checksums of the files in question, and they don’t match the originals. Is it possible that OSMC can’t even copy files from one local mounted drive to another correctly?!? I’ve done this using the Files menu in Kodi’s Settings, and manually using the good old “tar” program through a pipe to another instance thereof, with similar results.
Is there any way that OSMC can be used to make a reliable copy of a file? Or do I have to take all my removable media out and mount them on a standard Linux laptop to do that simplest and most common of all possible computer tasks!?!
Hi @Tim02130,
your topic migrated from ‘can’t add music … to … library’ to a data corruption issue by copying files?
Reading your provided link, you seem to transfer files using an SD card. Perhaps, some structure is need to make some progress, here:
-
Are you sure that that source and target media and file systems are intact? I.e. h2testw is a nice tiny program to test sd-cards and other medias if you can mount them on Windows. A file system check is mandatory as well.
-
We need a complete log set and a better problem description from such a copy action which results in a corrupted file. For this the following instructions which help to upload such debug data:
To get a better understanding of the problem you are experiencing we need more information from you. The best way to get this information is for you to upload logs that demonstrate your problem. You can learn more about how to submit a useful support request here.
Depending on the used skin you have to set the settings-level to standard or higher, in summary:
-
enable debug logging at settings->system->logging
-
reboot the OSMC device twice(!)
-
reproduce the issue
-
upload the log set (all configs and logs!) either using the Log Uploader method within the My OSMC menu in the GUI or the ssh method invoking command grab-logs -A
-
publish the provided URL from the log set upload, here
Thanks for your understanding. We hope that we can help you get up and running again shortly.
OSMC skin screenshot:
If you use ftp to copy a file, make sure you activate binary mode.
I tend to configure NFS, and copy it over if required. Or I use scp to move files over.
I already said that my copies were between two locally mounted storage devices: An sd card and a usb memory stick. And I also said that i used the Files manager in the System Settings, as well as a “(cd /media/memorystick/subdir; tar -cf - .) | tar -xf -” shell command from within the target “/media/MyKodiLib/Music/subdir”, with similar results.
I subsequently also tried the standard linux “cp” command, and although it did not always yield a matching check sum, it usually did. So I eventually thought I’d finally gotten everything copied correctly, and then I hit an even worse problem, as described in my reply to JimKnopf above.
OK, I’ve now enabled debug logging as per your suggestion, and after rebooting (twice) I will attempt to reproduce the check sum problem. Before I begin, however, I have to tell you I’ve hit an even worse problem: Upon booting up OSMC this morning after shutting it down from the topmost “power icon” menu last night, I found that most of the music files on /media/MyKodiLib" sd card had been deleted completely! Just two folders seem to have been untouched. And I have no clue as to why.
Needless to say, I can’t use my Vero V for much until ALL these problems get fixed! And there are so many I hard even know where to start. I guess I’ll recopy all my Music files from the memory stick which contains them back to my sd card where the library is using the “System Settings → File manager” and then check their check sums. But how do you want me to try to figure out why the system is deleting my library? It seems to happen at random!
To copy files from device A to B (remote or not), I tend to use rsync in mirror mode.
Way more efficient, and it makes sure the checksums etc. do match automatically between source and target.
Isn’t that what the Kodi File Manager running under OSMC is supposed to do? Whatever the underlying linux command it uses may be, it should work!
OK, I’ve now enabled debug loggging, rebooted twice, and copied one of the mysteriously deleted folders (Cape Breton) and all its subfolders from my memory stick /media/ALLMYCDS to /media/MyKodiLib/Music (where my Kodi library resides) again. And once again none of the check sums of the original and copied files match!
The log was uploaded to https://paste.osmc.tv/atimojerom
And here is the terminal output from my computation of the check sums (I haven’t tried to play the copied files but given what happened last time I’m quite sure they’re too corrupt to play):
osmc@Kodi3:/media/MyKodiLib/Music/Cape Breton/Barry Shears - A Cape Breton Piper$ md5sum *
e1dc8e8623c4c389fd15e49f45722316 01 - Barry Shears - The Blackthorn Stick-The Stool of Repentance-I Was Kissed Yesterdat-The River Bend.flac
661f832ac406baa2f1a2c1210447cb91 02 - Barry Shears - The Iron Man-The Red Shoes-Jenny Bowser-Johnnie Made a’ Waddin o’it.flac
236747a1ef66cd314bbbedfb5468a291 03 - Barry Shears - Hector the Hero.flac
340f74b5152b58ffc3860986ed768893 04 - Barry Shears - Welcome to the Trossacks-Father Eugene’s Welcome to Cape North-Trip to Mabou Ridge.flac
a42b7df40c2d1ae0e3a19c525637d303 05 - Barry Shears - Rothiemurchus Rant-Lady Louden-The Eight Men of Moidart-Miss Proud, Angus Ross-Traditional Reel.flac
ca4d50e500401f80669d9e26e15fdf0e 06 - Barry Shears - Rann do Rob Mac-an-t-saoir, Piobair Mhic’ic Ailein.flac
1d6d027af784dc873a7a5b79552433df 07 - Barry Shears - The Hills of Glenorch-The French Road-Joe Hughie MacIntyre.flac
8fa76ff515082d38487c0265aba52648 08 - Barry Shears - Glencoe-Strathlorne.flac
d9e09d8f18dbd31a37a3617cee7f5075 09 - Barry Shears - Ann MacQuarrie’s-The Brolum-Scotsville-Put Me in the Big Chest-The Wedding Reel.flac
872e9cfbe039bf2c9d06c76e789afb9b 10 - Barry Shears - Gille Crubach-The Yetts of Muchart-The Smith’s Burn-The Black Mill.flac
517c5f87ad8dba96bea6d71e9a3ebe3f 11 - Barry Shears - The Foxhunter’s Reel.flac
5075a31ac8998887e2140d222a3e2c55 12 - Barry Shears - My Daughters Lullabye-Cronin’s Favourite-Milk in the Well-The Joys of Wedlock.flac
94c539f442af852462d026f033d3a026 13 - Barry Shears - Mary of the Bewitching Eyes-Cagaran Gaolach-I Would Willingly Return.flac
b44ffa6545f54978e529b31d2738bb8f 14 - Barry Shears - Elizabeth’s Big Coat.-Traditional Cape Breton Reel-Thompson’s Dirk-The Grey Old Lady of Raasay-The Birds Nest.flac
osmc@Kodi3:/media/MyKodiLib/Music/Cape Breton/Barry Shears - A Cape Breton Piper$ md5sum /media/ALLMYCDS/Cape\ Breton/Barry\ Shears\ -\ A\ Cape\ Breton\ Piper/*
cf6f5bdcf1ae6caa4708f806326906e7 /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/01 - Barry Shears - The Blackthorn Stick-The Stool of Repentance-I Was Kissed Yesterdat-The River Bend.flac
05b50311850b4bfad776805e644f945b /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/02 - Barry Shears - The Iron Man-The Red Shoes-Jenny Bowser-Johnnie Made a’ Waddin o’it.flac
bf38b748e69e217fbe24ee06dbee9663 /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/03 - Barry Shears - Hector the Hero.flac
0af87a1a437bd26810be61a8b975efc3 /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/04 - Barry Shears - Welcome to the Trossacks-Father Eugene’s Welcome to Cape North-Trip to Mabou Ridge.flac
037712103e076e72a826bb6bab7466b5 /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/05 - Barry Shears - Rothiemurchus Rant-Lady Louden-The Eight Men of Moidart-Miss Proud, Angus Ross-Traditional Reel.flac
3a0631287f4e63163b9e9d867245e388 /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/06 - Barry Shears - Rann do Rob Mac-an-t-saoir, Piobair Mhic’ic Ailein.flac
945912a7f2693d7a2cdf4ddd5a9f60c7 /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/07 - Barry Shears - The Hills of Glenorch-The French Road-Joe Hughie MacIntyre.flac
93a4341c7158b4186e2180e15f3a3c57 /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/08 - Barry Shears - Glencoe-Strathlorne.flac
3b767d98fef535caa8ad0bb8ddae922a /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/09 - Barry Shears - Ann MacQuarrie’s-The Brolum-Scotsville-Put Me in the Big Chest-The Wedding Reel.flac
4470a5950db730f916ef61766f4fea4b /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/10 - Barry Shears - Gille Crubach-The Yetts of Muchart-The Smith’s Burn-The Black Mill.flac
fce17962c3cb978527e998c58f8c561f /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/11 - Barry Shears - The Foxhunter’s Reel.flac
ca4d4367d2426a0ee2792ff7803e24fa /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/12 - Barry Shears - My Daughters Lullabye-Cronin’s Favourite-Milk in the Well-The Joys of Wedlock.flac
4d15bb0779bdf5c682c9d8a364f6f91a /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/13 - Barry Shears - Mary of the Bewitching Eyes-Cagaran Gaolach-I Would Willingly Return.flac
cf8f1e61558b909587d7ce9c66843046 /media/ALLMYCDS/Cape Breton/Barry Shears - A Cape Breton Piper/14 - Barry Shears - Elizabeth’s Big Coat.-Traditional Cape Breton Reel-Thompson’s Dirk-The Grey Old Lady of Raasay-The Birds Nest.flac
To continue the saga: I verified that the corrupted files could not be played, and then executed a “reboot” command from the shell. And upon logging back in, and found out that once again all the files in all the directories but for one had been deleted during the reboot!
For your debugging, I uploaded the log file immediately following that observation to: https://paste.oscm.tv/retukigago
Please tell me how I can use your system to reliably copy files from a memory stick to a Kodi library on an sd card, and keep them from being capriciously deleted after each reboot.
If files are disappearing then it sounds like a failing SD card that has the operating system on it, I presume this is a Raspberry Pi
First log:
This is a VeroV with 2 TB sd card inserted and a Sandisk Extreme Pro USB flash drive.
Ad-hoc, I do not see any activity in the logs that files were copied.
Second log:
Unfortunately this is a non-working link.
My advice: First check whether your devices (sd card and flash drive) are working without any issue.
- Perhaps, you copy all your data content from the sd card to a safe hdd on your PC, reformat the sd card with the formatter from the SD Association and use a tool l like h2testw (web page is German) to verifiy whether the sd card is intact.
- The same do with the USB flash drive but you can reformat it with WIndows standard tools. Also use a tool like h2testw to verify the media is error free.
If that is fine for both devices, try to reproduce the issue. If you think it’s a problem of OSMC, create detailed instructions what to do, so someone can reproduce your actions.
Let us know how far you got.
What filesystem do you have on the SD Card? I know that some Windows FS’s had issues with certain file sizes, and file name sizes too.
As the logfile link is not valid, I can’t take a look at it.
The sd card was formatted using the exfat format; here’s the output of “mount” for it:
/dev/mmcblk1p1 on /media/MyKodiLib type exfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0111,dmask=0000,allow_utime=0022,iocharset=utf8,namecase=0,errors=remount-ro,uhelper=udisks)
I’ve clicked the links in my previous posts and the first one works just fine. Here it is again:
This log was downloaded after copying the files from the memory stick to the sd card using the Kodi File Manager:
https://paste.osmc.tv/atimojerom
The log was downloaded after rebooting and logging back in to find that most of the files I’d copied had been deleted for no good reason,
https://paste.oscm.tv/retukigago
seems to have stopped working for no obvious reason (I’m quite sure I copied it right). All I can do is try again …
All your symptons (especially loss of files after reboot) sound like a failing SD Card.
Suggestion is try the card in another system and/or test it with H2Testw
Just looked at the logs.
There is nothing in the kodi logs. I suspect it is an issue with the exfat.
The only thing that I think could cause issues is the filesystem you use. Linux and Mac for instance have a very bad habit to use different type of charsets for files even though both are named UTF8. And if you add exfat on top of that 
Can you check if there is anything in the kernel ring buffer. Issuing a:
sudo dmesg -T
should show plenty. Look for filesystem related errors.
Could you make a test by formatting the sdcard as ext4 or btrfs filesystem?
Do a:
mkfs.ext4 -m 0 /dev/mmcblk1p1
That should do it. The “-m 0” just tells it to not reserve blocks for the super-user.
Once that is done, adapt the mount for it with - in /etc/fstab
/dev/mmcblk1p1 /media/MyKodiLib ext4 defaults,noatime 0 0
Then copy the files on it. While you copy the files, on another console monitor the kernel ring buffer with: sudo dmesg -Tw
I don’t understand why the link to the second log file isn’t working; I’m quite sure I copied it correctly.
I also don’t understand why the first log file contains no information about the File Manager copy command I used; that definitely happened and resulted in (corrupted) files getting copied! I did search for “File manager” in that log file, and found a reference to it on this line:
2024-11-20 11:06:09.830 T:2888 debug : ------ Window Deinit (FileManager.xml) ------
Unfortunately I have no way to analyze what happened after that.
The sd card was formatted as exfat on my linux laptop. I don’t have a windows system, and I would hope you don’t require your users to do so. I do have an old Mac and could perhaps reformat it from there, but it would be absolutely crazy if a formatting issue could do something like this; if the format is screwed up it shouldn’t work at all!
What I think I’ll do first instead is mount the sd card on my linux laptop and see if I can copy files to it without corruption.
FYI, here’s the “mount” commands output for the sd card on OSMC:
/dev/mmcblk1p1 /media/MyKodiLib exfat rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0111,dmask=0000,allow_utime=0022,iocharset=utf8,namecase=0,errors=remount-ro 0 0
Connect it to your linux laptop. Copy files on it, check md5sum, reboot, check md5sum.
I have a preference for exfat because the Mac I used to rip my CD’s can mount that but it can’t mount ext4. I believe (it was a month ago) that I formatted the sd card on the Vero V using the “mkfs.exfat” command. Unfortunately, I doubt I’m going to be able to decipher the output of “dmesg -Tw” on my own.
I guess I’ll try formatting it as “ext4” anyway, which should at least tell me if there’s something wrong with the card or the exfat format …
I really think the main issue comes from the filenames UTF8 encoding on the Mac then copy the files over. I spent hours trying to figure it out. I then stopped doing it all on mac and booted Linux to convert the disks with handbrake and stuff.
Mac UI is nice and juicy, but if you don’t use Mac afterwards problems arise.
That would be good to see if that is the issue! 
