[Howto] Newbies guide to filesystems

Yes, but you’ll need to let the media center provide the merged view of the media. Each file system on each disk needs to remain separate. I.e. no advanced file system capabilities like LVM can provide. That really shouldn’t be an issue, so it isn’t a big deal like it was 10 yrs ago.

I use ext2 on boot partitions on linux. They are tiny and almost always read-only, so using a journal isn’t all that helpful.

Journalled file systems help with recovery time after a hard failure, usually caused by power shutoff. In the old days, when most file systems were not journalled, running an fsck might take hours. With a journal, it is seconds.

I don’t go out of my way to use it, but some installers choose ext2 for /boot, which is fine. I don’t upgrade them to ext3 (which is an easy thing) to get journalling added.

As for flash drive file systems, fat32/exfat are probably the most popular because every device handles them. If you only need Linux support, then there are 5 other file systems for flash that include POSIX permission support. I think Samsung just released their newest FS as F/LOSS. Most use the FUSE interface, so any Linux can make use of them. It can be fun to play with them for a few days to see what it is all about. Wikipedia has a list.

Older androids won’t handle exFAT IIRC. I had to format a 128G card as FAT32.

Ok, here are now the full test results from Vero, next is the Pi3
The bad performance of exfat via Samba must be related to the issue @jimkopf sees. It worked both times I tried (Linux client) but both times took double the time the other filesystems needed.

Command Vero
Time (secs) SyS Time (Secs) CPU (%)
System User
time sh -c “dd if=/dev/zero of=/mnt/ntfs_test/test.tmp bs=32k count=2000000 && sync” 1844 284 66 40
time sh -c “dd if=/dev/zero of=/mnt/btrfs_test/test.tmp bs=32k count=2000000 && sync” 1609 137 16 0.1
time sh -c “dd if=/dev/zero of=/mnt/ext2_test/test.tmp bs=32k count=2000000 && sync” 1640 303 7.4 0.1
time sh -c “dd if=/dev/zero of=/mnt/ext4_test/test.tmp bs=32k count=2000000 && sync” 1596 299 9.6 0.2
time sh -c “dd if=/dev/zero of=/mnt/exfat_test/test.tmp bs=32k count=2000000 && sync” 1615 156 5.4 0.2
time sh -c “dd if=/dev/zero of=/mnt/ntfs_test/test.tmp bs=4k count=10000000 && sync” 1497 241 50 34
time sh -c “dd if=/dev/zero of=/mnt/btrfs_test/test.tmp bs=4k count=10000000 && sync” 1010 126 14 0.1
time sh -c “dd if=/dev/zero of=/mnt/ext2_test/test.tmp bs=4k count=10000000 && sync” 1050 206 5 0.2
time sh -c “dd if=/dev/zero of=/mnt/ext4_test/test.tmp bs=4k count=10000000 && sync” 1002 192 11 0.1
time sh -c “dd if=/dev/zero of=/mnt/exfat_test/test.tmp bs=4k count=10000000 && sync” 1016 151 8.4 0.6
time sh -c “dd if=/mnt/ntfs_test/test.mkv of=/dev/null bs=4k && sync” (44G File) 1195 146 49 4
time sh -c “dd if=/mnt/btrfs_test/test.mkv of=/dev/null bs=4k && sync” (44G File) 1091 182 42 1
time sh -c “dd if=/mnt/ext2_test/test.mkv of=/dev/null bs=4k && sync” (44G File) 1152 256 30 1.6
time sh -c “dd if=/mnt/ext4_test/test.mkv of=/dev/null bs=4k && sync” (44G File) 1126 175 38 1.4
time sh -c “dd if=/mnt/exfat_test/test.mkv of=/dev/null bs=4k && sync” (44G File) 1176 130 36 2.2
time sh -c "dd if=/mnt/ntfs_test/test.mkv of=/dev/null bs=32k && sync (44G File) 1182 115 38 2.5
time sh -c “dd if=/mnt/btrfs_test/test.mkv of=/dev/null bs=32k && sync” (44G File) 1083 187 30 0.5
time sh -c “dd if=/mnt/ext2_test/test.mkv of=/dev/null bs=32k && sync” (44G File) 1161 245 28 0.6
time sh -c “dd if=/mnt/ext4_test/test.mkv of=/dev/null bs=32k && sync” (44G File) 1150 229 36 0.5
time sh -c “dd if=/mnt/exfat_test/test.mkv of=/dev/null bs=32k && sync” (44G File) 1201 142 47 2.2
Samba 44G file to NTFS 1675 76 66 39
Samba 44G file to BTRFS 1607 36 43 1
Samba 44G file to EXT2 1226 26 60 1
Samba 44G file to EXT4 1237 27 55 1
Samba 44G file to EXFAT 2424 78 32 2
1 Like

Wouldn’t any Android that doesn’t support exFAT need to be updated for security problems?

exFAT isn’t a hardware thing, it is software.

I really don’t know, but my Nexus4 handles exFAT since I was forced to a different OS when google dropped support well before end-of-life.

Maybe, but not version bumped if the manufacturer can’t be bothered (wants to sell new devices).

Did you try to open the file you copied? When I tested it I could get a large file transferred from Win10 to a exFAT over SMB (slow and no status) but the file was not readable. Repeating gave the same results. If I put a file on the exFAT in windows and then let that automount in OSMC I could read and transfer the file from OSMC just fine.

Yes I can play it and md5sum is also identical.
But I also just tried from Win10 and can confirm there it quite fast fails with an error.
So summary in the moment with Linux Client works but slow with Win10 it fails.

1 Like

Did I miss the part where you addressed USB performance?

Uh, transfer speeds over network are below 40MBs and plugging your USB3 drive directly in gets 150MBs+.
I have NTFS and ext4 and when I need to transfer a couple UHD Remuxes, I always unplug the drive from the Vero and plug it in to my laptop. Either that or setup the transfer before I go to sleep if I’m not planning on watching it that evening.
Why?
Because USB3 is WAY FASTER than USB2.
If you are transferring the file via Samba, you are throttled by the USB2 performance regardless of file system.
It’a not that hard to unplug a USB drive from the Vero and then plug it in to your laptop.

like i said before welcome to the new age we got something called network call me a douche for calling users like you out on it im perfectly fine with it, but there is a kernel of truth in it instead of getting offended by some of my words look at the empirical data that collected in this post when using ntfs on linux.

just think that shuffling around a usb disk is something we did in the 90s when we had shitty wifi and or shitty routers

Hey douche (you told me to), I have my laptop and Vero hardwired to the same ASUS gigabit router.
Network speeds are not an issue.
Neither are file systems.
The bottle neck is USB2.

USB 2.0 was released in April 2000, it’s now September 2019.

sigh well i see there isnt much use telling you to read the posts that both the mods and other users has added data to instead you wanna argue about words so im not that interested in continuing this discussion cause its not productive and a waste of time.

gl hf

Figures…
BTW, I have three USB 3.0 hard drives formatted with ext4 attached to my Vero 4K+, so it’s not like I’m some NTFS fan boy.
Like I said before, many times, when it comes to file transfers, which file system you use doesn’t really matter because they all perfrom faster than USB 2.0 specs.
Now if you want to run torrents in the back ground while your watching John Wick, well then yeah, you better be using ext4.
Cheers Toast!

PS: Humor me if you will, show me a benchmark of your transfrer speeds to a USB drive attached to your Vero over your network.

Samba 44G file

2 Likes

If I’m reading your post correctly it took approximately 21 minutes to transfer a 44GB file to ext4 (~35MBs) and approximately 28 minutes to transfer the same file to NTFS (~26MBs)?
I’m not sure what drive you were transferring to, if it was the same for both test, if the drive was empty, etc.
In my testing, I did a clean format to ext4 and was able to sustain 36~38MBs.
Reformatted the same drive to NTFS and got 34~36MBs.
That’s Windows 10 laptop SSD > CAT6 > ASUS Gigabit router > CAT6 > Vero 4K+ > USB3.0 Seagate HDD.
So a little bit faster with ext4, but marginal.
Both well below USB3.0 speeds.
If I do Windows 10 laptop SSD > USB3.0 Seagate NTFS HDD I get 170MBs+.
If I do Windows 10 laptop SSD > USB3.0 Seagate ext4 HDD I get 110MBs (this requires Linux File Systems for Windows by Paragon Software which cost me $20).
If the Vero had USB3.0 I would assume that both file systems would achieve 100MBs+ easily over gigabit network.

CPU usage is very interesting with NTFS using 66% system and 39% user where ext4 is using 55% system and only 1% user, but that’s to be expected.

Yes, same drive (1TB 3.5" HDD in a USB3 Enclosure) with different partitions for each filesystem.

I don’t have a Vero4k+ so had to test with a Gigabit USB Adapter which also would impact my results.

Well with a USB2.0 interface that would be expected :wink:

Yeah as I wrote fundamentally the speed difference can be ignored while the CPU usage might be the crucial point if you use any other programs that need CPU.

That could throw off your test right there.

Agreed.

I just picked up another Seagate 8TB USB3.0 HDD today from Costco (still on sale and my other one is full).
Formatted to ext4 and did a test transfer.

osmc@osmc:~$ df
Filesystem           1K-blocks       Used  Available Use% Mounted on
devtmpfs                774688          0     774688   0% /dev
tmpfs                   899236      16936     882300   2% /run
/dev/vero-nand/root   14499760    9510508    4229652  70% /
tmpfs                   899236          0     899236   0% /dev/shm
tmpfs                     5120          0       5120   0% /run/lock
tmpfs                   899236          0     899236   0% /sys/fs/cgroup
/dev/sdb1           7811937256      94240 7811826632   1% /media/Seagate8TB-2
/dev/sdc1           3904735508 2649830092 1254889032  68% /media/Seagate4TB
/dev/sdd1           4881179132 3694321532 1186841216  76% /media/Seagate5TB
/dev/sda1           7811937256 7704999820  106921052  99% /media/Seagate8TB
tmpfs                   179844          0     179844   0% /run/user/1000

sdb1 is the new drive.
Here’s what I get in the real world with ext4 over Gigabit network:

Untitled

It starts off with a 100MBs burst but levels off around 34MBs.

Why?

It won’t at all.

Spinning rust hard drives transfer data faster near the outer edge of the disk, because more bits pass under the read head per rotation.

If you have multiple partitions on the disk, the one that is closest to the outer edge will be faster on sequential reads. Unless something has changed, the outer edge of the disk has always contained the lowest numbered blocks/sectors/etc., so the first partition will usually be fastest on sequential reads and writes.