Hello everyone!
Unfortunately, quasar and other torrent streaming clients works unstable. Sometimes kids want to watch “some new crap” and i faced with the need to copy 1-2Gb avi to pi. And it is really sloooooow, around 600kbps.
I have rpi0w and rpi3b with stock clocks.
Firstly i launched iperf that showed me 17Mbps. Thats ok for my wlan.
So, copying 1.5Gb file via scp i got 800Kbps (avg)
I installed samba, got 1.04Mbps
ftp not tested yet.
Yea, my HDD connected to Pi without hub and powered by 1500W. Quasar showed me 1.4-1.7Mbps, so i think its not about power, but about frequency.
Generally kodi.bin on average loads cpu by 25%.
Cpu load (avg1) when idling is 1.56, scp - 3.67, smb - 2.87.
What the protocol you use for file transfer or how i can speed up copying?
Thanks!
Yep, there is ntfs. I’ll try to test ext4 on my job tomorrow, but for some reasons i’m not sure in success. Anyway, i’ll post results with smb, ssh, ftp, iperf and iometer
The fastest method would be to put the movie onto storage using a different machine, then unplug the storage and connect it to the Pi. Sneakernet.
The fastest network method should be over NFS to ext4. Setup static IP for the NFS server. NFS is hard to use with Windows, so most people setup samba. Win10 has become picky about some Samba parameters, so be certain you correct those in the smb.conf to get the required minimal version.
If you care about security, then use either sftp or rsync to transfer the files. Both of these protocols leverage ssh for the network encryption and authentication. There is a performance/transfer cost, especially on SBC hardware. If you setup ssh keys between the two systems for the userid(s) involved, it is both more convenient AND more secure. In general, sftp or rsync over ssh is considered secure enough for transfers over the internet, assuming you do not use passwords for authentication.
Make certain the storage is externally powered and use USB3 capable storage even though the Pi is only USB2 capable. The main issue is that any network transfers take bus bandwidth away from the USB storage. The pi architecture shares a single USB2 bus for all USB2 ports AND the ethernet connection. Trying to be cheap does have trade-offs.
For many people, the better choice is to use the Pi as a playback device, connected via ethernet (not wifi) and use NFS to stream local content from a GigE server on the network. DLNA can be used to stream content too, but it has effectively ZERO security. Anyone on the LAN can use any rogue DLNA controller to access all the content from the DLNA server.
Definitely avoid NTFS or FAT-whatever. NTFS can only be accessed using slow FUSE file system drivers. Ext4 uses fast, kernel, drivers. If you don’t have any option but to use NTFS, check out adding the “big_writes” option to the mount command. It is known to drastically improve performance. I don’t think FAT has any similar option. Best to stuck with ext4, if that is possible.
I would recommend completely eliminating the need to copy a file before being able to watch it.
Either have your files on your network (NFS) and simply add the folders as source to your Kodi Library or attach a USB harddrive and add it to your library.
Thanks for your detailed response!
I finally found time to test filesystems. I was mistaken in my skepticism, the results are really very different.
As Tom_Doyle said, the NTFS really creates a bottleneck.
I could not find the external hdd and used the sandisk 16gb usb flash drive as storage.
All tests performed on rpi3b kernel 4.14.78-4-osmc 2018.12-1.
Firstly, iperf2 with 1gbe switch between 1000baseT win-pc [client] and 100baseT rpi3 [server].
iperf showed 64Mbit/s what equals (64/8)*1024=8192,6 kB/s. The similar result was obtained between enterprise NAS running under linux and rpi. So, i took 8192kBps as limit, as ideal.
In fact, in most cases Windows PC is used for video uploads, so it makes sense to consider smb and ftp protocols in the first place.
I noticed, linux client transfering data in all cases a bit faster, so i used client tools for windows, such as pscp. It is fair.
Ok, the best time to finish the explanations, you probably tired to read this water.
As you see, XFS looks more magnetically than others. At the end of the test I stopped at it.
ext4 did not impress me. The fact is that the transfer rate is unstable through smb and resembles a sine wave. Probably filesystem not accepting new data in cache until it writes all existing cached data.
ssh and rsync use embedded encryption, that creates another bottleneck on cpu. Maybe on rpi0w and newer armv7 boards result could be better.
Of course i’ll try setup nfs, just for fun. I really think its a bad idea to host nfs on rpi, but it have a sense to try. I dont want to make some new home server where i could store media for osmc, couse I strive to reduce the amount of devices.
BTW, if you want to use rsync, and don’t need security, you can set up your pi as an rsync server, which will be faster than the default rsync-over-ssh.
The only negative for using a non-GigE Pi as the NFS server is network performance and hardware power issues. If you have a r-pi v4, with an external, self-powered, USB3 storage device using a Unix file system (not NTFS or FAT-whatever), then running it as an NFS server should be fine and fast.
But really the intent would be for the r-pi to be an NFS CLIENT, not the server unless you have a v4 pi or newer with real GigE and real USB3.
If you really want to avoid setting up servers, then the quickest method I have found by far is netcat, its great and I use it for transferring large files all the time, very low overheads, no encryption whatsoever.
If you’re using ntfs that is unfortunately your bottleneck, so all transfers will be slow.
