SSL/TLS error connecting to network share over FTPS

[Tom_Doyle]

Hi,

Try increasing the readfactor to 5, if that doesn’t help please provide mediainfo for stan & ollie

Thanks Tom.

[bmillham]

@richy

Thank you for this but unfortunately my DNS 320 does not support SSH, at least not without significant modifications (by installing “fun_plug”) which are too cumbersome in my circumstances. It would also involve my NAS hard drives being wiped of all data which is not happening.

That’s sad… And a strange limitation.

I assume without SSH access that you can’t install iperf3?

Here is a way to test the copy speed from the Pi or Vero:

dd if=/mnt/PATH/TO/FILE of=/dev/null bs=1024 count=10M

That will copy the first 10G of a file, showing the speed once a second.

[richy]

Yea it’s only an entry level NAS but still

I assume without SSH access that you can’t install iperf3?

You would really know better than me but I think it’s not an option

[richy]

Okay so adding only the readfactor at 5 in my cache settings has done the trick. Playback starts pretty quickly and there is no skipping, judder etc.

Some minor issues just that are left:

1. skipping ahead or back more than like 10 seconds during playback often results in video freezing and then RAM maxing out. Only resolution is to press stop and wait about 10 minutes for life to return, or a hard reboot.

2. idle time for users on the FTP server is maxed to 10 mins, so if a user does not start any video/music within 10 mins of boot up then the pi must be restarted in order for the FTP to become accessible again.

Again, minor issues but would be nice to get them resolved.

I should be able to test the remote connection in a few days time and will update again.

Thank you @bmillham and @Tom_Doyle for all your help with this.

[Tom_Doyle]

Hi,

@richy

skipping ahead or back more than like 10 seconds during playback often results in video freezing and then RAM maxing out. Only resolution is to press stop and wait about 10 minutes for life to return, or a hard reboot.

Is this on the pi, is memorysize are default?

@richy

idle time for users on the FTP server is maxed to 10 mins, so if a user does not start any video/music within 10 mins of boot up then the pi must be restarted in order for the FTP to become accessible again.

Is this a new issue or did kodi ftps have the same issue? sounds like a server side issue.

Thanks Tom.

[richy]

@Tom_Doyle

Is this on the pi, is memorysize are default?

Yes, on the pi with default memorysize. Should I try tweaking it? It’s the Pi B+ fyi.

@Tom_Doyle

Is this a new issue or did kodi ftps have the same issue? sounds like a server side issue.

I think this was always an issue but has only become apparent now, and yes very much looking like this is just the server limitation which I will just have to live with, unless using autofs instead of fstab might help?

[darwindesign]

@richy

Yes, on the pi with default memorysize. Should I try tweaking it?

If you only set the readfactor and nothing else on a RPi then Kodi should not use any significant amount of RAM than it was without that change. You might want to use top to try to see where the RAM is being used. Unless you also set the buffermode then I don’t even think that setting should have been active with a system mount. I think we would need to see exactly what you put in your advancedsettings.xml file.

@richy

unless using autofs instead of fstab might help?

fstab is going to open the connection and assume the connection is good. If your NAS is going to drop on idle then autofs would be the most likely solution as it closes the connection on idle and reconnects on demand.

[Tom_Doyle]

HI,

Ok worked autofs, hope this helps the idle issue. Overview of autofs can be found here:

Mounting network shares with autofs (alternative to fstab)

This how-to describes how to mount your network shares via autofs. Autofs should be as performant as fstab based mounting but has the advantage that your shares are mounted on demand which also should give you more reliability in case of network interuptions. First install the autofs package sudo apt-get install autofs If you already have mounted your shares already via fstab comment out that line in fstab and reboot. For this how-to we are going to use a directory for each server, if you wa…

for ftps:

/etc/auto.master:

/- /etc/auto.ftp.shares --timeout 60, --ghost

@richy The below autofs entry wasn’t working as it needed a space before the port (I’d only tested using 21 this morning) I’ve corrected this below now.

/etc/auto.ftp.shares:

/mnt/ftp -fstype=fuse,ssl,allow_other,no_verify_peer,no_verify_hostname,user=USER:PASWORD curlftpfs#192.168.1.13:3688

/mnt/ftp -fstype=fuse,ssl,allow_other,no_verify_peer,no_verify_hostname,user=USER:PASWORD curlftpfs#192.168.1.13 :3688

Try that and let us know how you get on.

As for the seeking issue, you try increasing the readfactor to 6 or 7; I wouldn’t go any higher though.

Thanks Tom.

[richy]

Okay so I now have the RPi in a remote location and have been able to test it out.

I decided to stick with fstab for mounting the remote share as autofs seemed to be glitchy.

I’m able to log into SSH and FTP on the pi remotely, however the fstab is not able to successfully mount the remote share. I’ve made sure to check that the remote share address was changed in the fstab beforehand but it just won’t connect.

I’m using duckdns as a dynamic DNS provider. Details here and here.

I was thinking it must be something to do with the port, but I can easily login from a remote location to FTPS using Android apps using the same address+port, so really not sure what the issue is here.

Attempting to connect via command line looks like it succeeds (I think) but the when checking the mounted directory it is empty and so clearly didn’t work. Here is the log for that.

Appreciate any suggestions thanks.

[Tom_Doyle]

Hi,

Please provide the fstab entry for the remote share and full kodi logs.

Thanks Tom.

[richy]

fstab entry (credentials omitted)

curlftpfs#user:password@mydns.duckdns.org:3688 /mnt/ftp fuse ssl,no_verify_peer,no_verify_hostname,allow_other,x-systemd.automount,noauto 0 0

Kodi log
https://paste.kodi.tv/enopuvutah.kodi

thanks

[bmillham]

If you are willing to let me experiment, could you setup another user/password and PM me the details? I’ll give this a try here to see if I can get it working if you will.

[richy]

Have messaged you with the details. thanks

[bmillham]

I think I found what’s going on. I’m able to connect, but when I try to do an ls, I get:

ls: reading directory '.': Input/output error

Turning on the debug option, I noticed that after the initial connection, ftpfs was trying to use your internal IP, so while the control channel seems open, it’s unable to transfer data. I added skip_pasv_ip to the options, so my fstab looks like this now:

user:password@mydns.duckdns.org:3688 /mnt/ftptest fuse.curlftpfs ssl,no_verify_peer,no_verify_hostname,allow_other,skip_pasv_ip,x-systemd.automount,noauto 0 0

and now it works! Notice one other small change that I made, instead of using the older curlftpfs# syntax, I used the newer fuse.curlftpfs syntax. Not really needed, but looks cleaner.

[richy]

Awesome, I can confirm it works! Will test further but for now it looks like I’m all set up! Thank you @bmillham and @Tom_Doyle for all your help with this

It might be worth condensing this thread into a guide similar to those on configuring fstab for samba and NFS shares.

I don’t suppose any of this would be possible on an unrooted android smartphone?

One final somewhat unrelated issue I noticed when setting the pi up in the new remote location is that in MyOSMC add-on in networking I only have a menu item for wired ethernet but nothing for wireless (WiFi), is this a known issue on 18.5?

[darwindesign]

@richy

One final somewhat unrelated issue I noticed when setting the pi up in the new remote location is that in MyOSMC add-on in networking I only have a menu item for wired ethernet but nothing for wireless (WiFi), is this a known issue on 18.5?

Are you using an older RPi with a USB wifi dongle? Support was dropped for some adapters and if you wish to know more about that there are several lengthy threads covering that topic.

[bmillham]

@richy

It might be worth condensing this thread into a guide similar to those on configuring fstab for samba and NFS shares.

I was thinking of doing that. I’ll also get it working with autofs so I can update that topic.

How is the performance for you using ftpfs? When I tested between 2 systems at home, the performance was not real good. Probably fine for up to 1080p, but I think it would have problems with 4K UHD. I haven’t tried yet, but I believe that there is an option to only encrypt the control channel and not the data, so login information, etc. would be encrypted, but the actual file data would not be. This would probably give you a major speed boost. I’ll update this thread once I’ve tested and confirmed that.

[bmillham]

I did some more local testing. I wanted to verify that the changes would also work on your NAS, but it looks like you removed the account you setup for me so I couldn’t. So give this new fstab entry a try and see how it works for you:

user:password@mydns.duckdns.org:3688 /mnt/ftptest fuse.curlftpfs ssl_control,ftp_method=singlecwd,no_verify_peer,no_verify_hostname,allow_other,skip_pasv_ip,gid=1000,uid=1000,x-systemd.automount,noauto 0 0

I made a few changes:
ssl_control will only encrypt control messages. With this option data transfer is not encryted, and is about 6 times faster in my local testing.
ftp_method=singlecwd should speed up things a little bit also.
gid and uid=1000 so that files will be shown as being owned by osmc/osmc

let me know if these options work for you.

[richy]

Testing 1080p playback worked okay today and seemed to hold up just fine with encryption. I think I will keep the encryption enabled as there is no real need to disable it for increased throughput in my particular circumstances as the RPi I have can’t handle 4K content anyway so no biggy. I suppose others may wish to use this option though.

I will test with the other parameters (ftp_method, gui and uid) in the next few days.

I have no easy/good way of getting detailed performance statistics on my FTP server, so I’ve enabled your test FTP account again and placed a decent bitrate 1080p file in the share if you want to test a little more. Just FYI my connection has 6.7 Mbit/s upload currently.

[bmillham]

If it’s fast enough for you, then probably leave things as they are, but I would recommend that you add the gid/uid options.

You can test the speed from the Pi with this command:

dd if=/mnt/PATH/TO/FILE of=/dev/null bs=1024 count=10M status=progress

Since you turned on the account again, I was able to confirm that the options do work on your NAS. I can’t really do a speed test, as I have satellite internet. It would not be a real-world test.
I did notice that just doing a cd and ls is noticeably faster, probably it’s the ssl_control option speeding it up.

Go ahead and disable the account again, as I’ve confirmed what wanted to know. (You may want to leave it disabled if you can so you could turn it on again if we need to do more testing).