[SOLVED] Recent update issues......xml guide file, ssh password request delay

Help and Support Raspberry Pi
1

Wanted to circle back around with an update to this thread…and how it got solved (mostly luck :slight_smile: and to thank those who offered up suggestions, help and support…

Without repeating the whole posting, here is a link for anyone who’s experienced something similar, or is just plain curious…

How this problem was solved

Once again, not sure if this is the right place…so apologies in advance…

Everything seemed to be working fine…great, in fact. Then, a couple of mornings ago I went to check something out and noticed the TV Shows/Guide window had no program listings. I checked the .xml file (mine’s called tvguide.xml) and there were listings in there, but nothing was showing on the screen. The other thing I noticed (which or may not be relevant) is that the last update to the tvguide.xml file was 11/26/16, ~2 weeks ago (I have it set up to update at 3AM every morning).

I did install the recent (11/26?) update in the system, prob’ly a day or 2 after it was available.

One other thing I noticed (and unsure if it’s related either) is that before the update when I would ssh into the OSMC system, it used to respond very quickly asking for the password (set up script to do the osmc@ part). Now (after update) it seems to take 5 or 10 secs before I get the password request.

Anyone have any insight what might be going on here?

Thanks and cheers in advance…

2

Check your DNS/Network

  1. Check how fast are DNS queries answered. Log into osmc and run ping <any domain> and time how long it for the first line with the IP to show up
  2. Check your network speed either with iperf3 within your network or speedtest for internet related speed
  3. Check with sudo journalctl if your journal file is corrupted (how long does it take to show on screen)
3

About your ssh delay only: it’s a very common issue on linux. You can solve it by avoiding DNS resquest during ssh connection. Below the steps:

sudo nano /etc/ssh/sshd_config

Add this line anywhere in the file

UseDNS no

Restart sshd daemon

sudo systemctl restart sshd

1 Like
4

That is not a smart way, fix your DNS issues than you don’t have to do a workaround!

1 Like
5

I don’t agree. This ssh issue is well known and has nothing to do with a DNS issue. The delay may appear whatever your DNS status is, I invite you to look around to verify.

6

Well you claim that such a thing exists please show a confirmed bug report on that.
I never have came across such an issue in clean network/dns setups with linux machines.

1 Like
7

Me neither and as a DNS/nerwork issue would explain all of the OP’s symptoms then I think it is quite reasonable to assume his issue with SSH is simply a small side-effect of a more serious underlying problem that needs to be fixed rather than covered up.

8

Gentlemen, gentlemen,

Really appreciate the help. Didn’t mean to cause a conflict. Discussion is good. Let’s be civil about it :slight_smile:

Thanks for the input. Here are the results from @fzinken suggestions:

  1. ping from RPI 3 to local subnet: avg time= .55 ms. 1st line shows up almost immediately.

  2. Will take a bit to learn how to use iperf3. Speedtest (run from my Win 10 sys on same subnet as RPI3/OSMC) showed best case 1ms ping time, 78 Mbits/sec download. Cross country (SF Bay Area<-> NY) is 77 ms ping time, 72 Mbits/sec download. That’s pretty consistent.

  3. sudo journalctl shows up immediately.

As I thought about it a bit more, there seems to 2 things going which appear to be related (not sure how the ssh issue fits in): the .xml file used for storing downloaded station and guide info is not being updated and the information that is in there is not being displayed. The last part may not really be an issue and could be due to the fact that I set 14 days as the max guide view and it was 14 days (or more) when I noticed the problem(s). So maybe the part about the on-screen guide being empty is consistent and isn’t really an issue in this situation. Can’t tell at the moment.

I did check (ls -al) the ~/.xmltv/tvguide.xml and permissions/ownership seem to be the same.

I mentioned in my OP that this appeared to happen after I did the last “sudo apt-get update”, “sudo apt-get dist-upgrade”. So, to me it seems something in the update threw a glitch into things.

I tried @shamael’s suggestion about UseDNS No and now I can’t even get into the system (Connection refused). I’ll have to go use the direct keyboard/mouse interface and change that back :frowning: :frowning:

Open to other input.

Thanks and cheers…

9

I’m pretty sure I disabled DNS lookups by default in the SSH configuration already.

10

Hi all,

No problem on my side, the ssh reverse dns lookup is common :slight_smile: . And yes, it’s not disabling the DNS but well the reverse DNS lookup. If you rely on the ISP DNS it makes sense not using it. However I’m sorry you lost access to your device with this action, I tried on 2 other boxes I own right now and no issue, this variable is well documented in open-ssh documentation

Cheers

11

OK. Got ssh access back (removed “UseDNS No” from the config file) and it’s back to 10 secs or so before I get a request for a password). Maybe I did something wrong when I added the command to the config file. I’ll check it out again later.

Back to the pressing issue…again, everything was working fine until what would seem to be after the last update. I lost access to any channel guide info (is this related to the ssh issue?). One thing I noticed in doing a reboot just a bit ago is that 1 or 2 programs/add-ons came up stating that tvdb is not accessible. This is also new (since the most recent update). I’m using zap2xml to load up my tvguide.xml file with channel programming info. However, iirc, zap2xml uses tvguide.com for this info. So, the loss of access to tvdb shouldn’t be part of the issue I’m experiencing.

Help :slight_smile:

12

Decided to try running (thru ssh) the tv_grab_zap2xml file in /usr/lib/. Had a bunch of (what looked like) program guide info roll across the screen. So, that works, to a point. I then decided to look at the zap2xml.log file in ~/.kodi/userdata/addon_data/script.module.za2xml.

While the log file showed today’s date, the tvguide.xml file remained at Nov. 26. However, I found some things at the end of the log file that I hope someone else might understand better than me.

…
Getting: http://tvschedule.zap2it.com/tvlistings/gridDetailService?rtype=pgmimg&pgmId=SH013863950000
[I] Parsing: SH013863950000
Exception
error<type ‘exceptions.ValueError’>
Traceback (most recent call last):
File “/home/osmc/.kodi/addons/script.module.zap2xml/zap2xml.py”, line 2065, in
main()
File “/home/osmc/.kodi/addons/script.module.zap2xml/zap2xml.py”, line 1975, in main
parseGrid(fn) #data read in and unicode it
File “/home/osmc/.kodi/addons/script.module.zap2xml/zap2xml.py”, line 1082, in parseGrid
p.feed(b)
File “/usr/lib/python2.7/HTMLParser.py”, line 117, in feed
self.goahead(0)
File “/usr/lib/python2.7/HTMLParser.py”, line 161, in goahead
k = self.parse_starttag(i)
File “/usr/lib/python2.7/HTMLParser.py”, line 327, in parse_starttag
self.handle_starttag(tag, attrs)
File “/home/osmc/.kodi/addons/script.module.zap2xml/zap2xml.py”, line 704, in handle_starttag
globals()[‘on_%s’ % tag](self, tag, attrs)
File “/home/osmc/.kodi/addons/script.module.zap2xml/zap2xml.py”, line 338, in on_td
parseJSOND(fn)
File “/home/osmc/.kodi/addons/script.module.zap2xml/zap2xml.py”, line 851, in parseJSOND
t = json.loads(b)
File “/usr/lib/python2.7/json/init.py”, line 338, in loads
return _default_decoder.decode(s)
File “/usr/lib/python2.7/json/decoder.py”, line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File “/usr/lib/python2.7/json/decoder.py”, line 384, in raw_decode
raise ValueError(“No JSON object could be decoded”)
ValueError: No JSON object could be decoded

It would appear some error(s) have occurred (which might explain why tvguide.xml is not getting updated) but it’s unclear if related to what I’m experiencing. I don’t know python at all.

As I’ve said, all this happened after the update. Could it be that the file wants python 2.7 but when I did the recent upgrade, 2.7 was deprecated and 3.5X was installed?

Thanks in advance…

13

You should have both 2.7 and 3.5 installed side by side. An unfortunate requirement brought about by major changes at 3.0. So I wouldn’t worry about seeing updates for Python 3.5X.

I think the last line of the error you posted is the problem. The script can’t find a JSON object to convert to XML. Maybe wherever tvguide is getting it’s information from has changed the format of their response and the add-on hasn’t yet been updated.

It may be worth checking the add-on’s thread at the kodi.tv forum or with the add-on developer.

14

Hi sam_nazarko

If relevant my installation was done with 08/2016 image and I added it myself, but this was maybe changed at a different moment. Except this change the sshd config was not changed since the install.

PS: thanks for all your work

BR’s

15

Not related with the ssh. The last answer from yknivag3h (json object) seems the right direction

16

@shamael @yknivag

OK. I’ve posted on the kodi.tv forum hoping I can get this straightened out.

Yesterday I did a hard reset (pulled power) and ran tv_grab_zap2xml (thru ssh) and saw the flurry of data roll across the screen. Still, no change in modification date for my tvguide.xml file (and still no info in the TV Shows/Guide screen). However, a lot of the previous errors are gone and now only 1 is left:

Traceback (most recent call last):
File “/home/osmc/.kodi/addons/script.module.zap2xml/zap2xml.py”, line 92, in
import mechanize
ImportError: No module named mechanize

1 other odd thing I noticed is that the latest date for a show in my tvguide.xml file is “20161209” (which may make sense since it doesn’t appear to be getting updated). However, that also seems to be the case when I run tv_grab_zap2xml thru ssh.

Maybe this is the telltale clue :frowning: I guess this really does seem like a zap2xml issue that’s popped up.

Any suggestions for another TV Show scraper that works and seems to be well-maintained?

BTW - For some reason ssh is very responsive this morning. So, you’re right, it looks like ssh was a separate issue, but no idea why it’s changed…

Thanks and cheers…

17

This is a very dangerous thing to do and is likely to do permanent damage to your system by corrupting the sd card. If you need to restart from the command line the correct way is to run sudo reboot.

All the logs point to a problem with the add-on, I suggest you raise it with the add-on developer.

18

I do understand the risks. Thanks. But when I can’t get access to the system (either directly or by ssh), I don’t have much choice. I try to make sure no disk accesses are going at the time, but then I am only so quick at the plug pulling exercise :frowning:

My plan is to install a s/w reset button/routine to have a more organized shutdown. Just haven’t gotten to it yet.