osmc@OSMCBR:~$ time wget --server-response --timeout=4 --max-redirect 0 --spider www.google.com
Spider mode enabled. Check if remote file exists.
--2015-11-06 11:25:51-- http://www.google.com/
Resolving www.google.com (www.google.com)... 110.164.16.88, 110.164.16.108, 110.164.16.104, ...
Connecting to www.google.com (www.google.com)|110.164.16.88|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 302 Found
Location: http://www.google.co.th/?gws_rd=cr&ei=0Cs8VsD3LYHn0AT7j4DgBA
Cache-Control: private
Content-Type: text/html; charset=UTF-8
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
Date: Fri, 06 Nov 2015 04:25:52 GMT
Server: gws
Content-Length: 261
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Set-Cookie: PREF=ID=1111111111111111:FF=0:TM=1446783952:LM=1446783952:V=1:S=ZPM2lB10lPRqGwH0; expires=Thu, 31-Dec-2015 16:02:17 GMT; path=/; domain=.google.com
Set-Cookie: NID=73=EwxNi6NzgEvoiBnRqLzRpiMT2cjtbM11dX9z7XzWXmTJFDN_-GFIUmzM7iGV961slQ64wmso_ZPGswMdgRuvDxQNx67-mTNaaf3nr7HCf52ispGgvcC1UbRcSLzVZpsWj6FK7xvt30Q4n2o6OxUAHuR1Xmaw9yE; expires=Sat, 07-May-2016 04:25:52 GMT; path=/; domain=.google.com; HttpOnly
Location: http://www.google.co.th/?gws_rd=cr&ei=0Cs8VsD3LYHn0AT7j4DgBA [following]
0 redirections exceeded.
real 0m1.181s
user 0m0.060s
sys 0m0.010s
==
osmc@OSMCBR:~$ time sudo /usr/bin/http-time
Updated time from Fri Nov 6 04:27:08 UTC 2015 to Fri Nov 6 04:27:08 UTC 2015 using HTTP query to 139.162.18.76
real 0m5.673s
user 0m0.110s
sys 0m0.130s
I wonder if this has anything to do with redirection? Note how the wget is redirected to www.google.co.th. Google’s DNS server may handle that more suavely than my ISP’s.
Ok, seems it is working now. Would suggest you monitor your original time/ntp issue but I assume if you change the DNS settings on both machines to this setup you should be fine.
The redirection should not be a problem. As long as any webpage is reached. As the script just will grep for the the “Date” and take that as reference.
Just saw your next message, so now this comment can be ignored
Just as a side note (might not impact you) by using the Google DNS Server directly instead of using your local router DNS first you would loose the capabilities of local DNS names that you configure in your router.
So it appears you have very slow DNS resolution from your ISP’s DNS server. Anything slower than about 1 second is considered to be slow - the 4 second timeout in http-time was timing out, so that is very slow indeed.
Most likely your ISP’s DNS server is broken or badly configured - because it should be caching the DNS entries for Apple, Google and Microsoft.
One reason we query these servers is that the DNS servers of every ISP on the planet are going to have cached entries for these domains, so rather than having to go fetch the address by contacting the root servers, then querying Apple/Google/Microsoft’s DNS servers, they can retrieve the already cached entry they have saved from the thousands of other customers that just looked up the same domains before you did…
The response time for looking up one of these domains should only be the latency of your internet connection to your ISP’s DNS server - only the order of a few hundred milliseconds. If it were me I would complain to your ISP, although you may be better off to just keeping using Google DNS.
One problem that using Google DNS (or any other DNS servers in a different country) may cause you though, is slower throughput when downloading from a CDN network as the CDN may incorrectly identify your location and send you to a slower more distant server.
Yes, the Thai ISPs seem to be incompetent in many ways. Twice now they have introduced routing errors making it impossible for me to reach my own web site. Both times they needed help from my web hosting company in order to resolve the issue.
I understand about using a non-local DNS server, but I’m not sure what choice I have. Do you know of any reliable and trustworthy public DNS servers that are physically located in SE Asia?
Finally, and I know you don’t want to hear this, but….
This Pi has been remarkably stable in the year or so that I’ve had it. I’ve had to restart Kodi now and then, but that’s it. During the 48 or so hours that I had ntp logging enabled, it crashed every few hours. In the 48 or so hours since I disabled ntp logging, it hasn’t crashed or frozen even once. Now, I realize that temporal proximity is not proof of causation, but it makes you think, doesn’t it?
I’ve read claims that Google Public DNS can slow down certain multimedia applications or websites. Are these true?
Many sites that provide downloadable or streaming multimedia host their content with DNS-based third-party content distribution networks (CDNs), such as Akamai. When a DNS resolver queries an authoritative nameserver for a CDN’s IP address, the nameserver returns an address which is closest (in network distance) to the resolver, not the user. In some cases, for ISP-based resolvers as well as public resolvers such as Google Public DNS, the resolver may not be in close proximity to the users. In such cases, the browsing experience could be slowed down somewhat. Google Public DNS is no different from other DNS providers in this respect.
To help reduce the distance between DNS servers and users, Google Public DNS has deployed its servers all over the world. In particular, users in Europe should be directed to CDN content servers in Europe, users in Asia should be directed to CDN servers in Asia, and users in the eastern, central and western U.S. should be directed to CDN servers in those respective regions. We have also published this information to help CDNs provide good DNS results for multimedia users.
In addition, Google Public DNS engineers have proposed a technical solution called EDNS Client Subnet. This proposal allows resolvers to pass in part of the client’s IP address (the first 24/64 bits or less for IPv4/IPv6 respectively) as the source IP in the DNS message, so that nameservers can return optimized results based on the user’s location rather than that of the resolver. To date, we have deployed an implementation of the proposal for many large CDNs (including Akamai) and Google properties. The majority of geo-sensitive domain names are already covered.
Glad to see the case is closed – as we suspected, regional ISPs can cause a lot of problems, and it just goes to show how many variables we have to deal with when diagnosing an issue. I suspected things may be afflicted locally as we resolved the few time issues we had several months ago with an early time service resolved via HTTP.
osmc@OSMCBR:~$ sudo /usr/bin/http-time
Updated time from Sun Nov 8 21:15:49 UTC 2015 to Sun Nov 8 22:27:21 UTC 2015 using HTTP query to 139.162.18.76
osmc@OSMCBR:~$ time wget --server-response --timeout=4 --max-redirect 0 --spider www.google.com
Spider mode enabled. Check if remote file exists.
--2015-11-09 05:47:09-- http://www.google.com/
Resolving www.google.com (www.google.com)... 110.164.6.217, 110.164.6.231, 110.164.6.216, ...
Connecting to www.google.com (www.google.com)|110.164.6.217|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 302 Found
Location: http://www.google.co.th/?gws_rd=cr&ei=79A_VuexC4KxuAT76KGYDA
Cache-Control: private
Content-Type: text/html; charset=UTF-8
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
Date: Sun, 08 Nov 2015 22:47:11 GMT
Server: gws
Content-Length: 261
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Set-Cookie: PREF=ID=1111111111111111:FF=0:TM=1447022831:LM=1447022831:V=1:S=Hg9r4QKpsyURIpNK; expires=Thu, 31-Dec-2015 16:02:17 GMT; path=/; domain=.google.com
Set-Cookie: NID=73=ciF4lXqpINC81yB_v54ZNaZTMPCD2ulZjaTy2W3JTZwdfM3zRmuQEonJuQ4VQrHLUxeBafol_TDXHLiIMQb0FMuQY3nuhx2EdW1eO1iPpdk6rEG0-Bt2H2kBsgDkVqAv1B7TGJg-EnkPXhdoFqpQOnRwkcf4vg; expires=Mon, 09-May-2016 22:47:11 GMT; path=/; domain=.google.com; HttpOnly
Location: http://www.google.co.th/?gws_rd=cr&ei=79A_VuexC4KxuAT76KGYDA [following]
0 redirections exceeded.
real 0m0.965s
user 0m0.000s
sys 0m0.000s
osmc@OSMCBR:~$ time sudo /usr/bin/http-time
Updated time from Sun Nov 8 22:47:38 UTC 2015 to Sun Nov 8 22:47:39 UTC 2015 using HTTP query to 139.162.18.76
real 0m1.387s
user 0m0.000s
sys 0m0.000s
Since you’re unable to enable ntp logging without your system freezing I would suggest that your next step is to do a completely fresh install and see if the problems persist.
We are now at the point where you must either have corruption (file system corruption, a failed update etc) or a hardware stability issue. A fresh install would prove or rule out a software corruption issue.
ntpd crashing and the whole system freezing just from enabling ntp debug logs is simply not normal, and nobody else has reported these kind of problems before, so I think it’s time for you to do a fresh install. (Or just put up with it if you’re not willing to)
I’ve enabled ntp logging again. No crash so far, after a couple of hours. I did make two changes.
In ntp.conf I replaced the Debian pool time servers with the Asia pool servers.
I modified the http-time script as follows:
if connmanctl state | grep -iq 'online'
then break
elif connmanctl state | grep -iq 'ready'
then break
fi
This seems more in line with the documentation:
A network is in state ‘online’ if ConnMan has verified connectivity to Internet, i.e. it has managed to look up and connect to ipv4.connman.net or ipv6.connman.net. For all practical purposes ‘ready’ and ‘online’ are usually equivalent for the intended connectivity experience.
and prevents timeouts when the device is actually connected but, for whatever reason, ConnMan has been unable to either lookup or connect to ipv4.connman.net.
I won’t post to this thread again unless I find something significant.
http-time checks for ‘online’ not ‘ready’ on purpose.
Ready means that the local network is up (the interface has an IP address, and the DHCP server responded in the case of DHCP) but that internet connectivity is not available, as determined by not being able to connect to ipv4.online.osmc.io.
So by changing the test to ready instead of online, you have basically disabled the test because now it will attempt to connect even if it thinks it is not online…
osmc@OSMCBR:~$ connmanctl state
State = ready
OfflineMode = False
SessionMode = False
osmc@OSMCBR:~$ sudo /usr/bin/http-time
Updated time from Tue Nov 17 00:05:19 UTC 2015 to Tue Nov 17 00:05:20 UTC 2015 using HTTP query to www.google.com
osmc@OSMCBR:~$ connmanctl state
State = ready
OfflineMode = False
SessionMode = False
osmc@OSMCBR:~$ date
Tue Nov 17 07:05:35 ICT 2015
The problem I’m seeing is that, at least from here, “conmanctl state” does not reliably return “online” even when there is an internet connection. In fact, with both my Vero and Pi “conmanctl state” almost always returns “ready”. The result is that whenever either machine boots I see the timeout message: “No internet connection was available within 60 seconds, giving up.” Since editing http-time, both machines reliably get a “temporary” date and time at boot.
The http-time script fails gracefully if unable to set the time using an HTTP query. Why not just let it fail if the connection state is either “ready” or “online”?
It seems to work for me.
I only mentioned ipv4.connman.net because I read somewhere that this is the site conmanctl attempts to connect with in order to decide what state to return. Sorry if what I read was wrong: Intel Open Source - Conman
No need to scold me over and over again. I’m only trying to learn here.
