Tether only WiFi traffic over VPN

ChromeQ · 22 November 2019 04:17

Dear all,
I have a Raspberry Pi3 connected via ethernet.
I have connman working to create a wifi hotspot tethered to the ethernet. I have added the route as described in Tethering and OpenVPN - #2 by matosch

iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE

It all works fine to connect my phone to the wifi created.
Now I added OpenVPN, and simply enough it works and ipinfo.io shows me in another country from both the Pi directly and the devices connected over wifi.

However, I would like to only route the wifi traffic over the VPN and leave the rest of ethernet traffic to go through my regular gateway so that I can SSH into my Pi from outside the network.

My attempts have included adding route-nopull and route 192.168.1.0 255.255.255.0 to my OpenVPN config as described here windows - OpenVPN: Only route a specific IP addresses through VPN? - Super User but it stops all traffic working on the devices over wifi.

I have messed around with the iptables a bunch but I think I am running out of trials to error.

How can I route all wifi traffic (192.168.1.*) through the established tun0 without affecting ethernet traffic?
I could live with all traffic going through the VPN if I can set a route to be able to access the Pi from outside the network, maybe this is easier?

Here are some outputs of relevant files in the working state: Please ask for more if it helps:

root@RaspberryPi:/# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth0
10.8.2.0        0.0.0.0         255.255.255.0   U     0      0        0 tun0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
192.168.0.1     0.0.0.0         255.255.255.255 UH    0      0        0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 tether

The route nat table:

root@RaspberryPi:/# iptables -L -t nat -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
connman-POSTROUTING  all  --  0.0.0.0/0            0.0.0.0/0
MASQUERADE  all  --  0.0.0.0/0            0.0.0.0/0

Chain connman-POSTROUTING (1 references)
target     prot opt source               destination         
MASQUERADE  all  --  192.168.1.0/24       0.0.0.0/0

Hope somebody can help, it’s driving me crazy. Thanks

sam_nazarko · 22 November 2019 14:23

You’d probably be best doing this on a per-adapter basis, i.e. iptables rules for eth0 and wlan0.

dillthedog · 22 November 2019 18:26

Unfortunately, I’m unable to test it right now but you might find that OSMC on PI2 and OPENVPN client : unable to connect to the PI from outside the LAN (ssh, transmission, ...) - #19 by dillthedog helps. As mentioned, it’s trying to ensure that traffic that comes in through eth0 also returns through eth0, which seems to fit your situation. You’ll need to tweak the commands a bit since your eth0 network is 192.168.0.* and the Pi’s IP address isn’t going to be 192.168.1.17, but I’m sure you’ll figure it out.
The routing table, as shown, isn’t going to send anything through the VPN. I’d suggest that you reboot the device and start again.

ChromeQ · 23 November 2019 05:38

Thanks @sam_nazarko, that is indeed the route I am heading down, I created a new ip table for the interface and that works fine now to connect to the internet.
My devices use the VPN IP address but not the DNS so they are leaking.
So I am using update-resolv-conf script to let openvpn update the resolv.conf file, and disabled the connman.prefs dnsproxy=no, which should use the resolve.conf right?

But it doesn’t seem to, I can see the correct DNS servers pushed by the VPN server written to /etc/resolve.conf, but no internet connection when dnsproxy=no

I have even moved dns to the front of /etc/nsswitch.conf as it was after the return statement, still no luck.

Should setting dnsproxy=no simply force the traffic to use the nameservers in /etc/resolv.conf?

sam_nazarko · 23 November 2019 10:44

If you disable DNS proxying ConnMan doesn’t pass any DNS via DHCP when tethering

ChromeQ · 23 November 2019 11:38

Oh right, I didn’t realise that. What solutions are there to solve that problem?
I’m so close to my desired setup that I’ve been messing around with IP routes and tables thinking in missing something, and I can’t believe I’m the first with this problem. The connman docs are quite scarsce for no DNS proxy

ChromeQ · 25 November 2019 03:27

Hi again,
I am a lot closer to my desired result. I have decided to keep things simple and put the VPN on the whole raspberry pi, so ethernet traffic is now directed through the tunnel.
I will figure out the way to access the Pi from outside the network separately with some help from your link @dillthedog .

The last remaining issue:
I have connected to VPN using openvpn and tethering is working fine. The IP address of wifi devices is showing the correct IP.
However the DNS is leaking, so I am receiving the DNS from the server and using a modified version of update-resolv-conf I am sticking the nameservers into connmanctl:
connmanctl config ethernet_b827eb26fa82_cable --nameservers "$NMSRVRS"
This conveniently restarts the connman service and the DNS on the device.
The DNS is showing as changed to the values as provided by the VPN server!

Success? Not quite, the problem is the DNS servers on https://ipleak.net/ show the exact values provided. When I connect to vpn on my other machine the DNS on the website is not the provided ones, it is changed to the same country as the IP is.

I would expect this to be a routing or connman issue, but if I stick in the opendns servers into that command above then the DNS addresses change on both the Pi and the devices connected to the tether.

Any clues why? P.S. I had to add an ip route to make the wifi devices work whilst connected to the vpn, without it there is no internet.
iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE

# iptables -L -nv -t nat
Chain PREROUTING (policy ACCEPT 3 packets, 180 bytes)
 pkts bytes target     prot opt in     out     source               destination    

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 1 packets, 60 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 1 packets, 60 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  757  132K connman-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    3   180 MASQUERADE  all  --  *      tun0    0.0.0.0/0            0.0.0.0/0           

Chain connman-POSTROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   76 29622 MASQUERADE  all  --  *      eth0    192.168.1.0/24       0.0.0.0/0

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.8.1.1        128.0.0.0       UG    0      0        0 tun0
0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth0
10.8.1.0        0.0.0.0         255.255.255.0   U     0      0        0 tun0
37.120.133.74   192.168.0.1     255.255.255.255 UGH   0      0        0 eth0
128.0.0.0       10.8.1.1        128.0.0.0       UG    0      0        0 tun0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
192.168.0.1     0.0.0.0         255.255.255.255 UH    0      0        0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 tether

Really hope you can help and see if I am missing something simple such as a ip route, thanks

dillthedog · 25 November 2019 21:06

I find your post difficult to understand. Phrases such as:

However the DNS is leaking, so I am receiving the DNS from the server
…
The DNS is showing as changed to the values as provided by the VPN server!
…
the problem is the DNS servers on https://ipleak.net/ show the exact values provided.
…
When I connect to vpn on my other machine the DNS on the website is not the provided ones, it is changed to the same country as the IP is.

I find to be too vague and I’m finding it hard to understand what you have done and what problems you are seeing.

If you say something is or isn’t right, please specify why it is or isn’t right. and provide specific details for each case, with actual IP addresses.

ChromeQ · 25 November 2019 22:15

Sorry if it was hard to understand. I’ll show you what I mean.
When not connected to the VPN the relevant files look like this:

me@RaspberryPi:~$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
192.168.0.1     0.0.0.0         255.255.255.255 UH    0      0        0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 tether

root@RaspberryPi:/home/dav# iptables -nv -L -t nat
Chain PREROUTING (policy ACCEPT 48 packets, 6529 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain INPUT (policy ACCEPT 12 packets, 1619 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 43 packets, 5093 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 43 packets, 5093 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   70  7259 connman-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain connman-POSTROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   34  4814 MASQUERADE  all  --  *      eth0    192.168.1.0/24       0.0.0.0/0

root@RaspberryPi:/home/dav# cat /etc/resolv.conf 
# Generated by Connection Manager
nameserver ::1
nameserver 127.0.0.1

root@RaspberryPi:/home/dav# cat /var/lib/connman/ethernet_b827eb26fa82_cable/settings
[ethernet_b827eb26fa82_cable]
Name=Wired
AutoConnect=true
Modified=2019-11-25T21:35:04.301767Z
IPv4.method=manual
IPv4.netmask_prefixlen=24
IPv4.local_address=192.168.0.169
IPv4.gateway=192.168.0.1
IPv6.method=off
IPv6.privacy=disabled
IPv6.DHCP.DUID=00010001256dc19fb827eb26fa82
Nameservers=8.8.8.8;8.8.4.4;

Everything works as expected, my tethered wireless devices can access the internet and as you can see, the DNS is resolved to 23 servers between the US and Australia as my DNS is set to googles 8.8.8.8.

Now, when I connect the VPN:

me@RaspberryPi:~$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.8.2.1        128.0.0.0       UG    0      0        0 tun0
0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth0
10.8.2.0        0.0.0.0         255.255.255.0   U     0      0        0 tun0
37.120.133.74   192.168.0.1     255.255.255.255 UGH   0      0        0 eth0
128.0.0.0       10.8.2.1        128.0.0.0       UG    0      0        0 tun0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
192.168.0.1     0.0.0.0         255.255.255.255 UH    0      0        0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 tether

I have to add the following route to access the internet on devices connected to the tether:
iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE
Which results in this iptables:

root@RaspberryPi:/home/dav# iptables -nv -L -t nat
Chain PREROUTING (policy ACCEPT 76 packets, 7510 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain INPUT (policy ACCEPT 26 packets, 3006 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 547 packets, 34737 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 530 packets, 33461 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  595 39145 connman-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
   64  5570 MASQUERADE  all  --  *      tun0    0.0.0.0/0            0.0.0.0/0           

Chain connman-POSTROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    1   114 MASQUERADE  all  --  *      eth0    192.168.1.0/24       0.0.0.0/0

The connman settings and the resolv.conf have not changed and this is the screen shot I get:

Access to the internet is fine and you can see my IP address is in the UK, but the DNS servers are still coming from google, not my VPN providers.

So, in the openvpn config settings I’ve added:

script-security 2
up /etc/openvpn/update-resolv-conf
down /etc/openvpn/update-resolv-conf

Which I’ve added this line to the vpn up connmanctl config ethernet_b827eb26fa82_cable --nameservers "$NMSRVRS" and this one to the down connmanctl config ethernet_b827eb26fa82_cable --nameservers 8.8.8.8 8.8.4.4
This updates the connman settings to look like this when vpn is up (and restores it when taken down):

root@RaspberryPi:/etc/openvpn# cat /var/lib/connman/ethernet_b827eb26fa82_cable/settings
[ethernet_b827eb26fa82_cable]
Name=Wired
AutoConnect=true
Modified=2019-11-25T21:50:19.679825Z
IPv4.method=manual
IPv4.netmask_prefixlen=24
IPv4.local_address=192.168.0.169
IPv4.gateway=192.168.0.1
IPv6.method=off
IPv6.privacy=disabled
IPv6.DHCP.DUID=00010001256dc19fb827eb26fa82
Nameservers=103.86.96.100;103.86.99.100;

These values (103.86.96.100;103.86.99.100) are the DNS servers provided to me during the VPN connection.
It results in:

Now you can see the DNS servers are exactly as provided, they are not being resolved properly as 8.8.8.8 was. And when I hard code opendns servers they are resolved like the google ones are.
When I connect to the VPN directly on my phone so taking the Pi out of the picture I result in this:

This is the resolved DNS which I expect to see when the connman nameservers are set to the VPN provided servers.
Interestingly the resolved DNS address is in my route table:
37.120.133.74 192.168.0.1 255.255.255.255 UGH 0 0 0 eth0
And if I put this address directly into the connman nameserver settings then internet connection is lost.

I am completely at a loss on how to fix this and I really need help to figure this one out. Sorry for the extra long post but want to be super clear this time with all my findings. Thanks for your help so far.

dillthedog · 25 November 2019 22:42

Thanks. That’s a lot clearer. I’ll take a look at this tomorrow and see if I can come up with an answer.

Two possibilities, just off the top of my head: (1) you change the DNS settings on eth0 with the connmanctl command but you don’t seem to have restarted tethering. Maybe it would then pick up the new DNS settings. (2) Perhaps tethering will take it’s DNS configuration from the wlan0 configuration, rather than from eth0.

ChromeQ · 25 November 2019 22:56

Thanks, I’ll await your answer.

I wasn’t aware there is a tethering restart command, but the dns changes are visible to the device immediately after running the connmanctl command and I’ve tried rebooting too which would restart the tethering.
I don’t have any wlan0 config in my /var/lib/connman dir, and I’ve not seen any nameserver options elsewhere in any config files
I think it is a routing issue most likely but hope you can figure it out. Thanks again

dillthedog · 26 November 2019 21:53

I’ve had little time this evening, but I’ve managed to set up tethering on a Pi3B and, using the connmanctl command, change the nameservers used on eth0. On every occasion, the tethered device has picked up the correct (new) DNS servers. NB this is testing the basic functionality without a VPN running. So, AFAICT, this works as advertised.

That is an incorrect route. The line should contain the address of the VPN server, in this case 37.120.133.75, not that of the DNS server. All your DNS traffic will be going out via eth0, not tun0. My best guess is that either the VPN server is pushing a bad command to you (unlikely) or you need to double-check your modified update-resolv-conf script. If you share your script and system journal (grab-logs -J), something might also show up.

ChromeQ · 26 November 2019 22:48

I have the same experience when not connected to the VPN, updating the dns nameservers with the connmanctl command does reflect immediately on the device. It also works whilst on the VPN and using public dns addresses such as googles and opendns.
The issue is really when the dns provided by the vpn is pushed, they do not get resolved to the correct ip address and I don’t know why.

I’m not sure that is correct, I am connecting to the exact same vpn server on my laptop and the route is applied the same as the pi, and the result in ipleak.net is correct there.

Is there a command that I can run which will show my dns ip? Something like tcpdump or netstat or trace (I’m not familiar with them or their usage) so that I can see what the dns is over ethernet to see if this is only an issue on tether interface. I don’t think it is since the other dns addresses work fine.
Unless there is a browser I can access on osmc but I’ve not had luck with that yet either.

I’ll supply my logs and script here:

/etc/openvpn/update-resolv-conf

root@RaspberryPi:/etc/openvpn# cat update-resolv-conf 
#!/bin/bash
# 
# Parses DHCP options from openvpn to update resolv.conf
# To use set as 'up' and 'down' script in your openvpn *.conf:
# up /etc/openvpn/update-resolv-conf
# down /etc/openvpn/update-resolv-conf
#
# Used snippets of resolvconf script by Thomas Hood and Chris Hanson.
# Licensed under the GNU GPL.  See /usr/share/common-licenses/GPL. 
# 
# Example envs set from openvpn:
#
#     foreign_option_1='dhcp-option DNS 193.43.27.132'
#     foreign_option_2='dhcp-option DNS 193.43.27.133'
#     foreign_option_3='dhcp-option DOMAIN be.bnc.ch'
#

[ -x /sbin/resolvconf ] || exit 0
[ "$script_type" ] || exit 0
[ "$dev" ] || exit 0

split_into_parts()
{
	part1="$1"
	part2="$2"
	part3="$3"
}

case "$script_type" in
  up)
	NMSRVRS=""
	SRCHS=""
	for optionvarname in ${!foreign_option_*} ; do
		option="${!optionvarname}"
		split_into_parts $option
		if [ "$part1" = "dhcp-option" ] ; then
			if [ "$part2" = "DNS" ] ; then
				NMSRVRS="${NMSRVRS:+$NMSRVRS }$part3"
			elif [ "$part2" = "DOMAIN" ] ; then
				SRCHS="${SRCHS:+$SRCHS }$part3"
			fi
		fi
	done
	R=""
	[ "$SRCHS" ] && R="search $SRCHS
"
	for NS in $NMSRVRS ; do
        	R="${R}nameserver $NS
"
	done

#	echo -n "$R" | /sbin/resolvconf -a "${dev}.openvpn"

	connmanctl config ethernet_b827eb26fa82_cable --nameservers "$NMSRVRS"
	;;
  down)
#	/sbin/resolvconf -d "${dev}.openvpn"

	connmanctl config ethernet_b827eb26fa82_cable --nameservers 8.8.8.8 8.8.4.4
	;;
esac

With this I have commented out the /sbin/resolvconf call and tried with and without it as openresolv does update the /etc/resolv.conf file correctly and I tested if this was causing an issue when it wasn’t pointing back to localhost. It doesn’t work either way.

And here is a grab logs from a fresh reboot I just did:
https://paste.osmc.tv/enulimelaq

Hope they can shed some light.

ChromeQ · 26 November 2019 23:35

I found this GitHub - macvk/dnsleaktest: An open source script tests VPN connection for DNS Leak. which seems to work and my result shows that the DNS is leaking over eth0 and not just the tethered devices

root@RaspberryPi:/etc/openvpn# ./dnsleaktest.sh
Your IP:
37.120.133.75 [Romania AS3210 Secure Data Systems SRL]

You use 2 DNS servers:
103.86.96.100 [Australia AS36351 SoftLayer Technologies Inc.]
103.86.99.100 [Singapore AS36351 SoftLayer Technologies Inc.]

Conclusion:
DNS may be leaking.

dillthedog · 27 November 2019 07:46

I’ve only had time to look at the log but it shows that your VPN server is 37.120.133.74.

Nov 26 22:38:56 RaspberryPi ovpn-uk975.nordvpn.com.udp[335]: UDP link remote: [AF_INET]37.120.133.74:1194

and here are your DNS servers:

Nov 26 22:39:13 RaspberryPi ovpn-uk975.nordvpn.com.udp[335]: PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1,dhcp-option **DNS 103.86.96.100,dhcp-option DNS 103.86.99.100**,sndbuf 524288,rcvbuf 524288,explicit-exit-notify,comp-lzo no,route-gateway 10.8.3.1,topology subnet,ping 60,ping-restart 180,ifconfig 10.8.3.20 255.255.255.0,peer-id 18,cipher AES-256-GCM'

ChromeQ · 27 November 2019 08:11

Yes, the DNS servers are in the push reply, and they get added to the connman settings with my update-resolv-conf script.
I don’t do anything weird or special with the IP addresses and just let openvpn do it’s thing.

37.120.133.74 should be the DNS address. I can confirm that when I connect to the same Nord server on my phone or laptop and check the ipleak.net site and it has the same IP, but not the same DNS. Connman is the only difference between my laptop/phone and my pi.

The IP address is always the same, DNS is not.

It also does “resolve” the DNS address when I manually connmanctl the nameservers to any public DNS address such as Googles 8.8.8.8 or opendns or cloudflares 1.1.1.1 so this seems to be only an issue when connman gets VPN DNS addresses

dillthedog · 27 November 2019 15:40

No it isn’t. It’s the VPN server’s address. I checked here.

The DNS server addresses are shown in the PUSH command: 103.86.96.100 and 103.86.99.100.

Therefore this:

root@RaspberryPi:/etc/openvpn# ./dnsleaktest.sh
Your IP:
37.120.133.75 [Romania AS3210 Secure Data Systems SRL]

You use 2 DNS servers:
103.86.96.100 [Australia AS36351 SoftLayer Technologies Inc.]
103.86.99.100 [Singapore AS36351 SoftLayer Technologies Inc.]

seems to be correct. The IP address 37.120.133.75 will be the outbound side of the VPN server.

My conclusion is that it all appears to be working correctly on the Pi side. Can you please recheck on the tethering side.

ChromeQ · 27 November 2019 22:23

Yes that seems to be the VPN server’s address, so why does that show as the DNS address when connected to the VPN on my laptop and directly via the nord app on my phone?
I think that is the correct behaviour as they work fine for geo-restricted content but a device on the tether wifi does not.

I’m still confused why connman resolves the DNS addresses when they are entered as googles or opendns, but they display as 103.86.96.100 & 103.86.99.100 when they are the nameservers provided.

It seems there is a problem with routing or DNS resolving but I’m not sure what.

dillthedog · 28 November 2019 07:54

Surely that’s a question for the NordVPN forum. What servers does GRC | DNS Nameserver Spoofability Test show?

If you mean that the Google and OpenDNS IP addresses show as a DNS name, that’s probably because those two IP addresses, 103.86.96.100 and 103.86.99.100, don’t have a DNS name assigned.

Once again, please clearly define what you mean by “does not work”.

ChromeQ · 28 November 2019 11:48

I really don’t think this is an issue with Nord VPN. It does a load of route adding and it is normal, it is doing the exact same on my laptop and that is not displaying the provided DNS addresses in any leak test.

No I don’t mean they show a name, I mean when I put 8.8.8.8 into connmanctl nameservers then the ipleak tests do not show 8.8.8.8 but resolves to DNS address in the us and Australia.
On my laptop connected to the same Nord server results in the same pushed DNS addresses and they resolve to 37.120.133.74 on all IP leak test websites. My laptop does not use connman which is why I think there is a problem there.

When on the laptop or phone connected to the same Nord server then no such issues playing content.
The only difference I can tell is the DNS address is leaking.

Does connman-vpn service work for osmc yet or is it still in progress? Maybe I can use that and avoid a separate openvpn instance.
My only other solution after that is to ditch tethering with connman and use hostapd and dnsmasq if possible.