Xplornet double NAT: VPN edition

Previously, I wrote about using a reverse SSH tunnel to escape a double NAT (specifically, the one provided by Xplornet). Without looking into why (maybe poor, intermittent connection and particularly awful uplink), my previous solution was not stable. Even with autossh, the connection kept dropping and not picking back up. I’m exclusively accessing this remotely, so when I notice the service being down I’m in pretty much the worst position to fix it.

Grab a public server with a static IP – for example a 5$/month Linode or Droplet. I’ve seen reference to cheaper international products, but I have no experience with them.

Linode1024

If you’ve picked Linode:

  • deploy an Ubuntu image
  • boot it up
  • SSH to the machine
  • do regular server stuff – make sure it’s up to date, generally read over Digital Ocean’s guide here for inspiration

Set up OpenVPN server

On the public computer (OpenVPN server/cloud instance):

The first time I did this, I set up OpenVPN myself. It’s not awful, there are some pretty comprehensive guides (like this one), but it definitely sucks enough to look for an alternative. Googling around shows two compelling public scripts – Nyr’s openvpn-install and Angristan’s version based off Nyr’s. Looking over the two, I ended up picking Angristan’s version without all that much consideration.

SSH to the machine and execute the script on your pubic server to set up the certificates and keys for your client. The defaults for the script all seem sensible – you don’t have to feel bad if you just mash enter until the name prompt comes up, then give your client a reasonable name

$ wget https://raw.githubusercontent.com/Angristan/OpenVPN-install/master/openvpn-install.sh
$ chmod +x openvpn-install.sh
$ ./openvpn-install.sh

You should notice at the end of the script execution a line that looks something like this:

...
Finished!

Your client config is available at /root/unifi-video-server.ovpn
If you want to add more clients, you simply need to run this script another time!

Take note of the location of your .ovpn file, as you’ll need it for the next step.

Set up OpenVPN client

On the private computer (machine that’s behind the double NAT):

On your client machine, get the OVPN configuration file that was generated from the previous step. scp is likely the easiest way to do this. From the client machine, you can retrieve the file like:

scp {server user}@{server host}:{remote path to ovpn} {local path}

For example:

$ scp root@37.48.80.202:/root/unifi-video-server.ovpn .

This will copy the file to the current directory on the machine. An extremely quick sanity check to ensure you can connect:

sudo openvpn unifi-video-server.ovpn

You should see:

Initialization Sequence Completed

once you do, you can ctrl + c your way out. If this wasn’t successful… something has gone wrong and you should fix it.

To make sure your client connects on start up:

  • rename your .ovpn file to be a .conf file
  • move the .conf file to /etc/ovpn
  • Edit /etc/default/openvpn to ensure AUTOSTART is configured to start your connection

At this stage, you have an OpenVPN server set up and an OpenVPN client that automatically connects to the server. All that’s left is to do the internet part.

Set up server traffic forwarding to client

On the public computer (OpenVPN server/cloud instance):

What we want now is to forward traffic that hits a particular port on the public server to the private computer. Not only that, but you want the private computer to think the traffic is coming from the public server, so it doesn’t respond directly to whoever sent the internet request.

First things first, toggle the server to allow forwarding traffic (if you don’t do this, you’ll end up insanely frustrated and convinced iptables is the devil:

sysctl -w net.ipv4.ip_forward=1

We need two pieces of information:

  • the public WAN (internet) IP address of the server
  • the virtual address of the OpenVPN client

Finding the public address can be done with:

$ curl ipinfo.io/ip
37.48.80.202

The virtual address of the OpenVPN client can be found in the OpenVPN status log with the client connected (see above for how to set up the connection for now). The log seems like it’s either in either /etc/openvpn/openvpn-status.log or /etc/openvpn/openvpn.log

$ cat /etc/openvpn/openvpn.log
OpenVPN CLIENT LIST
Updated,Sun Nov 5 01:37:33 2017
Common Name,Real Address,Bytes Received,Bytes Sent,Connected Since
unifi-video-server,37.48.80.202:49014,39837,52165,Sun Nov 5 01:02:05 2017
ROUTING TABLE
Virtual Address,Common Name,Real Address,Last Ref
10.8.0.2,unifi-video-server,37.48.80.202:49014,Sun Nov 5 01:36:54 2017
GLOBAL STATS
Max bcast/mcast queue length,1
END

Now we’ll need a source routing NAT rule and a destination routing NAT rule for every port that is going to be forwarded. They’ll look something like this:

iptables -t nat -A PREROUTING -d {server WAN IP} -p tcp --dport {port} -j DNAT --to-dest {client virtual address}:{port}
iptables -t nat -A POSTROUTING -d {client virtual address} -p tcp --dport {port} -j SNAT --to-source {server virtual address}

Practically speaking, with the following:

  • public server whose Internet accessible IP address is 37.48.80.202
  • public server whose OpenVPN virtual address is 10.8.0.1
  • private computer whose OpenVPN virtual address is 10.8.0.2
  • Forwarding port 7080 on the public server to port 7080 on the private computer

It’d look something like this:

iptables -t nat -A PREROUTING -d 37.48.80.202 -p tcp --dport 7080 -j DNAT --to-dest 10.8.0.2:7080
iptables -t nat -A POSTROUTING -d 10.8.0.2 -p tcp --dport 7080 -j SNAT --to-source 10.8.0.1

Now the only thing left is to make sure the routing rules persist across reboots.

$ sudo apt install iptables-persistent
$ sudo netfilter-persistent save
$ sudo netfilter-persistent reload

And that’s it. In my experience this seems to be both a more robust solution to the double NAT problem, and uses tools in a more conventional way. I visited 37.48.80.202:7080, and (subject to the awful uplink speed from Xplornet), my page loaded!

Advertisements

9 thoughts on “Xplornet double NAT: VPN edition

  1. Alex H

    Hey there,

    This post and the last one made me laugh… not in a haha way, but in a sad way. I’ve been dealing with xplornet’s intolerable double NAT for a long time now trying to find workarounds to it. I do some setups for people that use IP cameras and want to be able to view them remotely. The cameras contact a mediator server to create a NAT tunnel association for the inbound, but they (xplornet) must have their timeout set to something incredibly aggressive as the cameras don’t stay visible from the outside for long. Probably because they have 700 people sharing a single static IP.

    I’ve been putting together a solution that puts a Mikrotik hex at the client end, and using L2TP/IPSec to get to a head end where I have a public subnet. Dispensing addresses and then allowing the client to reach their home over the public address. Everything working as planned in the lab, but I haven’t been out to a client site yet to test it out.

    Any idea if L2TP will play nicely with their setup? I could use OpenVPN, but I am trying to take advantage of the hardware encryption chip in the $80 hex to inexpensively solve the problem. If there was lots of money to throw around, it wouldn’t be a problem at all… Bell LTE hub with Static IP, Cisco DMVPN solution, etc etc etc but they are all more expensive than what I’m attempting.

    If they would just get IPv6 deployed already, at least I could tunnel over that in a reliable way but when you can stretch a /24 to serve 8,000 people there’s no hurry.

    Would love to hear your thoughts, feel free to email me with the address below.

    Reply
    1. Genny

      Alex, I work for a company that is marketing monitoring and control solutions to agricultural customers. xplornet is the bane of our existence. We really need a solution to access cameras through a double NAT from Xplornet. I have one customer with a remote site with an internet connection, but no permanent computer on the site. Would like to deploy a solution through the router. If there is any help you or the author of this blog post could provide, it would be very much appreciated.!!! My networking knowledge is rudimentary, but increasing through necessity. A lot of these posts are difficult to understand at the moment.

      Reply
    2. tobymurray Post author

      > the cameras don’t stay visible from the outside for long
      I’ve found that as soon as I started looking at my Internet connection it was even worse than I thought. On a daily basis my upstream will drop to single digit Kbps. Between that and reasonably regular outages, I wouldn’t be surprised if you were running into raw bandwidth issues.

      > Any idea if L2TP will play nicely with their setup?
      I haven’t tried – if it works for you, please write about it somewhere/comment here. I’d be interested in the “best” solution to this problem.

      > I could use OpenVPN, but I am trying to take advantage of the hardware encryption chip in the $80 hex to inexpensively solve the problem
      I’m not clear on what part of the problem you’re solving inexpensively there – is it that there is no existing hardware on the customers’ site that would be able to act as a VPN client? Or you don’t want to have to deal with whatever arbitrary equipment a customer has, aiming to drop a box, plug it in, and go?

      The good thing about ridiculously terrible upstream speeds is that virtually any device would be able to satisfy the performance needs. I don’t know much about the Mikrotik HEX, but nothing good is coming up on google in the area of ease of use or robustness around the problem you’re trying to solve. If there is already a router you can piggy back off, I’m sure you could find something super cheap to act as the client for a VPN? Raspberry Pi isn’t the best value, but still half the price of the hex.

      Reply
  2. Jeff Lamer

    Thanks for the blog :). I’m stuck on the “Set up OpenVPN client” I followed your directions and have a server setup on a DigitalOcean Droplet. I am trying to get outside access to my raspberry pi webserver. I had issues with the scp command so I used winScp to copy the file. Now when I run sudo openvpn XplornetOpenVpn.ovpn I get sudo: openvpn: command not found.
    Do I need to install openvpn on the client ? I know I’m over my head a bit, but live and learn !

    Reply
    1. tobymurray Post author

      > so I used winScp to copy the file

      Can you clarify where you used WinSCP? Did you install Windows on your Raspberry Pi? I’ve never done that – sounds interesting.

      > when I run sudo openvpn XplornetOpenVpn.ovpn I get sudo: openvpn: command not found.

      That comment is intended for a Linux environment. I have no experience with OpenVPN clients on Windows.

      > Do I need to install openvpn on the client ?
      Yes, you’ll need a VPN client for your Raspberry Pi. I imagine if it’s in Windows, you’ll launch your client and then load the configuration file somehow – all it’s trying to do is run the client with the configuration in the file.

      Reply
  3. Jeff Lamer

    Hey Toby,
    Hoping to get some help, Was able to follow all instructions and still can’t access my local page on port 1880.

    Reply
  4. Phil

    This has been extremely helpful to me as well. I too am trying to set up cameras via xplornet, though I am significantly downscaling the video quality on site before trying to send it out. I was able to get the ngrok solution working, but has the downfalls of if there’s a power bump, when things restart you have a different ngrok address, unless you’re paying for a link.

    I’ll have to try this particular VPN solution on another day, as my onsite staff had to fly out and we inevitably had issues with inverter faults.

    Just wanted to say thanks for some good explanation of the root cause, and some innovative solutions. Xplornet Jupiter systems don’t leave you much to work with.

    Reply
    1. tobymurray Post author

      > I am significantly downscaling the video quality on site before trying to send it out
      Ugh, tell me about it. A reliable 1 Mbps up would be a welcome improvement over what I have now.

      Great to hear I’m not alone in struggling with Xplornet. I’ve had this running on and off for a while as projects fade in and out and it seems to consistently be the most robust solution. I tried a few other strategies, and when it comes down to it the power failure/Internet failure made everything else less robust – often for difficult to understand reasons. VPN client re-connection is bog standard and seems to not suffer the same issues if it isn’t immediately successful.

      Plus, 5$/month is basically free, and you could theoretically serve multiple customers via the same public facing machine. If you’re in the unenviable situation of having to solve this problem, this is the best solution I’ve come across. If you come up with anything different or notice anything to improve, please let me know!

      Reply
  5. Clem Morton

    Hello everyone. I’m also struggling with xplornet’s double NAT. There are several things I’ve tried. I’m running an ubiquity edge router lite. With it I have been able to pull in and distribute to my network IPv6 addresses. So I have native IPv6 on my network. However the addresses are on a lease, and they change with regularity. Also, I randomly seem to loose IPv6 connectivity? Its not been stable. I have tried OpenVPN running on the edge router in client mode, yes. It does work, however the speeds are terrible. There is no offloading for it. If anyone has a guide on how to get LT2P and Ipsec working on one of these devices behind a double NAT with offloading. Please do a write up and share. I’ve been struggling on that for months now. I’m not even sure it will work.
    Other options I am looking at are the pfsense firewalls/routers, if one of those can’t do it, then nothing can.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s