Background

Both this site and my home server use HTTPS certificates provided by Let's Encrypt. Previously I was using the http-01 challenge method, but once wildcard certificates became available, I decided to switch to the dns-01 method for simplicity.

My domains are currently registered through Namecheap which provides an API for modifying DNS records. The only catch is that it can only be accessed via manually-whitelisted IP addresses. Because the server I'm trying to access the API from is assigned a dynamic IP, this restriction was a bit of an issue.

Back in November when I initially made the switch to the DNS-based challenge, I set it up the easy way: I manually added my current IP to the whitelist, added a TODO entry to fix it somehow, and set a reminder scheduled for the week before the cert would expire telling me to update the IP whitelist. Fast-forward to today when I was reminded to update my IP whitelist. Instead of continuing to kick the can down the road, I decided to actually fix the issue.

Setting up a temporary proxy

My home server is connected to the internet though a normal residential connection which is assigned a dynamic IP address. However, the server that I run this website on is configured with a static IP since it's a hosted VPS. By proxying all traffic to the Namecheap API through my VPS, I could add my VPS's static IP to the whitelist and not have to worry about my home IP changing all the time.

SSH is perfect tool for this. The OpenSSH client supports port forwarding using the -D flag. By then setting the HTTP[S]_PROXY environment variables to point to the forwarded port, programs that support those environment variables will transparently forward all their requests through the proxy.

After a bunch of research and testing, I came up with the following script to easily set up the temporary proxy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

#!/bin/sh
# Source this file to automatically setup and teardown an HTTP* proxy through cmetcalfe.ca
# Use the PROXY_PORT and PROXY_DEST environment variables to customize the proxy

PROXY_PORT="${PROXY_PORT:-11111}"
PROXY_DEST="${PROXY_DEST:-<username>@cmetcalfe.ca}"
PID="$$"

# Teardown the SSH connection when the script exits
trap 'ssh -q -S ".ctrl-socket-$PID" -O exit "$PROXY_DEST"' EXIT

# Set up an SSH tunnel and wait for the port to be forwarded before continuing
if ! ssh -o ExitOnForwardFailure=yes -M -S ".ctrl-socket-$PID" -f -N -D "$PROXY_PORT" "$PROXY_DEST"; then
    echo "Failed to open SSH tunnel, exiting"
    exit 1
fi

# Set environment variables to redirect HTTP* traffic through the proxy
export HTTP_PROXY="socks5://127.0.0.1:$PROXY_PORT"
export HTTPS_PROXY="$HTTP_PROXY"

This script is saved as ~/.config/tempproxy.rc and can be sourced to automatically set up a proxy session and have it be torn down when the script exits.

You'll want to use key-based authentication with an unencrypted private key so that you don't need to type anything to initiate the SSH session. For this reason you'll probably want to create a limited user on the target system that can only really do port forwarding. There's a good post on this here.

Talking to the API through the proxy

To allow programs that use the tempproxy.rc script to talk to the Namecheap API, the IP address of the VPS was added to the whitelist. Now that the proxying issue was taken care of, I just needed to wire up the actual certificate renewal process to use it.

The tool I'm using to talk to the Namecheap DNS API is lexicon. It can handle manipulating the DNS records of a ton of providers and integrates really nicely with my ACME client of choice, dehydrated. Also, because it's using the PyNamecheap Python library, which in turn uses requests under the hood, it will automatically use the HTTP*_PROXY environment variables when making requests.

The only tricky bit is that the base install of lexicon won't automatically pull in the packages required for accessing the Namecheap API. Likewise, requests won't install packages to support SOCKS proxies. To install all the required packages you'll need to run a command like:

1

pip install 'requests[socks]' 'dns-lexicon[namecheap]'

Since lexicon can use environment variables as configuration, I created another small source-able file at ~/.config/lexicon.rc:

1
2
3
4
5
6
7
8

#!/bin/sh
# Sets environment variables for accessing the Namecheap API

# Turn on API access and get an API token here:
# https://ap.www.namecheap.com/settings/tools/apiaccess/
export PROVIDER=namecheap
export LEXICON_NAMECHEAP_USERNAME=<username>
export LEXICON_NAMECHEAP_TOKEN=<api token>

With that in place, the existing automation script that I had previously set up when I switched to using the dns-01 challenge just needed some minor tweaks to source the two new files. I've provided the script below, along with some other useful ones that use the DNS API.

Scripts

do-letsencrypt.sh

This is the main script that handles all the domain renewals. It's called via cron on the first of every month.

Fun fact: The previous http-01 version of this script was a mess - it involved juggling nginx configurations with symlinks, manually opening up the .well-known/acme-challenge endpoint, and some other terrible hacks. This version is much nicer.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

#!/bin/sh
# Generate/renew Let's Encrypt certificates using dehydrated and lexicon

set -e

# Set configuration variables and setup the proxy
source ~/.config/lexicon.rc
source ~/.config/tempproxy.rc

cd /etc/nginx/ssl

# Update the certs.
# Hook script copied verbatim from
# https://github.com/AnalogJ/lexicon/blob/master/examples/dehydrated.default.sh
if ! dehydrated --accept-terms --cron --challenge dns-01 --hook dehydrated.default.sh; then
    echo "Failed to renew certificates"
    exit 1
fi

# Reload the nginx config if it's currently running
if ! systemctl is-active nginx.service > /dev/null; then
    echo "nginx isn't running, not reloading it"
    exit 0
fi

if ! systemctl reload nginx.service; then
    systemctl status nginx.service
    echo "Failed to reload nginx config, check the error log"
    exit 1
fi

dns.sh

This one is really simple - it just augments the normal lexicon CLI interface with the proxy and configuration variables.

1
2
3
4
5
6
7

#!/bin/sh
# A simple wrapper around lexicon.
# Sets up the proxy and configuration, then passes all the arguments to the configured provider

source ~/.config/lexicon.rc
source ~/.config/tempproxy.rc
lexicon "$PROVIDER" "$@"

1
2
3
4
5

$ ./dns.sh list cmetcalfe.ca A
ID       TYPE NAME             CONTENT         TTL
-------- ---- ---------------- --------------- ----
xxxxxxxx A    @.cmetcalfe.ca   xxx.xxx.xxx.xxx 1800
xxxxxxxx A    www.cmetcalfe.ca xxx.xxx.xxx.xxx 1800

dns-update.sh

An updated version of my previous script that uses the API instead of the DynamicDNS service Namecheap provides. Called via cron every 30 minutes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

#!/bin/sh
# Check the server's current IP against the IP listed in its DNS entry.
# Set the current IP if they differ.

set -e

resolve() {
    line=$(drill "$1" @resolver1.opendns.com 2> /dev/null | sed '/;;.*$/d;/^\s*$/d' | grep "$1")
    echo "$line" | head -1 | cut -f5
}

dns=$(resolve <subdomain>.cmetcalfe.ca)
curr=$(resolve myip.opendns.com)
if [ "$dns" != "$curr" ]; then
    source ~/.config/lexicon.rc
    source ~/.config/tempproxy.rc
    if ! lexicon "$PROVIDER" delete cmetcalfe.ca A --name "<subdomain>.cmetcalfe.ca"; then
        echo "Failed to delete old DNS record, not setting the new one."
        exit 1
    fi
    if lexicon "$PROVIDER" update cmetcalfe.ca A --name "<subdomain>.cmetcalfe.ca" --content "$curr" --ttl=900; then
        echo "Server DNS record updated ($dns -> $curr)"
    else
        echo "Server DNS record update FAILED (tried $dns -> $curr)"
    fi
fi

In addition to running this website, I also run a home server. For convenience, I point a subdomain of cmetcalfe.ca at it so even though it's connected using a dynamic IP (and actually seems to change fairly frequently), I can get access to it from anywhere.

As a bit of background, the domain for this website is registered and managed through Namecheap. While they do provide a recommended DDNS client for keeping a domain's DNS updated, it only runs on Windows.

Instead, after enabling DDNS for the domain and reading Namecheap's article on using the browser to update DDNS I came up with the following dns-update script.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

#!/bin/sh

# Abort if anything goes wrong (negates the need for error-checking)
set -e

# Uses drill instead of dig
resolve() {
    #dig "$1" @resolver1.opendns.com +short 2> /dev/null
    line=$(drill "$1" @resolver1.opendns.com 2> /dev/null | sed '/;;.*$/d;/^\s*$/d' | grep "$1")
    echo "$line" | head -1 | cut -f5
}

dns=$(resolve <subdomain>.cmetcalfe.ca)
curr=$(resolve myip.opendns.com)
if [ "$dns" != "$curr" ]; then
    if curl -s "https://dynamicdns.park-your-domain.com/update?host=<subdomain>&domain=cmetcalfe.ca&password=<my passkey>" | grep -q "<ErrCount>0</ErrCount>"; then
        echo "Server DNS record updated ($dns -> $curr)"
    else
        echo "Server DNS record update FAILED (tried $dns -> $curr)"
    fi
fi

It basically checks if the IP returned by a DNS query for the subdomain matches the current IP of the server (as reported by an OpenDNS resolver) and if it doesn't, sends a request to update the DNS. The echo commands are there just to output some record of the IP changing. Maybe I'll do some analysis of it at some point.

To run the script every 30 minutes and redirect any output from it to the syslog, the following crontab entry can be used:

1

*/30 * * * * /path/to/dns-update | /usr/bin/logger -t dns-update

With the script automatically running every 30 minutes I can now be confident that my subdomain will always be pointing at my home server whenever I need access to it.

GitHub currently doesn't provide a way to convert an issue to a pull request in their interface. However, the capability exists in their Pull Request API.

To call the API using a simple curl command, run the command:

1
2
3
4

curl --user "[github username]" \
     --request POST \
     --data '{"issue": "[issue num]", "head": "[branch to merge from]", "base": "[branch to merge into]"}' \
     https://api.github.com/repos/[user]/[repo]/pulls

For example, to make user1 change issue 13 into a pull request to merge branch test_branch into master in the testing_repo repository belonging to user2, the command would be:

1
2
3
4

curl --user 'user1' \
     --request POST \
     --data '{"issue": "13", "head": "test_branch", "base": "master"}' \
     https://api.github.com/repos/user2/testing_repo/pulls

To specify a fork of a repository to merge from, put the username followed by a semicolon in front of the branch name like so: "username:branch_name"

After running the command, you will be prompted for your GitHub password. Enter it and curl should output the JSON response from the API. Make sure to check this response for errors!