Enable letsencrypt SSL setup

[jamesmontalvo3]

Our HAProxy role has some leftover portions (from the tutorial used to create it) commented out which enable letsencrypt SSL setup. It'd be great to automate that for meza.

[clauded]

Here's how to manually enable letsencrypt:

1. Make sure port 54321 is open in the firewall
2. Run the following commands (as sudo) to setup letsencrypt:

yum install -y certbot
export DOMAIN_NAME=mywiki.com
export ADMIN_EMAIL=admin@mywiki.com
certbot certonly --non-interactive --email ${ADMIN_EMAIL} \
  --preferred-challenges http --standalone --agree-tos --renew-by-default \
  --webroot-path /opt/htdocs -d ${DOMAIN_NAME} --http-01-port=54321
cat /etc/letsencrypt/live/${DOMAIN_NAME}/fullchain.pem /etc/letsencrypt/live/${DOMAIN_NAME}/privkey.pem > /etc/haproxy/certs/meza-ls.pem
chmod 600 /etc/haproxy/certs/meza-ls.pem

3. Run the following commands (as sudo) to modify HAproxy config:

export HA_CFG=/etc/haproxy/haproxy.cfg
sed -i -e 's/meza.pem/meza-ls.pem/' ${HA_CFG}
sed -i -e 's/# acl letsencrypt-acl/acl letsencrypt-acl/' ${HA_CFG}
sed -i -e 's/# use_backend letsencrypt-backend/use_backend letsencrypt-backend/' ${HA_CFG}
sed -i -e 's/# backend letsencrypt-backend/backend letsencrypt-backend/' ${HA_CFG}
sed -i -e 's/# \tserver letsencrypt 127.0.0.1:54321/\tserver letsencrypt 127.0.0.1:54321/' ${HA_CFG}
systemctl reload haproxy

4. Run the following commands (as sudo) to create a script that will run on a daily basis to renew the cert:

cat > /etc/cron.daily/update-certs <<EOF
#!/bin/bash
# Renew the letsencrypt certificate
DOMAIN_NAME=mywiki.com
certbot renew --force-renewal --tls-sni-01-port=54321
# Concatenate new cert files
bash -c "cat /etc/letsencrypt/live/${DOMAIN_NAME}/fullchain.pem /etc/letsencrypt/live/${DOMAIN_NAME}/privkey.pem > /etc/haproxy/certs/meza-ls.pem"
# Reload  HAProxy
systemctl reload haproxy
exit 0
EOF
chmod +x /etc/cron.daily/update-certs

5. From this, it should be easy to create a script in Meza to automate it.

[jamesmontalvo3]

Thanks @clauded!

@freephile @hexmode I think you've both done some work with Meza+Letsencrypt. Thoughts on @clauded's inputs above? Thoughts on Ansible-izing/Meza-izing it?

[clauded]

Well, I just ran sudo meza deploy monolith and lost my haproxy.cfg settings...
So there should be 3 parameters to manage this with Ansible:

1. an on/off switch for letsencrypt (default off)
2. domain_name variable
3. admin_name variable

Also the letsencrypt port (54321) could be a variable and opened by Ansible.

[freephile]

Looks good to me.

I've tried 3 approaches:

1. ignore (supplement) meza.pem (with another LE pem) which requires modifying haproxy config
2. overwrite meza.pem so that it's actually a trusted cert so you don't have to modify haproxy config
3. bind haproxy to the cert directory rather than a file in it so the file can be named example.com.pem

This last approach requires that you delete the meza.cert and meza.key file in /etc/haproxy/certs (leaving only .pem files in the directory). In the example above
sed -i -e 's/meza.pem/meza-ls.pem/' ${HA_CFG}
becomes
sed -i -e 's/meza.pem//' ${HA_CFG}

[clauded]

I've tested the third approach and it works fine. Here's a script (install-le) to automate this while waiting for an Ansible solution:

#!/bin/bash
DOMAIN_NAME="$1"
ADMIN_EMAIL="$2"

echo "Script to install and configure LetsEncrypt"
if [ "$#" != "2" ]; then
  echo "Usage is: install-le domain_name admin_email"
  exit 1
fi

# this wont be needed if haproxy.cfg has letsencrypt enabled by default
echo "Adjusting HAproxy.cfg file..."
HA_CFG=/etc/haproxy/haproxy.cfg
sed -i -e 's/meza.pem//' ${HA_CFG}
sed -i -e 's/# acl letsencrypt-acl/acl letsencrypt-acl/' ${HA_CFG}
sed -i -e 's/# use_backend letsencrypt-backend/use_backend letsencrypt-backend/' ${HA_CFG}
sed -i -e 's/# backend letsencrypt-backend/backend letsencrypt-backend/' ${HA_CFG}
sed -i -e 's/# \tserver letsencrypt 127.0.0.1:54321/\tserver letsencrypt 127.0.0.1:54321/' ${HA_CFG}
#

rm -f /etc/haproxy/certs/meza.crt /etc/haproxy/certs/meza.key /etc/haproxy/certs/meza.pem

if [ -f /etc/haproxy/certs/${DOMAIN_NAME}.pem ]; then
  echo "LetsEncrypt already configured!"
else
  echo "Install LetsEncrypt..."
  yum install -y certbot
  echo "Generate SSL certificate..."
  certbot certonly --non-interactive --email ${ADMIN_EMAIL} \
    --preferred-challenges http --standalone --agree-tos --renew-by-default \
    --webroot-path /opt/htdocs -d ${DOMAIN_NAME} --http-01-port=54321
  echo "Concatenate certificate files for HAproxy..."
  cat /etc/letsencrypt/live/${DOMAIN_NAME}/fullchain.pem /etc/letsencrypt/live/${DOMAIN_NAME}/privkey.pem > /etc/haproxy/certs/${DOMAIN_NAME}.pem
  chmod 600 /etc/haproxy/certs/${DOMAIN_NAME}.pem
  echo "Create a cron job ran daily to renew the ssl certificate..."
  cat > /etc/cron.daily/update-certs <<EOF
#!/bin/bash
# Renew the LetsEncrypt certificate
DOMAIN_NAME=${DOMAIN_NAME}
certbot renew --force-renewal --tls-sni-01-port=54321
bash -c "cat /etc/letsencrypt/live/\${DOMAIN_NAME}/fullchain.pem /etc/letsencrypt/live/\${DOMAIN_NAME}/privkey.pem > /etc/haproxy/certs/\${DOMAIN_NAME}.pem"
service haproxy reload
exit 0
EOF
  chmod +x /etc/cron.daily/update-certs
fi

echo "Reloading HAproxy..."
systemctl reload haproxy
echo "Please make sure port 54321 is opened."

[revansx]

I'm trying to perform the steps outlined above for a meza 31.8.2 centos7 system. I've opened up firewalld to tcp traffic on port 54321, but the "certbot certonly.." command in step 2 is failing on the verification of the challenge with the following message:

 - The following errors were reported by the server:

   Domain: emw-meza.site
   Type:   unauthorized
   Detail: Invalid response from
   https://emw-meza.site/.well-known/acme-challenge/tOBlSG4Zvp84285RG067NW_dNtLQVPYGaB6MU-CxMT8
   [204.48.31.246]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

my site is indeed "emw-meza.site" but the "/.well-known/acme-challenge/..." path doesn't make sense to me. What am I missing?

[freephile]

You need to create that directory structure in the document root and make it writable by the web server.

[revansx]

thanks. but gonna have to wait now.. letsencrypt has a rate limit on failures per hour it will tolerate

[revansx]

I think certbot is indeed creating the challenge folder and token content, however, i think the problem is the .htaccess file that meza creates does not allow token file certbot is creating at /.well-known/acme-challenge to be seen by letsencrypt (or anyone). Is this likely? Is there an easy change I can make to .htaccess that will allow content off of /opt/htdocs/something_not_a_wiki/ to be seen?

[revansx]

I have confirmed this by creating the folder /opt/htdocs/.well-known/acme-challenge/test.txt and confirmed that https://mysite/.well-known/acme-challenge/test.txt results in a 404. This is a default install of meza 31.8.1.

[freephile]

I have a working version of this in my 34.x branch

I didn't implement the renewal cron because the packages installed by Debian and CentOS setup cron or system.d timers (https://certbot.eff.org/docs/using.html#automated-renewals) I am working on a renewal hook (or installer plugin for HAProxy) that would concatenate ALL certificates in case a Meza system serves more than a single FQDN.