Unable to renew existing certs, was working at last renewal in October

I’ve got a certificate that’s been working fine up till now for my mail server, but as of this renewal cycle I’m unable to renew. There was a minor update to the packages I’m running including some web content, but I’m not sure if that’s what has caused the problem.
The server is LinuxMagic’s Magicmail 3.0.1-2 which was released on the 20th.

My domain is:
https://mail.fpunet.com/

I ran this command:
/usr/bin/certbot renew
It produced this output:
/usr/bin/certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/mail.fpunet.com-0001.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator apache, Installer apache
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for autodiscover.fpunet.com
http-01 challenge for imap.fpunet.com
http-01 challenge for imap4.fpunet.com
http-01 challenge for mail.fpunet.com
http-01 challenge for pop.fpunet.com
http-01 challenge for pop3.fpunet.com
http-01 challenge for smtp.fpunet.com
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (mail.fpunet.com-0001) from /etc/letsencrypt/renewal/mail.fpunet.com-0001.conf produced an unexpected error: Failed authorization procedure. imap4.fpunet.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://imap4.fpunet.com/.well-known/acme-challenge/aE9Xr9-jOubvbKkQXrpKbjMU4Mp6YNMMxbqaNiMKa_k: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p", pop3.fpunet.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://pop3.fpunet.com/.well-known/acme-challenge/ckehK944BNXQ1ukz8TXxvOeftUYwdgUYmHfKAnwQFCY: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p", autodiscover.fpunet.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://autodiscover.fpunet.com/.well-known/acme-challenge/1SWd7gMKQT6fdEpEMbtl3wVHOGQX8zyrEdPrHfjgRmY: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p", pop.fpunet.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://pop.fpunet.com/.well-known/acme-challenge/1XpCi_sMQSGFwjmzrK6Mq3NVcGhnvRxIrmf9a6L3iYQ: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p", smtp.fpunet.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://smtp.fpunet.com/.well-known/acme-challenge/cMBT9bNaaSzGtFtkUClg3Sd63uW9YjLebfrHEPFjEUA: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p", mail.fpunet.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://mail.fpunet.com/.well-known/acme-challenge/-4abX-xH9j2VDIdo4K8S85kLCcAlcL5f12_EJKHw_xE: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p", imap.fpunet.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://imap.fpunet.com/.well-known/acme-challenge/CuBsN9ST05bnHwXtEbOnj2U5fQfYLw50Ca7IbBXAIvs: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p". Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/mail.fpunet.com-0001/fullchain.pem (failure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/mail.fpunet.com-0001/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: imap4.fpunet.com
   Type:   unauthorized
   Detail: Invalid response from
   http://imap4.fpunet.com/.well-known/acme-challenge/aE9Xr9-jOubvbKkQXrpKbjMU4Mp6YNMMxbqaNiMKa_k:
   "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   Domain: pop3.fpunet.com
   Type:   unauthorized
   Detail: Invalid response from
   http://pop3.fpunet.com/.well-known/acme-challenge/ckehK944BNXQ1ukz8TXxvOeftUYwdgUYmHfKAnwQFCY:
   "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   Domain: autodiscover.fpunet.com
   Type:   unauthorized
   Detail: Invalid response from
   http://autodiscover.fpunet.com/.well-known/acme-challenge/1SWd7gMKQT6fdEpEMbtl3wVHOGQX8zyrEdPrHfjgRmY:
   "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   Domain: pop.fpunet.com
   Type:   unauthorized
   Detail: Invalid response from
   http://pop.fpunet.com/.well-known/acme-challenge/1XpCi_sMQSGFwjmzrK6Mq3NVcGhnvRxIrmf9a6L3iYQ:
   "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   Domain: smtp.fpunet.com
   Type:   unauthorized
   Detail: Invalid response from
   http://smtp.fpunet.com/.well-known/acme-challenge/cMBT9bNaaSzGtFtkUClg3Sd63uW9YjLebfrHEPFjEUA:
   "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   Domain: mail.fpunet.com
   Type:   unauthorized
   Detail: Invalid response from
   http://mail.fpunet.com/.well-known/acme-challenge/-4abX-xH9j2VDIdo4K8S85kLCcAlcL5f12_EJKHw_xE:
   "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   Domain: imap.fpunet.com
   Type:   unauthorized
   Detail: Invalid response from
   http://imap.fpunet.com/.well-known/acme-challenge/CuBsN9ST05bnHwXtEbOnj2U5fQfYLw50Ca7IbBXAIvs:
   "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

https://pastebin.com/hEB1ra1h (with -vvvvvv)
https://pastebin.com/AmN4WjbC (with --debug-challenges)

My web server is (include version):
Server version: Apache/2.4.7 (Ubuntu)
Server built: Apr 18 2018 15:36:26

The operating system my web server runs on is (include version):
Distributor ID: Ubuntu
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Codename: trusty

My hosting provider, if applicable, is: None

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

certbot 0.28.0

Please show:
/etc/letsencrypt/renewal/mail.fpunet.com-0001.conf

Have you made any changes since the last renewal (possibly related to redirection)?
This certbot adjustment seems to fail now:
RewriteEngine on
RewriteRule ^/\.well-known/acme-challenge/([A-Za-z0-9-_=]+)$ /var/lib/letsencrypt/http_challenges/$1 [END]

cat /etc/letsencrypt/renewal/mail.fpunet.com-0001.conf
# renew_before_expiry = 30 days
version = 0.26.1
archive_dir = /etc/letsencrypt/archive/mail.fpunet.com-0001
cert = /etc/letsencrypt/live/mail.fpunet.com-0001/cert.pem
privkey = /etc/letsencrypt/live/mail.fpunet.com-0001/privkey.pem
chain = /etc/letsencrypt/live/mail.fpunet.com-0001/chain.pem
fullchain = /etc/letsencrypt/live/mail.fpunet.com-0001/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = apache
installer = apache
account = 72b81cfb7b1b5cf7ddac862033b7f6d8
server = https://acme-v02.api.letsencrypt.org/directory

I’ve only made changes in the form of updates to the mail and server packages… I’ve looked at the change logs for the mail server package (which includes updates to web configuration) and I don’t see anything that would have made anything major to the apache config files.

Does it edit an existing conf file or add a separate one?

It edited these 4 files, I think:

Adding a temporary challenge validation Include for name: mail.fpunet.com in: /etc/apache2/conf.d/magicmail_http.conf
Adding a temporary challenge validation Include for name: mail.fpunet.com in: /etc/apache2/http-conf.d/webmail.conf
Adding a temporary challenge validation Include for name: webmail.example.com in: /etc/apache2/http-conf.d/portal.conf
Adding a temporary challenge validation Include for name: None in: /etc/apache2/sites-enabled/000-default.conf

It was probably using TLS-SNI validation on port 443 before. But that’s deprecated, and Certbot recently switched to prefer HTTP validation on port 80. HTTP validation has probably never worked, but you probably never tried to use it.

What does, um, “sudo apache2ctl -t -D DUMP_VHOSTS” show?

1 Like

This is a very helpful thing to point out between now and February whenever users have newly failing renewals and have used --apache or --nginx. Although it doesn't solve the problem, it's very likely to be a part of the explanation for why things stopped working recently.

Okay three things I had to fix. One and two, two hosts didn’t have ServerName entries (they were actually subdirectory host declarations… the server is kinda confusing to work on but it makes sense once you look at it)

Three, I was forcing a ssl redirect (I’m not sure this was an issue, but I think it may have been? I actually did this first) which I disabled.

Thank you for the assistance.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.