Challenge failed for domain

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domains are: calolms.com and www.calolms.com and quercus.frantznursery.com

I ran this command: /usr/local/bin/certbot renew

Note that I ran that command in May and it worked fine. I don’t think anything substantial has changed on the server since then. The only difference, that I’m aware of, is that when I ran the command in May my previous certificates had not yet expired. However, when I run it today, my certificates ARE currently expired.

It produced this output:

[root@calolms ~]# /usr/local/bin/certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/calolms.com.conf

Cert is due for renewal, auto-renewing…
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for calolms.com
http-01 challenge for www.calolms.com
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification…
Challenge failed for domain calolms.com
Challenge failed for domain www.calolms.com
http-01 challenge for calolms.com
http-01 challenge for www.calolms.com
Cleaning up challenges
Attempting to renew cert (calolms.com) from /etc/letsencrypt/renewal/calolms.com.conf produced an unexpected error: Some challenges have failed… Skipping.

Processing /etc/letsencrypt/renewal/quercus.frantznursery.com.conf

Cert is due for renewal, auto-renewing…
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for quercus.frantznursery.com
Cleaning up challenges
Attempting to renew cert (quercus.frantznursery.com) from /etc/letsencrypt/renewal/quercus.frantznursery.com.conf produced an unexpected error: Problem binding to port 80: Could not bind to IPv4 or IPv6… Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/calolms.com/fullchain.pem (failure)
/etc/letsencrypt/live/quercus.frantznursery.com/fullchain.pem (failure)

All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/calolms.com/fullchain.pem (failure)
/etc/letsencrypt/live/quercus.frantznursery.com/fullchain.pem (failure)

2 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

My web server is (include version): Amazon linux (EC2)…

The operating system my web server runs on is (include version):

[root@calolms ~]# uname -a
Linux calolms.com 4.14.181-108.257.amzn1.x86_64 #1 SMP Wed May 27 02:43:03 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

My hosting provider, if applicable, is: AWS

I can login to a root shell on my machine (yes or no, or I don’t know): YES

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): NO

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):

[root@calolms ~]# certbot --version
certbot 0.40.1

[root@calolms ~]# certbot-auto --version
-bash: certbot-auto: command not found

Additional details you did not ask for… It appears as if there’s some kind of problem with the “well-defined/acme-challenge” mechanism. So, to be sure, here are the permissions of all the pertinent direcrtories:

[root@calolms ~]# ls -ald /
dr-xr-xr-x 26 root root 4096 Jun 6 22:34 /

[root@calolms ~]# ls -ald /var
drwxr-xr-x 20 root root 4096 Sep 16 2019 /var

[root@calolms ~]# ls -ald /var/www
drwxr-xr-x 8 root root 4096 May 14 11:13 /var/www

[root@calolms ~]# ls -ald /var/www/html
drwxr-xr-x 15 root root 4096 Jul 24 17:21 /var/www/html

[root@calolms ~]# ls -ald /var/www/html/.well-known
drwxr-xr-x 3 root root 4096 May 7 11:53 /var/www/html/.well-known

[root@calolms ~]# ls -ald /var/www/html/.well-known/acme-challenge
drwxr-xr-x 2 root root 4096 Aug 3 17:15 /var/www/html/.well-known/acme-challenge

[root@calolms ~]# ls -al /var/www/html/.well-known/acme-challenge
total 8
drwxr-xr-x 2 root root 4096 Aug 3 17:15 .
drwxr-xr-x 3 root root 4096 May 7 11:53 …
-rw-r–r-- 1 root root 0 May 7 11:53 O4JFsJ1UgzFX_doR6skfqjJGjceLst5imK9paMx1ctU

That last one may be telling… it at least shows that a file was built via this mechanism last May, which I think, maybe?, indicates that it worked back then. It also shows that the acme-challenge directory appears to have been modified today (Aug 3)… so something happened there today.

Ok, so what now?

2

One of your certificates is using the standalone authenticator, which isn’t going to work when Apache is running.

With the webroot, you could try dropping a file in there and seeing whether you can access it in your browser.

More generally, does the Apache authenticator work for you?

/usr/local/bin/certbot renew -a apache --dry-run
4

[root@calolms ~]# /usr/local/bin/certbot renew -a apache --dry-run

Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/calolms.com.conf

Cert is due for renewal, auto-renewing…

Plugins selected: Authenticator apache, Installer None

Renewing an existing certificate

Performing the following challenges:

http-01 challenge for calolms.com

http-01 challenge for www.calolms.com

Waiting for verification…

Cleaning up challenges

new certificate deployed without reload, fullchain is

/etc/letsencrypt/live/calolms.com/fullchain.pem

Processing /etc/letsencrypt/renewal/quercus.frantznursery.com.conf

Cert is due for renewal, auto-renewing…

Plugins selected: Authenticator apache, Installer None

Renewing an existing certificate

Performing the following challenges:

http-01 challenge for quercus.frantznursery.com

Waiting for verification…

Cleaning up challenges

new certificate deployed without reload, fullchain is

/etc/letsencrypt/live/quercus.frantznursery.com/fullchain.pem

** DRY RUN: simulating ‘certbot renew’ close to cert expiry

** (The test certificates below have not been saved.)

Congratulations, all renewals succeeded. The following certs have been renewed:

/etc/letsencrypt/live/calolms.com/fullchain.pem (success)

/etc/letsencrypt/live/quercus.frantznursery.com/fullchain.pem (success)

** DRY RUN: simulating ‘certbot renew’ close to cert expiry

** (The test certificates above have not been saved.)

IMPORTANT NOTES:

  • Your account credentials have been saved in your Certbot

configuration directory at /etc/letsencrypt. You should make a

secure backup of this folder now. This configuration directory will

also contain certificates and private keys obtained by Certbot so

making regular backups of this folder is ideal.

[root@calolms ~]#

5

Since the method suggested by @_az works, can you run
/usr/local/bin/certbot renew -a apache --dry-run -i apache

This will test if certbot get the certificate and reload your webserver.

If the above command executed without issue, try /usr/local/bin/certbot renew -a apache -i apache (this will renew the certificate in reality)

6

Ok, so --dry-run appears to have worked.

Are you saying that all I have to do is stop apache before running certbot? That’s easy enough, but I don’t recall having to do that before. If it is the correct thing to do, then I can certainly do that from now on.

7

Please don't do that.... That's the thing not to do.

8

Here you go...

[root@calolms ~]# /usr/local/bin/certbot renew -a apache --dry-run -i apache

Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/calolms.com.conf

Cert is due for renewal, auto-renewing...

Plugins selected: Authenticator apache, Installer apache

Renewing an existing certificate

Performing the following challenges:

http-01 challenge for calolms.com

http-01 challenge for www.calolms.com

Waiting for verification...

Cleaning up challenges

new certificate deployed with reload of apache server; fullchain is

/etc/letsencrypt/live/calolms.com/fullchain.pem

Processing /etc/letsencrypt/renewal/quercus.frantznursery.com.conf

Cert is due for renewal, auto-renewing...

Plugins selected: Authenticator apache, Installer apache

Renewing an existing certificate

Performing the following challenges:

http-01 challenge for quercus.frantznursery.com

Waiting for verification...

Cleaning up challenges

new certificate deployed with reload of apache server; fullchain is

/etc/letsencrypt/live/quercus.frantznursery.com/fullchain.pem

** DRY RUN: simulating 'certbot renew' close to cert expiry

** (The test certificates below have not been saved.)

Congratulations, all renewals succeeded. The following certs have been renewed:

/etc/letsencrypt/live/calolms.com/fullchain.pem (success)

/etc/letsencrypt/live/quercus.frantznursery.com/fullchain.pem (success)

** DRY RUN: simulating 'certbot renew' close to cert expiry

** (The test certificates above have not been saved.)

[root@calolms ~]#

9

Since everything will work perfectly, you can actually perform the renewal by running: /usr/local/bin/certbot renew -a apache -i apache

10

One note: that is assuming you want Certbot to configure the SSL certificate for you by rewriting your Apache configuration.

If you want to manage the Apache config by hand, a different command would be suitable.

11

Ok, thanks so much. That appears to have worked fine. But I don't understand why this happened this time, but not the last time in May, nor the time before that. I guess it doesn't much matter. I will use "/usr/local/bin/certbot renew -a apache -i apache" from now on.

Thanks again! I consider this closed... do I need to do something somewhere to "close" it?

12

You don’t need to execute anything additional from this point, the changes @_az suggested (and you executed) are stored in certbot renewal configuration automatically, so you just need to make sure certbot has a background cronjob running (use sudo crontab -l and identify if there’s anything similar to certbot renew -q .

14

Regarding: "… so you just need to make sure certbot has a background cronjob running (use sudo crontab -l and identify if there’s anything similar to `certbot renew -q "

I have never put anything in my crontab. EVERY TIME I renew my letsencrypt certificates, going back two years at least, something different seems to go wrong (though in all fairness, it did go without a hitch in May). So until I believe this is a stable system that will work consistently, I won’t be putting anything in my crontab, sorry.

Instead, my standard procedure is to wait until I receive the reminder email from letsencrypt.org that my certificates will be expiring shortly, then I handle it manually… because inevitably (and proven yet again by today’s experience) something goes wrong.