Certbot renew invalid response from .well-known/acme-challenge/

Help
1

I am having trouble renewing a certificate for one domain.

I ran this command: certbot renew --dry-run --debug

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/cloud.domain1.com.conf

Cert not due for renewal, but simulating renewal for dry run
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for cloud.domain1.com
Waiting for verification…
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0061_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0061_csr-certbot.pem

Processing /etc/letsencrypt/renewal/secure.domain2.nl.conf

Cert not due for renewal, but simulating renewal for dry run
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for secure.domain2.nl
Waiting for verification…
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0062_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0062_csr-certbot.pem

Processing /etc/letsencrypt/renewal/crm.domain3.com.conf

Cert is due for renewal, auto-renewing…
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for crm.domain3.com
Waiting for verification…
Cleaning up challenges
Unable to clean up challenge directory /var/www/suitecrm/.well-known/acme-challenge
Attempting to renew cert from /etc/letsencrypt/renewal/crm.domain3.com.conf produced an unexpected error: Failed authorization procedure. crm.domain.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://crm.domain3.com/.well-known/acme-challenge/QQ5-bi-PtXyVh3cQ8VDM0gTLPjL-CMgUEf8bJGO93Jk: "

<meta name="viewpo". Skipping.

Processing /etc/letsencrypt/renewal/domain3.com.conf

Cert is due for renewal, auto-renewing…
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for domain3.com
Waiting for verification…
Cleaning up challenges
Attempting to renew cert from /etc/letsencrypt/renewal/domain3.com.conf produced an unexpected error: Failed authorization procedure. domain3.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://domain3.com/.well-known/acme-challenge/_pnYqX2EtGSfOEctVe326-iHmyCsvORP9Z8XAuj-r5k: "

404 Not Found

Not Found

Th". Skipping. ** DRY RUN: simulating 'certbot renew' close to cert expiry ** (The test certificates below have not been saved.)

The following certs were successfully renewed:
/etc/letsencrypt/live/cloud.domain1.com/fullchain.pem (success)
/etc/letsencrypt/live/secure.domain2.nl/fullchain.pem (success)

The following certs could not be renewed:
/etc/letsencrypt/live/crm.domain3.com/fullchain.pem (failure)
/etc/letsencrypt/live/domain3.com/fullchain.pem (failure)
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates above have not been saved.)
Traceback (most recent call last):
File “/usr/bin/certbot”, line 11, in
load_entry_point(‘certbot==0.10.2’, ‘console_scripts’, ‘certbot’)()
File “/usr/lib/python2.7/dist-packages/certbot/main.py”, line 849, in main
return config.func(config, plugins)
File “/usr/lib/python2.7/dist-packages/certbot/main.py”, line 655, in renew
renewal.handle_renewal_request(config)
File “/usr/lib/python2.7/dist-packages/certbot/renewal.py”, line 430, in handle_renewal_request
len(renew_failures), len(parse_failures)))
Error: 2 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

My web server is (include version): Apache/2.4.10 (Raspbian)

The operating system my web server runs on is (include version): raspbian lite jessie

My hosting provider, if applicable, is: not applicable

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no

Other details:
it has worked and then stopped working and don’t remember changing anything that would cause this behavior
if i create testfile in /.well-known/acme-challenge then I can read the file in browser with no problems
i have http forewarded to https to force use of https

2

Hello @mdw

you are using a very old certbot. I think, actual 0.25 is online.

And: What's your domain? Without domain, it's hard to help.

3

problem (sub)domain is crm.markdewijn.com (domain3)

4

The domain doesn't respond. Is there a firewall? www.markdewijn.com works.

The IP-configuration doesn't look good:

D:>nslookup
Standardserver: fritz.box
Address: 192.168.178.1

www.markdewijn.com
Server: fritz.box
Address: 192.168.178.1

Nicht autorisierende Antwort:
Name: www.markdewijn.com
Addresses: 2a0b:7280:200:0:454:7eff:fe00:dc9
185.182.56.226

crm.markdewijn.com
Server: fritz.box
Address: 192.168.178.1

Nicht autorisierende Antwort:
Name: crm.markdewijn.com
Addresses: 2a0b:7280:200:0:454:7eff:fe00:dc9
145.130.144.88

The IPv6 - addresses are the same. But the Pv4-addresses are different.

5

The domain (www.)markdewijn.com is on my ISP server (185.182.56.226). The domain crm.markdewijn.com DNS points to different personal server (145.130.144.88). This is correct. The domain cloud.quimpro.com (domain1) is on the same server as crm.markdewijn.com (145.130.144.88) and resolves the certs just fine. Is don’t know wy the ipv6 adressen are the same.

6

But there is no webserver running. If you want to use http-01 - challenge, there must be a webserver / Port 80 and (temporarily) a file

http://crm.markdewijn.com/.well-known/acme-challenge/token-of-the-challenge

so that Letsencrypt can fetch this file. And you should remove the AAAA-record, if it is wrong. Because Letsencrypt may use it.

7

You weer absolute lyceum right, yesterday afternoon a new router was implementeren and not alle ports wereld correctly configured. Sorry for the inconvenience. Please try again.

8

I am hoping that you are still willing to help me with my cert problem.

9

There is no webservice on crm.markdewijn.com

So http-01 - challenge can’t work

10

Did you test again? crm.markdewijn.com and cloud.quimpro.com?
Because there definitely is a webserver running now which I can reach.
If you cannot see it, there is something else wrong entirely.

11

I do have http forewarded to https, can that be the problem? If so, how can one domain work with certbot and the other not?

12

That doesn't work. http + Port 80 - timeout. http-01 - challenge needs a Port 80, perhaps a redirect to port 443 / https.

https://crm.markdewijn.com/ responses, but there is the certificate expired:

crm.markdewijn.com verwendet ein ungĂźltiges Sicherheitszertifikat. Das Zertifikat ist am Freitag, 1. September 2017, 23:26 abgelaufen. Die aktuelle Zeit ist Dienstag, 19. Juni 2018, 21:18. Fehlercode: SEC_ERROR_EXPIRED_CERTIFICATE

13

Yes, this is exactly the problem, I cannot renew the certificate

About the forewarding, i have this in a script
<VirtualHost :80>
#ServerAdmin webmaster@localhost
RewriteEngine On
RewriteCond %{SERVER_PORT} !443
RewriteRule ^(/(.))?$ https://%{HTTP_HOST}/$1 [R=301,L]

14

This works without problems for the domain cloud.quimpro.com

15

No. Same - timeout on port 80, no connection. May be a firewall or a wrong router-configuration, which blocks port 80.

16

certbot renew --dry-run --debug
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/cloud.quimpro.com.conf

Cert not due for renewal, but simulating renewal for dry run
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for cloud.quimpro.com
Waiting for verification…
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0065_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0065_csr-certbot.pem

Processing /etc/letsencrypt/renewal/secure.familiedewijn.nl.conf

Cert not due for renewal, but simulating renewal for dry run
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for secure.familiedewijn.nl
Waiting for verification…
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0066_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0066_csr-certbot.pem

Processing /etc/letsencrypt/renewal/crm.markdewijn.com.conf

Cert is due for renewal, auto-renewing…
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for crm.markdewijn.com
Waiting for verification…
Cleaning up challenges
Unable to clean up challenge directory /var/www/suitecrm/.well-known/acme-challenge
Attempting to renew cert from /etc/letsencrypt/renewal/crm.markdewijn.com.conf produced an unexpected error: Failed authorization procedure. crm.markdewijn.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://crm.markdewijn.com/.well-known/acme-challenge/pa6Yp-wpRc1dnZMF-si5LwD99M9R1T6AXbrT9Nmf1bo: "

<meta name="viewpo". Skipping.

Processing /etc/letsencrypt/renewal/markdewijn.com.conf

Cert is due for renewal, auto-renewing…
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for markdewijn.com
Waiting for verification…
Cleaning up challenges
Attempting to renew cert from /etc/letsencrypt/renewal/markdewijn.com.conf produced an unexpected error: Failed authorization procedure. markdewijn.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://markdewijn.com/.well-known/acme-challenge/MKqVTY096OHxwcHv_4jt2_uCa5yKnwAhlw7fpz1VTUc: "

404 Not Found

Not Found

Th". Skipping. ** DRY RUN: simulating 'certbot renew' close to cert expiry ** (The test certificates below have not been saved.)

The following certs were successfully renewed:
/etc/letsencrypt/live/cloud.quimpro.com/fullchain.pem (success)
/etc/letsencrypt/live/secure.familiedewijn.nl/fullchain.pem (success)

The following certs could not be renewed:
/etc/letsencrypt/live/crm.markdewijn.com/fullchain.pem (failure)
/etc/letsencrypt/live/markdewijn.com/fullchain.pem (failure)
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates above have not been saved.)
Traceback (most recent call last):
File “/usr/bin/certbot”, line 11, in
load_entry_point(‘certbot==0.10.2’, ‘console_scripts’, ‘certbot’)()
File “/usr/lib/python2.7/dist-packages/certbot/main.py”, line 849, in main
return config.func(config, plugins)
File “/usr/lib/python2.7/dist-packages/certbot/main.py”, line 655, in renew
renewal.handle_renewal_request(config)
File “/usr/lib/python2.7/dist-packages/certbot/renewal.py”, line 430, in handle_renewal_request
len(renew_failures), len(parse_failures)))
Error: 2 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

17

Now http://cloud.quimpro.com/ works and redirects to https.

Now http://crm.markdewijn.com/ works also, with redirect.

Check Certbot again

18

same result

certbot renew --dry-run --debug
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/cloud.quimpro.com.conf

Cert not due for renewal, but simulating renewal for dry run
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for cloud.quimpro.com
Waiting for verification…
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0069_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0069_csr-certbot.pem

Processing /etc/letsencrypt/renewal/secure.familiedewijn.nl.conf

Cert not due for renewal, but simulating renewal for dry run
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for secure.familiedewijn.nl
Waiting for verification…
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0070_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0070_csr-certbot.pem

Processing /etc/letsencrypt/renewal/crm.markdewijn.com.conf

Cert is due for renewal, auto-renewing…
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for crm.markdewijn.com
Waiting for verification…
Cleaning up challenges
Unable to clean up challenge directory /var/www/suitecrm/.well-known/acme-challenge
Attempting to renew cert from /etc/letsencrypt/renewal/crm.markdewijn.com.conf produced an unexpected error: Failed authorization procedure. crm.markdewijn.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://crm.markdewijn.com/.well-known/acme-challenge/eOgYcIHdUI769_XA4oCp_u5uPNGwyclPgTN0gJMywdI: "

<meta name="viewpo". Skipping.

Processing /etc/letsencrypt/renewal/markdewijn.com.conf

Cert is due for renewal, auto-renewing…
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for markdewijn.com
Waiting for verification…
Cleaning up challenges
Attempting to renew cert from /etc/letsencrypt/renewal/markdewijn.com.conf produced an unexpected error: Failed authorization procedure. markdewijn.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://markdewijn.com/.well-known/acme-challenge/cdjL1Dr0_DbiBMj7SQbkz-otPLDhSZQvJWgTIutPMMg: "

404 Not Found

Not Found

Th". Skipping. ** DRY RUN: simulating 'certbot renew' close to cert expiry ** (The test certificates below have not been saved.)

The following certs were successfully renewed:
/etc/letsencrypt/live/cloud.quimpro.com/fullchain.pem (success)
/etc/letsencrypt/live/secure.familiedewijn.nl/fullchain.pem (success)

The following certs could not be renewed:
/etc/letsencrypt/live/crm.markdewijn.com/fullchain.pem (failure)
/etc/letsencrypt/live/markdewijn.com/fullchain.pem (failure)
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates above have not been saved.)
Traceback (most recent call last):
File “/usr/bin/certbot”, line 11, in
load_entry_point(‘certbot==0.10.2’, ‘console_scripts’, ‘certbot’)()
File “/usr/lib/python2.7/dist-packages/certbot/main.py”, line 849, in main
return config.func(config, plugins)
File “/usr/lib/python2.7/dist-packages/certbot/main.py”, line 655, in renew
renewal.handle_renewal_request(config)
File “/usr/lib/python2.7/dist-packages/certbot/renewal.py”, line 430, in handle_renewal_request
len(renew_failures), len(parse_failures)))
Error: 2 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

19

But now you should fix your ipv6 - error. Remove all ipv6-entries.

20

Please explain to me in more detail please.