Certbot renew invalid response from .well-known/acme-challenge/

mdw · June 18, 2018, 12:43pm

I am having trouble renewing a certificate for one domain.

I ran this command: certbot renew --dry-run --debug

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/cloud.domain1.com.conf

Cert not due for renewal, but simulating renewal for dry run
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for cloud.domain1.com
Waiting for verification…
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0061_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0061_csr-certbot.pem

Processing /etc/letsencrypt/renewal/secure.domain2.nl.conf

Cert not due for renewal, but simulating renewal for dry run
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for secure.domain2.nl
Waiting for verification…
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0062_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0062_csr-certbot.pem

Processing /etc/letsencrypt/renewal/crm.domain3.com.conf

Cert is due for renewal, auto-renewing…
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for crm.domain3.com
Waiting for verification…
Cleaning up challenges
Unable to clean up challenge directory /var/www/suitecrm/.well-known/acme-challenge
Attempting to renew cert from /etc/letsencrypt/renewal/crm.domain3.com.conf produced an unexpected error: Failed authorization procedure. crm.domain.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://crm.domain3.com/.well-known/acme-challenge/QQ5-bi-PtXyVh3cQ8VDM0gTLPjL-CMgUEf8bJGO93Jk: "

Processing /etc/letsencrypt/renewal/domain3.com.conf

Cert is due for renewal, auto-renewing…
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for domain3.com
Waiting for verification…
Cleaning up challenges
Attempting to renew cert from /etc/letsencrypt/renewal/domain3.com.conf produced an unexpected error: Failed authorization procedure. domain3.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://domain3.com/.well-known/acme-challenge/_pnYqX2EtGSfOEctVe326-iHmyCsvORP9Z8XAuj-r5k: "

404 Not Found

Not Found

Th". Skipping. ** DRY RUN: simulating 'certbot renew' close to cert expiry ** (The test certificates below have not been saved.)

The following certs were successfully renewed:
/etc/letsencrypt/live/cloud.domain1.com/fullchain.pem (success)
/etc/letsencrypt/live/secure.domain2.nl/fullchain.pem (success)

The following certs could not be renewed:
/etc/letsencrypt/live/crm.domain3.com/fullchain.pem (failure)
/etc/letsencrypt/live/domain3.com/fullchain.pem (failure)
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates above have not been saved.)
Traceback (most recent call last):
File “/usr/bin/certbot”, line 11, in
load_entry_point(‘certbot==0.10.2’, ‘console_scripts’, ‘certbot’)()
File “/usr/lib/python2.7/dist-packages/certbot/main.py”, line 849, in main
return config.func(config, plugins)
File “/usr/lib/python2.7/dist-packages/certbot/main.py”, line 655, in renew
renewal.handle_renewal_request(config)
File “/usr/lib/python2.7/dist-packages/certbot/renewal.py”, line 430, in handle_renewal_request
len(renew_failures), len(parse_failures)))
Error: 2 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

The following errors were reported by the server:

Domain: crm.domain3.com
Type: unauthorized
Detail: Invalid response from
http://crm.domain3.com/.well-known/acme-challenge/QQ5-bi-PtXyVh3cQ8VDM0gTLPjL-CMgUEf8bJGO93Jk:
"
<meta name="viewpo"
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A record(s) for that domain
contain(s) the right IP address.
The following errors were reported by the server:

Domain: domain3.com
Type: unauthorized
Detail: Invalid response from
http://domain3.com/.well-known/acme-challenge/_pnYqX2EtGSfOEctVe326-iHmyCsvORP9Z8XAuj-r5k:
"
404 Not Found
Not Found
Th"
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A record(s) for that domain
contain(s) the right IP address.

My web server is (include version): Apache/2.4.10 (Raspbian)

The operating system my web server runs on is (include version): raspbian lite jessie

My hosting provider, if applicable, is: not applicable

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no

Other details:
it has worked and then stopped working and don’t remember changing anything that would cause this behavior
if i create testfile in /.well-known/acme-challenge then I can read the file in browser with no problems
i have http forewarded to https to force use of https

JuergenAuer · June 18, 2018, 5:10pm

Hello @mdw

you are using a very old certbot. I think, actual 0.25 is online.

And: What's your domain? Without domain, it's hard to help.

mdw · June 18, 2018, 6:37pm

problem (sub)domain is crm.markdewijn.com (domain3)

JuergenAuer · June 18, 2018, 7:28pm

The domain doesn't respond. Is there a firewall? www.markdewijn.com works.

The IP-configuration doesn't look good:

D:>nslookup
Standardserver: fritz.box
Address: 192.168.178.1

www.markdewijn.com
Server: fritz.box
Address: 192.168.178.1

Nicht autorisierende Antwort:
Name: www.markdewijn.com
Addresses: 2a0b:7280:200:0:454:7eff:fe00:dc9
185.182.56.226

crm.markdewijn.com
Server: fritz.box
Address: 192.168.178.1

Nicht autorisierende Antwort:
Name: crm.markdewijn.com
Addresses: 2a0b:7280:200:0:454:7eff:fe00:dc9
145.130.144.88

The IPv6 - addresses are the same. But the Pv4-addresses are different.

mdw · June 18, 2018, 8:13pm

The domain (www.)markdewijn.com is on my ISP server (185.182.56.226). The domain crm.markdewijn.com DNS points to different personal server (145.130.144.88). This is correct. The domain cloud.quimpro.com (domain1) is on the same server as crm.markdewijn.com (145.130.144.88) and resolves the certs just fine. Is don’t know wy the ipv6 adressen are the same.

JuergenAuer · June 18, 2018, 8:29pm

But there is no webserver running. If you want to use http-01 - challenge, there must be a webserver / Port 80 and (temporarily) a file

http://crm.markdewijn.com/.well-known/acme-challenge/token-of-the-challenge

so that Letsencrypt can fetch this file. And you should remove the AAAA-record, if it is wrong. Because Letsencrypt may use it.

mdw · June 19, 2018, 4:58am

You weer absolute lyceum right, yesterday afternoon a new router was implementeren and not alle ports wereld correctly configured. Sorry for the inconvenience. Please try again.

mdw · June 19, 2018, 3:41pm

I am hoping that you are still willing to help me with my cert problem.

JuergenAuer · June 19, 2018, 3:58pm

There is no webservice on crm.markdewijn.com

So http-01 - challenge can’t work

mdw · June 19, 2018, 7:16pm

Did you test again? crm.markdewijn.com and cloud.quimpro.com?
Because there definitely is a webserver running now which I can reach.
If you cannot see it, there is something else wrong entirely.

mdw · June 19, 2018, 7:18pm

I do have http forewarded to https, can that be the problem? If so, how can one domain work with certbot and the other not?

JuergenAuer · June 19, 2018, 7:21pm

That doesn't work. http + Port 80 - timeout. http-01 - challenge needs a Port 80, perhaps a redirect to port 443 / https.

https://crm.markdewijn.com/ responses, but there is the certificate expired:

crm.markdewijn.com verwendet ein ungültiges Sicherheitszertifikat. Das Zertifikat ist am Freitag, 1. September 2017, 23:26 abgelaufen. Die aktuelle Zeit ist Dienstag, 19. Juni 2018, 21:18. Fehlercode: SEC_ERROR_EXPIRED_CERTIFICATE

mdw · June 19, 2018, 7:26pm

Yes, this is exactly the problem, I cannot renew the certificate

About the forewarding, i have this in a script
<VirtualHost :80>
#ServerAdmin webmaster@localhost
RewriteEngine On
RewriteCond %{SERVER_PORT} !443
RewriteRule ^(/(.))?$ https://%{HTTP_HOST}/$1 [R=301,L]

mdw · June 19, 2018, 7:30pm

This works without problems for the domain cloud.quimpro.com

JuergenAuer · June 19, 2018, 7:31pm

No. Same - timeout on port 80, no connection. May be a firewall or a wrong router-configuration, which blocks port 80.

mdw · June 19, 2018, 7:36pm

certbot renew --dry-run --debug
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/cloud.quimpro.com.conf

Cert not due for renewal, but simulating renewal for dry run
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for cloud.quimpro.com
Waiting for verification…
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0065_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0065_csr-certbot.pem

Processing /etc/letsencrypt/renewal/secure.familiedewijn.nl.conf

Cert not due for renewal, but simulating renewal for dry run
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for secure.familiedewijn.nl
Waiting for verification…
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0066_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0066_csr-certbot.pem

Processing /etc/letsencrypt/renewal/crm.markdewijn.com.conf

Cert is due for renewal, auto-renewing…
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for crm.markdewijn.com
Waiting for verification…
Cleaning up challenges
Unable to clean up challenge directory /var/www/suitecrm/.well-known/acme-challenge
Attempting to renew cert from /etc/letsencrypt/renewal/crm.markdewijn.com.conf produced an unexpected error: Failed authorization procedure. crm.markdewijn.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://crm.markdewijn.com/.well-known/acme-challenge/pa6Yp-wpRc1dnZMF-si5LwD99M9R1T6AXbrT9Nmf1bo: "

Processing /etc/letsencrypt/renewal/markdewijn.com.conf

Cert is due for renewal, auto-renewing…
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for markdewijn.com
Waiting for verification…
Cleaning up challenges
Attempting to renew cert from /etc/letsencrypt/renewal/markdewijn.com.conf produced an unexpected error: Failed authorization procedure. markdewijn.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://markdewijn.com/.well-known/acme-challenge/MKqVTY096OHxwcHv_4jt2_uCa5yKnwAhlw7fpz1VTUc: "

404 Not Found

Not Found

Th". Skipping. ** DRY RUN: simulating 'certbot renew' close to cert expiry ** (The test certificates below have not been saved.)

The following certs were successfully renewed:
/etc/letsencrypt/live/cloud.quimpro.com/fullchain.pem (success)
/etc/letsencrypt/live/secure.familiedewijn.nl/fullchain.pem (success)

The following certs could not be renewed:
/etc/letsencrypt/live/crm.markdewijn.com/fullchain.pem (failure)
/etc/letsencrypt/live/markdewijn.com/fullchain.pem (failure)
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates above have not been saved.)
Traceback (most recent call last):
File “/usr/bin/certbot”, line 11, in
load_entry_point(‘certbot==0.10.2’, ‘console_scripts’, ‘certbot’)()
File “/usr/lib/python2.7/dist-packages/certbot/main.py”, line 849, in main
return config.func(config, plugins)
File “/usr/lib/python2.7/dist-packages/certbot/main.py”, line 655, in renew
renewal.handle_renewal_request(config)
File “/usr/lib/python2.7/dist-packages/certbot/renewal.py”, line 430, in handle_renewal_request
len(renew_failures), len(parse_failures)))
Error: 2 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

The following errors were reported by the server:

Domain: crm.markdewijn.com
Type: unauthorized
Detail: Invalid response from
http://crm.markdewijn.com/.well-known/acme-challenge/pa6Yp-wpRc1dnZMF-si5LwD99M9R1T6AXbrT9Nmf1bo:
"
<meta name="viewpo"
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A record(s) for that domain
contain(s) the right IP address.
The following errors were reported by the server:

Domain: markdewijn.com
Type: unauthorized
Detail: Invalid response from
http://markdewijn.com/.well-known/acme-challenge/MKqVTY096OHxwcHv_4jt2_uCa5yKnwAhlw7fpz1VTUc:
"
404 Not Found
Not Found
Th"
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A record(s) for that domain
contain(s) the right IP address.

JuergenAuer · June 19, 2018, 7:49pm

Now http://cloud.quimpro.com/ works and redirects to https.

Now http://crm.markdewijn.com/ works also, with redirect.

Check Certbot again

mdw · June 19, 2018, 7:51pm

same result

certbot renew --dry-run --debug
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/cloud.quimpro.com.conf

Cert not due for renewal, but simulating renewal for dry run
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for cloud.quimpro.com
Waiting for verification…
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0069_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0069_csr-certbot.pem

Processing /etc/letsencrypt/renewal/secure.familiedewijn.nl.conf

Cert not due for renewal, but simulating renewal for dry run
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for secure.familiedewijn.nl
Waiting for verification…
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0070_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0070_csr-certbot.pem

Processing /etc/letsencrypt/renewal/crm.markdewijn.com.conf

Cert is due for renewal, auto-renewing…
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for crm.markdewijn.com
Waiting for verification…
Cleaning up challenges
Unable to clean up challenge directory /var/www/suitecrm/.well-known/acme-challenge
Attempting to renew cert from /etc/letsencrypt/renewal/crm.markdewijn.com.conf produced an unexpected error: Failed authorization procedure. crm.markdewijn.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://crm.markdewijn.com/.well-known/acme-challenge/eOgYcIHdUI769_XA4oCp_u5uPNGwyclPgTN0gJMywdI: "

Processing /etc/letsencrypt/renewal/markdewijn.com.conf

Cert is due for renewal, auto-renewing…
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for markdewijn.com
Waiting for verification…
Cleaning up challenges
Attempting to renew cert from /etc/letsencrypt/renewal/markdewijn.com.conf produced an unexpected error: Failed authorization procedure. markdewijn.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://markdewijn.com/.well-known/acme-challenge/cdjL1Dr0_DbiBMj7SQbkz-otPLDhSZQvJWgTIutPMMg: "

404 Not Found

Not Found

Th". Skipping. ** DRY RUN: simulating 'certbot renew' close to cert expiry ** (The test certificates below have not been saved.)

The following certs were successfully renewed:
/etc/letsencrypt/live/cloud.quimpro.com/fullchain.pem (success)
/etc/letsencrypt/live/secure.familiedewijn.nl/fullchain.pem (success)

The following certs could not be renewed:
/etc/letsencrypt/live/crm.markdewijn.com/fullchain.pem (failure)
/etc/letsencrypt/live/markdewijn.com/fullchain.pem (failure)
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates above have not been saved.)
Traceback (most recent call last):
File “/usr/bin/certbot”, line 11, in
load_entry_point(‘certbot==0.10.2’, ‘console_scripts’, ‘certbot’)()
File “/usr/lib/python2.7/dist-packages/certbot/main.py”, line 849, in main
return config.func(config, plugins)
File “/usr/lib/python2.7/dist-packages/certbot/main.py”, line 655, in renew
renewal.handle_renewal_request(config)
File “/usr/lib/python2.7/dist-packages/certbot/renewal.py”, line 430, in handle_renewal_request
len(renew_failures), len(parse_failures)))
Error: 2 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

The following errors were reported by the server:

Domain: crm.markdewijn.com
Type: unauthorized
Detail: Invalid response from
http://crm.markdewijn.com/.well-known/acme-challenge/eOgYcIHdUI769_XA4oCp_u5uPNGwyclPgTN0gJMywdI:
"
<meta name="viewpo"
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A record(s) for that domain
contain(s) the right IP address.
The following errors were reported by the server:

Domain: markdewijn.com
Type: unauthorized
Detail: Invalid response from
http://markdewijn.com/.well-known/acme-challenge/cdjL1Dr0_DbiBMj7SQbkz-otPLDhSZQvJWgTIutPMMg:
"
404 Not Found
Not Found
Th"
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A record(s) for that domain
contain(s) the right IP address.

JuergenAuer · June 19, 2018, 7:54pm

But now you should fix your ipv6 - error. Remove all ipv6-entries.

mdw · June 19, 2018, 7:55pm

Please explain to me in more detail please.