Certbot renew returns 404

jodi · August 23, 2020, 1:23pm

Hello!
I want to renew my certificate.

I ran this command: sudo certbot renew --dry-run

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/anjoapp.duckdns.org.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator nginx, Installer nginx
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for anjoapp.duckdns.org
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (anjoapp.duckdns.org) from /etc/letsencrypt/renewal/anjoapp.duckdns.org.conf produced an unexpected error: Failed authorization procedure. anjoapp.duckdns.org (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://anjoapp.duckdns.org/.well-known/acme-challenge/V6Qcnp_j8JYG5yybLLNvuaVGkiINSvZ4Vwb6rm4NHZM [77.185.114.151]: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>". Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/anjoapp.duckdns.org/fullchain.pem (failure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/anjoapp.duckdns.org/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: anjoapp.duckdns.org
   Type:   unauthorized
   Detail: Invalid response from
   http://anjoapp.duckdns.org/.well-known/acme-challenge/V6Qcnp_j8JYG5yybLLNvuaVGkiINSvZ4Vwb6rm4NHZM
   [77.185.114.151]: "<html>\r\n<head><title>404 Not
   Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404
   Not Found</h1></center>\r\n<hr><center>"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

My web server is: nginx/1.14.2

The operating system my web server runs on is: Raspberry Pi (Raspbian GNU/Linux 10 (buster))

I can login to a root shell on my machine: yes

I'm using a control panel to manage my site: no

The version of my client is: certbot 0.31.0

In nginx I have the following block for listening to port 80:

location ~ ^/.well-known/acme-challenge/ {
root /var/www/anjoapp.duckdns.org;
}

I have a testfile (contains only "thats only a test" ) in the directory

/var/www/anjoapp.duckdns.org/.well-known/acme-challenge

which can be accessed by

http://anjoapp.duckdns.org/.well-known/acme-challenge/testfile

So I think it's not a firewall problem.
Where does certbot store the temporary files (tokens)? Is it really /var/www/anjoapp.duckdns.org/.well-known/acme-challenge ?

Because nginx error log says "no such file or directory":

2020/08/23 14:25:32 [error] 1832#1832: *51 open() "/var/www/anjoapp.duckdns.org/.well-known/acme-challenge/V6Qcnp_j8JYG5yybLLNvuaVGkiINSvZ4Vwb6rm4NHZM" failed (2: No such file or directory), client: 34.211.60.134, server: anjoapp.duckdns.org, request: "GET /.well-known/acme-challenge/V6Qcnp_j8JYG5yybLLNvuaVGkiINSvZ4Vwb6rm4NHZM HTTP/1.1", host: "anjoapp.duckdns.org"

Please help! Thank you!

stevenzhu · August 23, 2020, 1:34pm

Hi,

Can you share the renewal attempts log in /var/log/letsencrypt/letsencrypt.log? Only this attempt is needed.
By the way, I think you’ll need the webroot authenticator instead of Nginx, since I think Nginx authenticator actually modify your Nginx configuration instead of placing the file, and might cause issue since you already have a block for acme-challenge specified.

jodi · August 23, 2020, 1:59pm

Thanks for responding!
As a new user I can not do some attachment. So I post the last lines of the log.

2020-08-23 15:48:25,504:INFO:certbot.auth_handler:Waiting for verification...
2020-08-23 15:48:25,506:DEBUG:acme.client:JWS payload:
b'{\n "resource": "challenge",\n "type": "http-01"\n}'
2020-08-23 15:48:25,533:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/98780969/_ei-Hw:
{
"protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8xMzk2NDIzMSIsICJub25jZSI6ICIwMDAyQjhBTGhSaFBXdHg4OUVHb2N5a01KZGp0eGllUGRCNnlkSDVmb05sbGJWZyIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9jaGFsbC12My85ODc4MDk2OS9fZWktSHcifQ",
"signature": "PdqPvMEkoXcvHErW0m_S7LzhxcELhenCwLdww2D31y44wnWmmi_u3rJvRkL3M1lMFkopObT9VXj8Gktha_QWgLGYeGSBOnhN2BjNeGX4-aD-JWgQ7u8s9mcwEGyr_0vsPGBA5xkMGckSoBRbTA4ddEbtwlYS7yfz6WyDE5gC_f3WyAw9mQ4tMbq1fBwe4iYciyCuQkVv6xr_dviLqo2-db9DD1Ki7lAKixju-nTFlTQZrnJh3j8_sJSfkxaqrcKswOlaN2zmNQKAKEA4oDu0mzO5T6J0L5kIWVQe5JxXeWo2yQJKx3Pl_zJeJZ9S5Vx5fZBHnvlcKIV5bOj5j8j5eg",
"payload": "ewogICJyZXNvdXJjZSI6ICJjaGFsbGVuZ2UiLAogICJ0eXBlIjogImh0dHAtMDEiCn0"
}
2020-08-23 15:48:25,701:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/chall-v3/98780969/_ei-Hw HTTP/1.1" 200 191
2020-08-23 15:48:25,704:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Sun, 23 Aug 2020 13:48:25 GMT
Content-Type: application/json
Content-Length: 191
Connection: keep-alive
Boulder-Requester: 13964231
Cache-Control: public, max-age=0, no-cache
Link: https://acme-staging-v02.api.letsencrypt.org/directory;rel="index", https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/98780969;rel="up"
Location: https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/98780969/_ei-Hw
Replay-Nonce: 0002hjjsHY9vUCUgeforNGHDm5fQRXSVuV5vtVwR87gPqjQ
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
"type": "http-01",
"status": "pending",
"url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/98780969/_ei-Hw",
"token": "7CFOT7CGfeGSUnd4PK7wxdm4rM3b6miXC9F-Y4QP35Y"
}
2020-08-23 15:48:25,705:DEBUG:acme.client:Storing nonce: 0002hjjsHY9vUCUgeforNGHDm5fQRXSVuV5vtVwR87gPqjQ
2020-08-23 15:48:28,711:DEBUG:acme.client:JWS payload:
b''
2020-08-23 15:48:28,736:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/98780969:
{
"protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8xMzk2NDIzMSIsICJub25jZSI6ICIwMDAyaGpqc0hZOXZVQ1VnZWZvck5HSERtNWZRUlhTVnVWNXZ0VndSODdnUHFqUSIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hdXRoei12My85ODc4MDk2OSJ9",
"signature": "lH2PqNm-xiK7pB08-pZaIhKDbdhtdiPRBosIpDXb1vfrWAB6e-MwrECEE8B05Ko7qDsWOmV38vrIZeNw2p2Y2Fs0qc38F9mvTU7J8CEGOacoLmWLH2Dv-lmOcC-BJScbFiPiy4_t1OIJUzeZIKzzk_6lcO0t69GNhGdUt1tne_FY73KhGQ_EbMBv_-KYltif1-ixrk3CoOzrNV_UZT7Vv8wWnZEvdHNqghbEwdzdN_6KxBlUX573gqIkLIxVPCg8pbT5CYHOcq339W7tG-Lxdqncen0t_iEWQ-Rpob-kbzJEU64pVdq_i99KENejZ1QcPrrQx9htVYSbySNKIoFQ",
"payload": ""
}
2020-08-23 15:48:28,899:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/98780969 HTTP/1.1" 200 1280
2020-08-23 15:48:28,902:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Sun, 23 Aug 2020 13:48:28 GMT
Content-Type: application/json
Content-Length: 1280
Connection: keep-alive
Boulder-Requester: 13964231
Cache-Control: public, max-age=0, no-cache
Link: https://acme-staging-v02.api.letsencrypt.org/directory;rel="index"
Replay-Nonce: 0001WiXl3n-gb-mNnfo03SKoFrI9_royKYcpBTnFkzSymg0
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
"identifier": {
"type": "dns",
"value": "anjoapp.duckdns.org"
},
"status": "invalid",
"expires": "2020-08-30T13:48:24Z",
"challenges": [
{
"type": "http-01",
"status": "invalid",
"error": {
"type": "urn:ietf:params:acme:error:unauthorized",
"detail": "Invalid response from http://anjoapp.duckdns.org/.well-known/acme-challenge/7CFOT7CGfeGSUnd4PK7wxdm4rM3b6miXC9F-Y4QP35Y [77.185.114.151]: "\u003chtml\u003e\r\n\u003chead\u003e\u003ctitle\u003e404 Not Found\u003c/title\u003e\u003c/head\u003e\r\n\u003cbody bgcolor=\"white\"\u003e\r\n\u003ccenter\u003e\u003ch1\u003e404 Not Found\u003c/h1\u003e\u003c/center\u003e\r\n\u003chr\u003e\u003ccenter\u003e"",
"status": 403
},
"url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/98780969/_ei-Hw",
"token": "7CFOT7CGfeGSUnd4PK7wxdm4rM3b6miXC9F-Y4QP35Y",
"validationRecord": [
{
"url": "http://anjoapp.duckdns.org/.well-known/acme-challenge/7CFOT7CGfeGSUnd4PK7wxdm4rM3b6miXC9F-Y4QP35Y",
"hostname": "anjoapp.duckdns.org",
"port": "80",
"addressesResolved": [
"77.185.114.151"
],
"addressUsed": "77.185.114.151"
}
]
}
]
}
2020-08-23 15:48:28,904:DEBUG:acme.client:Storing nonce: 0001WiXl3n-gb-mNnfo03SKoFrI9_royKYcpBTnFkzSymg0
2020-08-23 15:48:28,909:DEBUG:certbot.reporter:Reporting to user: The following errors were reported by the server:

Domain: anjoapp.duckdns.org
Type: unauthorized
Detail: Invalid response from http://anjoapp.duckdns.org/.well-known/acme-challenge/7CFOT7CGfeGSUnd4PK7wxdm4rM3b6miXC9F-Y4QP35Y [77.185.114.151]: "\r\n404 Not Found\r\n<body bgcolor="white">\r\n

404 Not Found

\r\n

"

To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address.
2020-08-23 15:48:28,912:DEBUG:certbot.error_handler:Encountered exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 82, in handle_authorizations
self._respond(aauthzrs, resp, best_effort)
File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 168, in _respond
self._poll_challenges(aauthzrs, chall_update, best_effort)
File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 239, in _poll_challenges
raise errors.FailedChallenges(all_failed_achalls)
certbot.errors.FailedChallenges: Failed authorization procedure. anjoapp.duckdns.org (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://anjoapp.duckdns.org/.well-known/acme-challenge/7CFOT7CGfeGSUnd4PK7wxdm4rM3b6miXC9F-Y4QP35Y [77.185.114.151]: "\r\n404 Not Found\r\n<body bgcolor="white">\r\n

404 Not Found

\r\n

"

2020-08-23 15:48:28,913:DEBUG:certbot.error_handler:Calling registered functions
2020-08-23 15:48:28,914:INFO:certbot.auth_handler:Cleaning up challenges
2020-08-23 15:48:30,560:WARNING:certbot.renewal:Attempting to renew cert (anjoapp.duckdns.org) from /etc/letsencrypt/renewal/anjoapp.duckdns.org.conf produced an unexpected error: Failed authorization procedure. anjoapp.duckdns.org (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://anjoapp.duckdns.org/.well-known/acme-challenge/7CFOT7CGfeGSUnd4PK7wxdm4rM3b6miXC9F-Y4QP35Y [77.185.114.151]: "\r\n404 Not Found\r\n<body bgcolor="white">\r\n

404 Not Found

\r\n

". Skipping.
2020-08-23 15:48:30,569:DEBUG:certbot.renewal:Traceback was:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/certbot/renewal.py", line 452, in handle_renewal_request
main.renew_cert(lineage_config, plugins, renewal_candidate)
File "/usr/lib/python3/dist-packages/certbot/main.py", line 1193, in renew_cert
renewed_lineage = _get_and_save_cert(le_client, config, lineage=lineage)
File "/usr/lib/python3/dist-packages/certbot/main.py", line 116, in _get_and_save_cert
renewal.renew_cert(config, domains, le_client, lineage)
File "/usr/lib/python3/dist-packages/certbot/renewal.py", line 310, in renew_cert
new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
File "/usr/lib/python3/dist-packages/certbot/client.py", line 353, in obtain_certificate
orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
File "/usr/lib/python3/dist-packages/certbot/client.py", line 389, in _get_order_and_authorizations
authzr = self.auth_handler.handle_authorizations(orderr, best_effort)
File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 82, in handle_authorizations
self._respond(aauthzrs, resp, best_effort)
File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 168, in _respond
self._poll_challenges(aauthzrs, chall_update, best_effort)
File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 239, in _poll_challenges
raise errors.FailedChallenges(all_failed_achalls)
certbot.errors.FailedChallenges: Failed authorization procedure. anjoapp.duckdns.org (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://anjoapp.duckdns.org/.well-known/acme-challenge/7CFOT7CGfeGSUnd4PK7wxdm4rM3b6miXC9F-Y4QP35Y [77.185.114.151]: "\r\n404 Not Found\r\n<body bgcolor="white">\r\n

404 Not Found

\r\n

"

2020-08-23 15:48:30,571:ERROR:certbot.renewal:All renewal attempts failed. The following certs could not be renewed:
2020-08-23 15:48:30,572:ERROR:certbot.renewal: /etc/letsencrypt/live/anjoapp.duckdns.org/fullchain.pem (failure)
2020-08-23 15:48:30,573:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
File "/usr/bin/certbot", line 11, in
load_entry_point('certbot==0.31.0', 'console_scripts', 'certbot')()
File "/usr/lib/python3/dist-packages/certbot/main.py", line 1365, in main
return config.func(config, plugins)
File "/usr/lib/python3/dist-packages/certbot/main.py", line 1272, in renew
renewal.handle_renewal_request(config)
File "/usr/lib/python3/dist-packages/certbot/renewal.py", line 477, in handle_renewal_request
len(renew_failures), len(parse_failures)))
certbot.errors.Error: 1 renew failure(s), 0 parse failure(s)

stevenzhu · August 23, 2020, 2:01pm

I think you were trying to respond to me but for some reason that reply is eaten by auto-mod ( I do saw that pop-up)

Can you try to paste the log in a pastebin.com or gist.github.com link? It will be much more helpful and easier to delete (by you) later.

jodi · August 23, 2020, 2:16pm

Yes. Thank you for responding!
please look: https://pastebin.com/V7bP4nwE
I can not do any attachment because I am a new user. And I think my post was to big...

What have I do for webroot authenticator?

sudo certbot --authenticator webroot --webroot-path /usr/share/nginx/letsencrypt -d anjoapp.duckdns.org

throws also an error look at https://pastebin.com/Dar2aJzQ

stevenzhu · August 23, 2020, 2:45pm

Try this: sudo certbot renew --dry-run --cert-name anjoapp.duckdns.org --authenticator webroot --webroot-path /var/www/anjoapp.duckdns.org

jodi · August 23, 2020, 2:58pm

Thank you very much! You saved my life

It worked!!

But what about the automatic renewal? Do I always have to renew this way?

stevenzhu · August 23, 2020, 2:59pm

No, once you've set it it should be saved and used automatically.

So if you set a cron or timer, the auto-renew will use the same configuration as now.

jodi · August 23, 2020, 3:02pm

Ok thanks a lot!
I’m really happy!

system · September 22, 2020, 3:03pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.