Certbot renew fails for primary domain but succeeds for sub-domain

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domains are: coda-usa.com, www.coda-usa.com, cfs.coda-usa.com

I ran this command: certbot renew --dry-run

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/cfs.coda-usa.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Simulating renewal of an existing certificate for cfs.coda-usa.com

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/coda-usa.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Simulating renewal of an existing certificate for coda-usa.com and www.coda-usa.com

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
  Domain: coda-usa.com
  Type:   unauthorized
  Detail: 3.239.174.56: Invalid response from https://www.coda-usa.com: "<!DOCTYPE html>\n<html lang=\"en-US\">\n<head>\n    <meta charset=\"utf-8\">\n    <meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge\">\n"

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

Failed to renew certificate coda-usa.com with error: Some challenges have failed.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
The following simulated renewals succeeded:
  /etc/letsencrypt/live/cfs.coda-usa.com/fullchain.pem (success)

The following simulated renewals failed:
  /etc/letsencrypt/live/coda-usa.com/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is (include version): nginx version: nginx/1.18.0 (Ubuntu)

The operating system my web server runs on is (include version): Ubuntu 22.04.4 LTS

My hosting provider, if applicable, is: Amazon US-EAST

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.21.0

CURL test: cfs.coda-usa.com

# curl -I https://cfs.coda-usa.com/.well-known/acme-challenge/Challenge-Test-123 HTTP/1.1 200 OK Server: nginx/1.18.0 (Ubuntu) Date: Thu, 25 Jul 2024 14:09:04 GMT Content-Type: text/html Content-Length: 445 Last-Modified: Wed, 17 Jul 2024 19:24:43 GMT Connection: keep-alive ETag: "66981a7b-1bd" Content-Security-Policy: font-src 'self' https://use.typekit.net/af/ https://use.typekit.net/xcy8qgg.css/; frame-ancestors 'self'; frame-src 'self'; style-src 'self' 'unsafe-inline' https://use.typekit.net/xcy8qgg.css/; style-src-elem 'self' 'unsafe-inline' https://use.typekit.net/xcy8qgg.css/; Accept-Ranges: bytes

CURL test: www.coda-usa.com

# curl -I https://www.coda-usa.com/.well-known/acme-challenge/Challenge-Test-123 HTTP/1.1 404 Not Found Server: nginx/1.18.0 (Ubuntu) Date: Thu, 25 Jul 2024 14:10:44 GMT Content-Type: text/html; charset=utf-8 Connection: keep-alive Set-Cookie: CODASESSIDs=quac71f1h8ke5n1ugerjrqubfh; path=/; secure; HttpOnly; SameSite=Lax Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate Pragma: no-cache X-Powered-By: ProcessWire CMS

CURL test: coda-usa.com

# curl -I https://coda-usa.com/.well-known/acme-challenge/Challenge-Test-123 HTTP/2 301 date: Thu, 25 Jul 2024 14:15:58 GMT content-type: text/html content-length: 167 location: https://www.coda-usa.com cache-control: max-age=3600 expires: Thu, 25 Jul 2024 15:15:58 GMT report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=M6duV6Z0XxPFr8OVS41eSBa00moYIcdklYvB6avr42jureY07PlxmhJhi%2F%2F%2Bc0poAcfTuxCm6Fw1Ir%2BHzhH9Vw9hmGd51wbZs3lhgpVRh2ycGStnjEeS45lAl3DnxC0%3D"}],"group":"cf-nel","max_age":604800} nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800} server: cloudflare cf-ray: 8a8cc3dd29fa9c1f-IAD

I'm not sure what else to check now. Any assistance would be greatly appreciated.

1 Like

Well, there are several things we could check but the first is your Cloudflare redirect for coda-usa.com is not right.

The Let's Encrypt server sends a request like below to validate your domain. But, notice the Cloudflare edge redirects to your www subdomain but loses the entire URI. This results in the LE server seeing your "home" page and not the correct challenge response.

curl -i http://coda-usa.com/.well-known/acme-challenge/Test404
HTTP/1.1 301 Moved Permanently
Location: https://www.coda-usa.com
Server: cloudflare

Your www domain also redirects but maintains the URI which is why it is not showing in the errors. And, the redirect is done by your nginx rather than the Cloudflare edge.

curl -i http://www.coda-usa.com/.well-known/acme-challenge/Test404
HTTP/1.1 301 Moved Permanently
Server: nginx/1.18.0 (Ubuntu)
Location: https://www.coda-usa.com/.well-known/acme-challenge/Test404

Perhaps review this topic for advice on Cloudflare redirects

5 Likes

Thank you at @MikeMcQ. That pointed me in the proper direction.

For those who find this post later in life....

You MUST disable PROXY STATUS in your Cloudflare DNS configuration:

1 Like

That is NOT true. You can choose to do that but you can get certs for origin servers with the domain proxied.

3 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.