Unable to Renew Root Domain (Subdomains Renew Fine)

Like the title says, I'm able to renew certs for my subdomains but not the root. Any help would be appreciated.

My domain is: https://jns.enterprises

I ran this command: sudo certbot renew --dry-run

It produced this output:
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator webroot, Installer None
Pre-hook command already run, skipping: service apache2 stop
Simulating renewal of an existing certificate for jns.enterprises
Performing the following challenges:
http-01 challenge for jns.enterprises
Waiting for verification...
Challenge failed for domain jns.enterprises
http-01 challenge for jns.enterprises
Cleaning up challenges
Failed to renew certificate jns.enterprises with error: Some challenges have failed.

The following errors were reported by the server:

Domain: jns.enterprises
Type: connection
Detail: 136.XXX.XXX.191: Fetching
http://jns.enterprises/.well-known/acme-challenge/FC2HJ-j6N3MZu6PJpN2mvangQcIxJG8JmzuzI2JGsxc:
Connection refused

My web server is (include version): Apache/2.4.56 (Raspbian)

The operating system my web server runs on is (include version): Raspbian 11 (Bullseye)

My hosting provider, if applicable, is: N/A

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 1.12.0

Your redirects need a little maintenance:

% curl -IL http://jns.enterprises/.well-known/acme-challenge/FC2HJ-j6N3MZu6PJpN2mvangQcIxJG8JmzuzI2JGsxc
HTTP/1.1 302 Found
Date: Thu, 07 Sep 2023 18:24:04 GMT
Server: Apache
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=63072000; includeSubdomains
X-Content-Type-Options: nosniff
Location: https://jns.enterprises/.well-known/acme-challenge/FC2HJ-j6N3MZu6PJpN2mvangQcIxJG8JmzuzI2JGsxc
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 404 Not Found
Date: Thu, 07 Sep 2023 18:24:05 GMT
Server: Apache
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=63072000; includeSubdomains
X-Content-Type-Options: nosniff
Content-Type: text/html; charset=iso-8859-1

% curl -IL http://football.jns.enterprises/.well-known/acme-challenge/FC2HJ-j6N3MZu6PJpN2mvangQcIxJG8JmzuzI2JGsxc
HTTP/1.1 404 Not Found
Date: Thu, 07 Sep 2023 18:24:20 GMT
Server: Apache
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=63072000; includeSubdomains
X-Content-Type-Options: nosniff
Content-Type: text/html; charset=iso-8859-1

% curl -IL http://webmail.jns.enterprises/.well-known/acme-challenge/FC2HJ-j6N3MZu6PJpN2mvangQcIxJG8JmzuzI2JGsxc
HTTP/1.1 302 Found
Date: Thu, 07 Sep 2023 18:24:36 GMT
Server: Apache
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=63072000; includeSubdomains
X-Content-Type-Options: nosniff
Location: https://webmail.jns.enterprises.well-known/acme-challenge/FC2HJ-j6N3MZu6PJpN2mvangQcIxJG8JmzuzI2JGsxc
Content-Type: text/html; charset=iso-8859-1

curl: (6) Could not resolve host: webmail.jns.enterprises.well-known
                                                                                     

but this has nothing to do with the error you get here. Check if your firewall is blocking some IP subnets.

3 Likes
2 Likes

I completely disabled the firewall and retested; same errors.

And the subdomains work still?

3 Likes

Yes. I got thinking about what you said about the redirects, and remembered I added this to my Apache config file back when I was getting everything set up - could it be an issue?

#Used for LetsEncrypt validation
AccessFileName .htaccess
AliasMatch ^/.well-known/acme-challenge/(.*)$ /var/www/html/.well-known/acme-challenge/$1
Alias /.well-known/acme-challenge /var/www/html/.well-known/acme-challenge/


<Directory /var/www/html/.well-known/acme-challenge>
        Order allow,deny
        Allow from all
        Options None
        AllowOverride None
        ForceType text/plain
        RedirectMatch 404 "^(?!/\.well-known/acme-challenge/[\w-]{43}$)"
</Directory>

Did the pre-hook stop Apache? Because with webroot you should leave it running

Are you using a different hook (or no hook) when you renew the cert with 3 domain names?

And, since you have a cert with the root in it (and the other 2) do you even need a separate one?

3 Likes

That could interfere but I don't know what RedirectMatch does.

Anyway you shouldn't need that. Certbot should handle all of that if you use the Apache plugin.

3 Likes

Yes, the pre-hook stopped Apache.

Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator standalone, Installer None
Running pre-hook command: service apache2 stop
Simulating renewal of an existing certificate for football.jns.enterprises
Performing the following challenges:
http-01 challenge for football.jns.enterprises
Waiting for verification...
Cleaning up challenges

Where did you get that list? There should actually be four domains on the certificate, not three. I probably don't need a separate cert, but the way the renewal is erroring out, in the terminal it doesn't look like the cert is being renewed for the root domain.

1 Like

No, your football domain has only ever been on its own cert. You can see that at https://crt.sh just know that crt.sh system is often badly delayed recently but it is still good for long-term history.

I was using the CT log system at another system (censys.io) but that requires an account. There are other CT log display providers (search google).

What does this show

sudo certbot certificates

UPDATE:
Oh, this is actually a very nice CT Search tool as long as your certs are only from Let's Encrypt
https://tools.letsdebug.net/cert-search?m=domain&q=football.jns.enterprises&d=2160

3 Likes
Certificate Name: football.jns.enterprises
    Serial Number: 3a26f6569e7bcf000883d2d2acd847656fc
    Key Type: RSA
    Domains: football.jns.enterprises
    Expiry Date: 2023-12-05 19:33:14+00:00 (VALID: 88 days)
    Certificate Path: /etc/letsencrypt/live/football.jns.enterprises/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/football.jns.enterprises/privkey.pem
  Certificate Name: jns.enterprises
    Serial Number: 40b212130f9d646746e75016b3a153f5045
    Key Type: RSA
    Domains: jns.enterprises
    Expiry Date: 2023-10-13 03:47:26+00:00 (VALID: 35 days)
    Certificate Path: /etc/letsencrypt/live/jns.enterprises/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/jns.enterprises/privkey.pem
  Certificate Name: recipes.jns.enterprises
    Serial Number: 42d42818ec779f5801a73bd3f30dc2c7703
    Key Type: RSA
    Domains: webmail.jns.enterprises jns.enterprises recipes.jns.enterprises
    Expiry Date: 2023-12-05 19:33:28+00:00 (VALID: 88 days)
    Certificate Path: /etc/letsencrypt/live/recipes.jns.enterprises/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/recipes.jns.enterprises/privkey.pem
  Certificate Name: webmail.jns.enterprises
    Serial Number: 3a5df0b2547090297b4a6d6db622ee5c400
    Key Type: RSA

Looks like you are correct - I didn't realize I had so many. Back when I was setting things up I knew even less than I do now (which isn't saying much), so looks like I duplicated some efforts. As far as cleaning this mess up...is there any benefit to separate certs vs all on one?

The best method is one that works well for you. I prefer to have one cert for each unique VirtualHost. For me that is easiest to manage.

Example, if example.com and www.example.com were in same VirtualHost (with one as a ServerAlias) they would be together on one cert.

I think your Certbot list is not complete. There should have been one with several domain names.

And, I think the methods you have used for some of them differ from the others. Once you have chosen how you want to proceed we can help straighten out the renewal methods.

UPDATE:
Oops, my bad. I just saw (properly) that one of those certs was for 3 domains. Apart from that the rest of my comment stands.

4 Likes

I like the VirtualHost suggestion. To go that route:

  1. To me, it looks like football and webmail are fine (added full text of webmail below, noticed it got cut off in the previous copy and paste).
  2. Looks like I would need to revoke and reissue the root and recipes certificates? I can look up the syntax for that fairly easily I think.
  3. Should I be concerned that the root didn't renew, but the recipe certificate (with the root on it) did? Not sure why one would work but not the other. Could it be because of the multiple domains? If so, I could add a www to the root to just make it work.
  Certificate Name: webmail.jns.enterprises
    Serial Number: 3a5df0b2547090297b4a6d6db622ee5c400
    Key Type: RSA
    Domains: webmail.jns.enterprises
    Expiry Date: 2023-12-05 19:33:31+00:00 (VALID: 88 days)
    Certificate Path: /etc/letsencrypt/live/webmail.jns.enterprises/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/webmail.jns.enterprises/privkey.pem

Thank you again for your assistance!

1 Like

Please don't revoke any certs for this reason.

4 Likes

It still has a few days left before it hits 30 days and would renew:

3 Likes

Hadn't we established that these lines during the dry run were bad, specifically the "challenge failed" piece?

Waiting for verification...
Challenge failed for domain jns.enterprises
http-01 challenge for jns.enterprises
Cleaning up challenges
Failed to renew certificate jns.enterprises with error: Some challenges have fai                        led.

Yes, should be concerned. I don't think it is related to the domain name but instead the method used in the renewal conf file for the respective certs.

Can you show the two renewal conf files

/etc/letsencrypt/renewal/jns.enterprises.conf
/etc/letsencrypt/renewal/recipes.jns.enterprises.conf

Just to be clear ... so at the end do you want 4 certs one for each of your domain names?

If so, that means we just need to fix the jns root renewal and then "shrink" recipes to just its own name.

Is that right?

3 Likes

All of your assumptions are correct, and looks like you nailed it on authentication method - webroot vs standalone. If i just change the renewal params to match those in recipe, should that fix the issue? Assuming it does, how do I shrink recipes?

jns.enterprises

# renew_before_expiry = 30 days
version = 1.12.0
archive_dir = /etc/letsencrypt/archive/jns.enterprises
cert = /etc/letsencrypt/live/jns.enterprises/cert.pem
privkey = /etc/letsencrypt/live/jns.enterprises/privkey.pem
chain = /etc/letsencrypt/live/jns.enterprises/chain.pem
fullchain = /etc/letsencrypt/live/jns.enterprises/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = webroot
account = cca51956f8d5f563f19f06abfdfc55fc
server = https://acme-v02.api.letsencrypt.org/directory
post_hook = service apache2 start
pre_hook = service apache2 stop
pref_challs = http-01,
[[webroot_map]]
jns.enterprises = /var/www/html

recipes


version = 1.12.0
archive_dir = /etc/letsencrypt/archive/recipes.jns.enterprises
cert = /etc/letsencrypt/live/recipes.jns.enterprises/cert.pem
privkey = /etc/letsencrypt/live/recipes.jns.enterprises/privkey.pem
chain = /etc/letsencrypt/live/recipes.jns.enterprises/chain.pem
fullchain = /etc/letsencrypt/live/recipes.jns.enterprises/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = standalone
account = cca51956f8d5f563f19f06abfdfc55fc
post_hook = service apache2 start
pre_hook = service apache2 stop
server = https://acme-v02.api.letsencrypt.org/directory
pref_challs = http-01,
1 Like

That's not the best approach. The webroot is much better than standalone when you have a working web server (which you do).

We normally caution against modifying the renewal conf files by hand but in this case you could make a backup of the jns.enterprises.conf file and then remove the 2 lines for the pre_hook and post_hook.

Then, test with this

sudo certbot renew --cert-name jns.enterprises --dry-run

If the dry-run works just remove --dry-run to get the production cert. No other changes are needed and it should auto-renew going forward

Shrinking recipes is entirely different. I'll need a minute but post back any results from above. Update: After further study I would like to see results of above before working on recipes.

3 Likes

Is that the correct webroot path?

2 Likes