Please help with renewal failing

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: isagog.com

I ran this command: sudo certbot --apache -d isagog.com

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Requesting a certificate for isagog.com
Performing the following challenges:
http-01 challenge for isagog.com
Waiting for verification...
Challenge failed for domain isagog.com
http-01 challenge for isagog.com
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: isagog.com
   Type:   unauthorized
   Detail: 65.109.97.254: Invalid response from
   http://isagog.com/.well-known/acme-challenge/GkaWR9qHWGo7xRuqARWZ1j9y24n3BNdg2j3jwN9F0BY:
   404

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

Please note that isagog.com does resolve correctly to the correct IP.

My web server is (include version):

Server version: Apache/2.4.56 (Debian)
Server built:   2023-04-02T03:06:01

The operating system my web server runs on is (include version):
Debian 11

My hosting provider, if applicable, is:
Hetzner

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.12.0

The sysadmin that configured our system fell in love and fled with a woman leaving no contacts (not joking, we wish him all the best but he's left us in some trouble)!

The Apache2 config is quite complex with 11 virtual hosts. The certificate renewal ( sudo certbot --apache) works for all of them save for isagog.com

Thanks for any help

1 Like
2

That got my attention :slight_smile:

Usually a 404 error using the --apache plug-in is because of something unusual with the Apache config.

First, your cert history shows you were using a cert with both isagog.com and www.isagog.com in it (which is normal). But, the cert you got recently just has the www name in it. That's a problem. So, best to not use your command below that only gets a cert with one name it.

Second, your root domain isagog.com is not responding to HTTP requests the same as your www domain. This is probably something wrong in your Apache config.

Can you show result of this? I know it will be long but it's the first step to seeing exactly what to look at.

apache2ctl -t -D DUMP_VHOSTS
5 Likes
3

Thank you so much! We're a weird team with myself being a medical doctor and the other team member a philosopher, so now that our tech guy has eloped we're struggling a bit and community support warms our hearts! :slight_smile: Here's what you you asked for:

bob@newisa:~$ sudo apache2ctl -t -D DUMP_VHOSTS
[sudo] password for bob: 
VirtualHost configuration:
*:443                  is a NameVirtualHost
         default server isagog.com (/etc/apache2/sites-enabled/001-isagog.com-ssl.conf:2)
         port 443 namevhost isagog.com (/etc/apache2/sites-enabled/001-isagog.com-ssl.conf:2)
                 alias www.isagog.com
         port 443 namevhost mattermost.isagog.com (/etc/apache2/sites-enabled/002-mattermost.isagog.com-ssl.conf:2)
                 alias www.mattermost.isagog.com
         port 443 namevhost mema.isagog.com (/etc/apache2/sites-enabled/003-mema.isagog.com-ssl.conf:2)
                 alias www.mema.isagog.com
         port 443 namevhost ai.isagog.com (/etc/apache2/sites-enabled/004-ai.isagog.com-ssl.conf:2)
                 alias www.ai.isagog.com
         port 443 namevhost jenkins.isagog.com (/etc/apache2/sites-enabled/005-jenkins.isagog.com-ssl.conf:2)
                 alias www.jenkins.isagog.com
         port 443 namevhost test.mema.isagog.com (/etc/apache2/sites-enabled/006-test.mema.isagog.com-ssl.conf:2)
                 alias www.test.mema.isagog.com
*:80                   is a NameVirtualHost
         default server isagog.com (/etc/apache2/sites-enabled/000-default.conf:1)
         port 80 namevhost isagog.com (/etc/apache2/sites-enabled/000-default.conf:1)
         port 80 namevhost isagog.com (/etc/apache2/sites-enabled/001-isagog.com.conf:1)
                 alias www.isagog.com
         port 80 namevhost mattermost.isagog.com (/etc/apache2/sites-enabled/002-mattermost.isagog.com.conf:1)
                 alias www.mattermost.isagog.com
         port 80 namevhost mema.isagog.com (/etc/apache2/sites-enabled/003-mema.isagog.com.conf:1)
                 alias www.mema.isagog.com
         port 80 namevhost ai.isagog.com (/etc/apache2/sites-enabled/004-ai.isagog.com.conf:1)
                 alias www.ai.isagog.com
         port 80 namevhost jenkins.isagog.com (/etc/apache2/sites-enabled/005-jenkins.isagog.com.conf:1)
                 alias www.jenkins.isagog.com
         port 80 namevhost test.mema.isagog.com (/etc/apache2/sites-enabled/006-test.mema.isagog.com.conf:1)
                 alias www.test.mema.isagog.com
         port 80 namevhost null (/etc/apache2/sites-enabled/999-catchall.conf:1)
                 wild alias *

So I see there are a default and TWO virtual hosts defined for isagog.com and the two Vhosts have two different config files ... this looks like asking for trouble.

Here are the lines active in 000-default.conf:

<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

and here the lines active in 001-isagog.com.conf:

<VirtualHost *:80>
        ServerAdmin thor@gmail.com
        ServerName isagog.com
        ServerAlias www.isagog.com
        Alias /.well-known/acme-challenge/ /var/www/isagog.com/.well-known/acme-challenge/
        RedirectMatch Permanent "^(/(?!\.well-known/).*)" https://isagog.com/$1
        ProxyPreserveHost On
        ErrorLog ${APACHE_LOG_DIR}/isagog-error.log
        CustomLog ${APACHE_LOG_DIR}/isagog-access.log combined
</VirtualHost>
4

I have merged, checked with apache2ctl in 001-isagog.com.conf as follows:

<VirtualHost *:80>
        ServerAdmin thor@gmail.com
        ServerName isagog.com
        ServerAlias www.isagog.com
        DocumentRoot /var/www/html
        Alias /.well-known/acme-challenge/ /var/www/isagog.com/.well-known/acme-challenge/
        RedirectMatch Permanent "^(/(?!\.well-known/).*)" https://isagog.com/$1
        ProxyPreserveHost On
        ErrorLog ${APACHE_LOG_DIR}/isagog-error.log
        CustomLog ${APACHE_LOG_DIR}/isagog-access.log combined
RewriteEngine on
RewriteCond %{SERVER_NAME} =isagog.com [OR]
RewriteCond %{SERVER_NAME} =www.isagog.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

got rid of 000-default.conf and restarted Apache. Web works and certbot --apache -d isagog.com -d www,isagog.com SUCCEEDS !!!

I also seem to understand that the Debian package does install a system service for attempting renewals:

root@newisa /etc/apache2/sites-available # systemctl status certbot.timer
● certbot.timer - Run certbot twice daily
     Loaded: loaded (/lib/systemd/system/certbot.timer; enabled; vendor preset: enabled)
     Active: active (waiting) since Fri 2023-05-19 21:08:36 CEST; 2 months 13 days ago
    Trigger: Wed 2023-08-02 15:55:55 CEST; 6h left
   Triggers: ● certbot.service

which is happily doing its thing.

Not sure I've done the right things but it works :slight_smile: Thanks a lot for pointing me in the right direction. From zero to (wannabe hero) :slight_smile:

1 Like
5

You've done the correct things from the looks of it. The double vhost was indeed something we'd wanted to check with the apache2ctl -t -D DUMP_VHOSTS command (it's a common cause of the challenge not working).

Instead of getting rid of the 000-default.conf, it was also possible to add a ServerName with a different value e.g. localhost or something else non-existent on the public internet. One reason to keep the default conf is that Apache will use the first virtualhost if a HTTP client makes a request for a non-existent hostname. It's common to have the default (first) Apache configuration pointing to an error page or some other simple placeholder, so that incorrect requests don't end up at your websites virtualhost.

Anyway, glad you fixed it! Greetings from a fellow M.D. :wink:

4 Likes
6

Thanks a lot my virtual friend and colleague :slight_smile: Are you involved in the OsirisX DICOM viewer? Take care.

2 Likes
7

Nope, Osiris is a fairly common nickname, the ancient Egyptian god of the underworld. Not sure how that name creeps up in medical devices/software so often :grimacing:

3 Likes
8

Well for our radiologist friends, living most of their life in darkness, it's either Osiris or Ades :wink:
Doubly so for the radiologists: " The name Osiris is boy's name of Egyptian origin meaning "with strong eyesight".

Take care

2 Likes
9

Nice work getting things running. You should probably tidy up your cert renewals though.

Run sudo certbot certificates for a list of all active cert profiles. The scheduled renew will attempt renewal of each of these. You should only renew needed certs.

You may see several with domain names related to isagog.com.

  1. Just isagog.com issued Apr27
  2. Just www.isagog.com issued Jul30
  3. One or two with both names both issued Aug2. These probably use the same cert profile. Your Apache is using the most recently issued one with these names

You should delete the unused cert(s) with

sudo certbot delete --cert-name X

where X is shown in the certificates command

If you want help just show result of certificates command and show just isagog related info. Cheers

4 Likes
10 
sudo certbot certificates Found the following certs:
 ...
  Certificate Name: isagog.com
    Serial Number: 3e667be31ec9f35c21656e6d40fc3a453a3
    Key Type: RSA
    Domains: isagog.com www.isagog.com
    Expiry Date: 2023-10-31 06:05:36+00:00 (VALID: 89 days)
    Certificate Path: /etc/letsencrypt/live/isagog.com/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/isagog.com/privkey.pem
...
  Certificate Name: www.isagog.com
    Serial Number: 35338de3769450441a2ad58719b93221f5f
    Key Type: RSA
    Domains: www.isagog.com
    Expiry Date: 2023-10-28 06:31:57+00:00 (VALID: 86 days)
    Certificate Path: /etc/letsencrypt/live/www.isagog.com/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/www.isagog.com/privkey.pem

this is what I see. Thanks a lot.

1 Like
11

You would only need this if you still have something wrong in your Apache. And, we will check that below. It only has the www domain in it and normal requests to your domain use the first cert shown expiring Oct31 6:05:36 (see link like this)

You would delete it with

sudo certbot delete --cert-name www.isagog.com

To be extra safe you can scan your Apache configs to make sure that cert path `.../www.isagog.com/...' is not referenced. This should not show anything

sudo grep -R live/www.isagog.com /etc/apache2

While you should see your active cert in your Apache with this

sudo grep -R live/isagog.com /etc/apache2
4 Likes
12

And if the certificate is also used in other services besides Apache (e.g. mailservers), then you should check that service too.

2 Likes
13

Grazie mille !!! Your greps turned out just as you expected and no I am not using certs for anything else therefore I deleted the www.isagog.com cert as you advised!

4 Likes
14

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.