Can not renew my LE certificate! Why?

Since yesterday my certificate is void. I can not access my website anymore, which includes my NextCloud.
I don't know why, 'cause nothing has changed on my web server. It's been running since 2018 without any cert problems & I never had to renew the certificate since then as it's a regular cron job running ever 20 days.

What can I do?


My domain is: www.woestmeyer.com

I ran this commands:

  1. sudo certbot certonly -a manual -d www.woestmeyer.com -d woestmeyer.com;
  2. certbot certonly --force-renew -d woestmeyer.com;
  3. sudo certbot --apache -d www.woestmeyer.com

It produced this output:

root@NextCloud:~# sudo certbot --apache -d www.woestmeyer.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Renewing an existing certificate for www.woestmeyer.com

Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
Domain: www.woestmeyer.com
Type: connection
Detail: 84.135.30.222: Fetching http://www.woestmeyer.com/.well-known/acme-challenge/8FlEUviywVJ9LYHgGv-_yrByKDvHsnLHu3rn0Bf9Ixs: Timeout during connect (likely firewall problem)

Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is (include version):
Apache2 (2.4.52)

The operating system my web server runs on is (include version):
Ubuntu 20.04 LTS

My hosting provider, if applicable, is:
Selfhosted.

I can login to a root shell on my machine (yes or no, or I don't know):
Yes!

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No.

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.21.0

Because your website isn't reachable on HTTP (port 80). Please open port 80 (HTTP) as recommended:

5 Likes

Hi Osiris,

thank you for the quick reply.
How or where do I open port 80 on my Ubuntu / NextCloud / Apache2? Where're the HSTS defined?

1 Like

I do not know, Google probably knows.

You can check your Apache virtualhosts with the command:

sudo apachectl -t -D DUMP_VHOSTS
3 Likes

You say you are self-hosted so do you have a router? If so, check that too.

Focus on any changes since Feb4 when you last renewed your cert. Something has changed since then. Hopefully your ISP has not blocked port 80 which is something some residential ISP's do.

3 Likes

root@NextCloud:~$ sudo nano /etc/apache2/sites-enabled/000-default-le-ssl.conf
GNU nano 6.2 /etc/apache2/sites-enabled/000-default-le-ssl.conf

<IfModule mod_ssl.c>
<VirtualHost *:443>
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        #ServerName www.example.com

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html

        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

What does that mean? Do I have to add here something?

Start with the output of:

3 Likes

root@NextCloud:~$ sudo apachectl -t -D DUMP_VHOSTS
VirtualHost configuration:
*:443 www.woestmeyer.com (/etc/apache2/sites-enabled/000-default-le-ssl.conf:2)
*:80 Unix-NextCloud-Rechner.fritz.box (/etc/apache2/sites-enabled/000-default.conf:1)

Content still like above.

I can reach the site via HTTP [TCP port 80].

curl -Ii http://www.woestmeyer.com/
HTTP/1.1 301 Moved Permanently
Date: Sat, 06 May 2023 18:08:14 GMT
Server: Apache/2.4.52 (Ubuntu)
Strict-Transport-Security: max-age=15768000; preload
Location: https://www.woestmeyer.com/
Content-Type: text/html; charset=iso-8859-1

But there is no HTTP vhost to serve the two names you are requesting:

What happened to their vhost config file?

curl -Ii http://woestmeyer.com/
HTTP/1.1 500 Internal Server Error
Date: Sat, 06 May 2023 18:10:53 GMT
Server: Apache/2.4.52 (Ubuntu)
Strict-Transport-Security: max-age=15768000; preload
Referrer-Policy: no-referrer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Robots-Tag: none
X-XSS-Protection: 1; mode=block
Upgrade: h2,h2c
Connection: Upgrade, close
Content-Type: text/plain; charset=utf-8

What shows?:
ls -l /etc/apache2/sites-available/

3 Likes

Even the HTTPS site names are not working:

curl -Iik https://www.woestmeyer.com/
HTTP/1.1 500 Internal Server Error
Date: Sat, 06 May 2023 18:12:21 GMT
Server: Apache/2.4.52 (Ubuntu)
Strict-Transport-Security: max-age=15552000; includeSubDomains
Referrer-Policy: no-referrer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Robots-Tag: none
X-XSS-Protection: 1; mode=block
Upgrade: h2,h2c
Connection: Upgrade, close
Content-Type: text/plain; charset=utf-8

curl -Iik https://woestmeyer.com/
HTTP/1.1 500 Internal Server Error
Date: Sat, 06 May 2023 18:12:27 GMT
Server: Apache/2.4.52 (Ubuntu)
Strict-Transport-Security: max-age=15552000; includeSubDomains
Referrer-Policy: no-referrer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Robots-Tag: none
X-XSS-Protection: 1; mode=block
Upgrade: h2,h2c
Connection: Upgrade, close
Content-Type: text/plain; charset=utf-8
3 Likes

As an aside, port 80 was definitely closed earlier but I also see it open now. Carry on :slight_smile:

3 Likes

Now, I can too.

Renewal might actually work now.

2 Likes

It would still fail [I think] - but for another reason.

2 Likes

root@NextCloud:~$ ls -l /etc/apache2/sites-available/
insgesamt 16
-rw-r--r-- 1 root root 1813 Feb 8 2021 000-default.conf
-rw-r--r-- 1 root root 1684 Feb 8 2021 000-default-le-ssl.conf
-rw-r--r-- 1 root root 6338 Jun 27 2018 default-ssl.conf

What happened to their vhost config file?
I don't know. All I know, that it worked for years like this.

Today the renewal worked:
root@NextCloud:~$ sudo certbot certonly --force-renew -d woestmeyer.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log

How would you like to authenticate with the ACME CA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Apache Web Server plugin (apache)
2: Spin up a temporary webserver (standalone)
3: Place files in webroot directory (webroot)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
*Select the appropriate number [1-3] then [enter] (press 'c' to cancel): *
1 Renewing an existing certificate for woestmeyer.com

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/woestmeyer.com/fullchain.pem
Key is saved at: /etc/letsencrypt/live/woestmeyer.com/privkey.pem
This certificate expires on 2023-08-05.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.
(Same for www.woestemyer.com)

But the site is still marked as "not safe" and is still not accessabel.
NET::ERR_CERT_DATE_INVALID
Still shows, that the expiration date was 05/05/23.

sudo certbot certonly -a manual -d www.woestmeyer.com -d woestmeyer.com:

Certbot failed to authenticate some domains (authenticator: manual). The Certificate Authority reported these problems:

Hint: The Certificate Authority failed to verify the manually created challenge files. Ensure that you created these in the correct location.

I still don't know where the problem is. :roll_eyes:

Yes, I opened this port now for my Ubuntu server in my firewall. That's what I've read somewhere as a recommendation here.

The problem is the internal server error 500.

Let's have a look at this file:
/etc/apache2/sites-available/000-default.conf

3 Likes

sudo nano /etc/apache2/sites-available/000-default.conf:

<VirtualHost *:80>
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        #ServerName www.example.com

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html

        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf
RewriteEngine on
RewriteCond %{SERVER_NAME} =www.woestmeyer.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

Removing the unused lines, this is all that is there:

<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined
RewriteEngine on
RewriteCond %{SERVER_NAME} =www.woestmeyer.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

And not much of that seems useful.

  • there is no server_name
  • the document root is default
  • the redirection only addresses one of the two names

I would replace the entire file with:

<VirtualHost *:80>
    location /.well-known/acme-challenge/ {
        root /some-unique-path/; # make a directory just for challenge files
        try_files $uri $uri/ =404;
    }
    RewriteEngine On
    RewriteCond %{HTTPS} off
    RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
</VirtualHost>
2 Likes

I am sorry, this wasn't the whole file:

<VirtualHost *:80>

	ServerAdmin webmaster@localhost
	DocumentRoot /var/www/html

	ErrorLog ${APACHE_LOG_DIR}/error.log
	CustomLog ${APACHE_LOG_DIR}/access.log combined

RewriteEngine on
RewriteCond %{SERVER_NAME} =www.woestmeyer.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
<Directory /var/www/>
       Options +FollowSymlinks
       AllowOverride All

      <IfModule mod_dav.c>
        Dav off
      </IfModule>

       SetEnv HOME /var/www
       SetEnv HTTP_HOME /var/www
     </Directory>

<IfModule mod_headers.c>
          Header always set Strict-Transport-Security "max-age=15768000; preload"
     </IfModule>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

This is all.
Don't know what it all does. Maybe some part of the below causes the problem.

Well, that's not correct :slight_smile: Your cert history is long (link here) but only for the domain name www.woestmeyer.com. HTTPS requests to woestmeyer.com would have shown a security violation all these years.

There are several problems I see. Let's first review the certs you now have and your Apache HTTPS config. Please show output of these

sudo certbot certificates

And, the contents of this. Please put 3 backticks before and after the Apache config like
```
contents of: /etc/apache2/sites-enabled/000-default-le-ssl.conf
```

3 Likes