Browsers sporadically getting old certificate dates after renewal

My domain is: everynoise.com

My web server is (include version): Apache/2.4.25

The operating system my web server runs on is (include version): Debian GNU/Linux 9.12

My hosting provider, if applicable, is: Google Cloud

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.16.0

Sporadically, for both me and others, going to everynoise.com will produce the warning that my certificate has expired, and checking the details shows the old date before the last renewal.

However, refreshing will usually make the warning go away, and the proper date appear for the certificate (which is currently valid through 2021-07-16 according to certbot certificates). It keeps happening, though.

What should I do?

1 Like

Welcome to the Let's Encrypt Community, Glenn :slightly_smiling_face:

It appears that everynoise.com is serving two certificate chains, one of which is led by an expired leaf certificate.

https://www.ssllabs.com/ssltest/analyze.html?d=everynoise.com

Check your Apache configuration beginning with:

sudo apachectl -S

1 Like

That reveals this, which I assume is showing the two chains. I didn't create those intentionally, though. Did I do certbot wrong, somehow, to cause this? Or how do I fix it?

VirtualHost configuration:
*:80 everynoise.com (/etc/apache2/sites-enabled/000-default.conf:1)
*:443 is a NameVirtualHost
default server everynoise.com (/etc/apache2/sites-enabled/000-default-le-ssl.conf:2)
port 443 namevhost everynoise.com (/etc/apache2/sites-enabled/000-default-le-ssl.conf:2)
port 443 namevhost everynoise-2.c.data-alchemy.internal (/etc/apache2/sites-enabled/default-ssl.conf:2)

1 Like

Difficult to say. We're going to find out though. :grin:

What are the contents of these files?

/etc/apache2/sites-enabled/000-default.conf
/etc/apache2/sites-enabled/000-default-le-ssl.conf
/etc/apache2/sites-enabled/default-ssl.conf

What are the outputs of these commands?

sudo certbot certificates
sudo ls -lRa /etc/letsencrypt
sudo ls -lRa /etc/apache2/sites-available
sudo ls -lRa /etc/apache2/sites-enabled

Please put three backticks above and below each content and output, like this:

```
content/output
```

1 Like

sudo certbot certificates:

Found the following certs:
  Certificate Name: everynoise.com
    Serial Number: 4c896cef394b9196df02b5a8c895d703d98
    Key Type: RSA
    Domains: everynoise.com
    Expiry Date: 2021-07-16 21:47:41+00:00 (VALID: 34 days)
    Certificate Path: /etc/letsencrypt/live/everynoise.com/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/everynoise.com/privkey.pem

sudo ls -lRa /etc/letsencrypt

/etc/letsencrypt:
total 44
drwxr-xr-x  9 root root 4096 Jun 12 00:03 .
drwxr-xr-x 78 root root 4096 Jun 11 23:31 ..
-rw-r--r--  1 root root   64 Feb 16 14:19 .updated-options-ssl-apache-conf-digest.txt
drwx------  4 root root 4096 Feb 16 14:28 accounts
drwx------  3 root root 4096 Feb 16 14:20 archive
drwxr-xr-x  2 root root 4096 Apr 17 22:47 csr
drwx------  2 root root 4096 Apr 17 22:47 keys
drwx------  3 root root 4096 Feb 16 14:20 live
-rw-r--r--  1 root root  952 Feb 16 14:19 options-ssl-apache.conf
drwxr-xr-x  2 root root 4096 Apr 17 22:47 renewal
drwxr-xr-x  5 root root 4096 Feb 16 14:19 renewal-hooks

/etc/letsencrypt/accounts:
total 16
drwx------ 4 root root 4096 Feb 16 14:28 .
drwxr-xr-x 9 root root 4096 Jun 12 00:03 ..
drwx------ 3 root root 4096 Feb 16 14:28 acme-staging-v02.api.letsencrypt.org
drwx------ 3 root root 4096 Feb 16 14:19 acme-v02.api.letsencrypt.org

/etc/letsencrypt/accounts/acme-staging-v02.api.letsencrypt.org:
total 12
drwx------ 3 root root 4096 Feb 16 14:28 .
drwx------ 4 root root 4096 Feb 16 14:28 ..
drwx------ 3 root root 4096 Feb 16 14:28 directory

/etc/letsencrypt/accounts/acme-staging-v02.api.letsencrypt.org/directory:
total 12
drwx------ 3 root root 4096 Feb 16 14:28 .
drwx------ 3 root root 4096 Feb 16 14:28 ..
drwx------ 2 root root 4096 Feb 16 14:28 dbba9d7b27bff4bf223b8cd84146fa71

/etc/letsencrypt/accounts/acme-staging-v02.api.letsencrypt.org/directory/dbba9d7b27bff4bf223b8cd84146fa71:
total 20
drwx------ 2 root root 4096 Feb 16 14:28 .
drwx------ 3 root root 4096 Feb 16 14:28 ..
-rw-r--r-- 1 root root   96 Feb 16 14:28 meta.json
-r-------- 1 root root 1632 Feb 16 14:28 private_key.json
-rw-r--r-- 1 root root   86 Feb 16 14:28 regr.json

/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org:
total 12
drwx------ 3 root root 4096 Feb 16 14:19 .
drwx------ 4 root root 4096 Feb 16 14:28 ..
drwx------ 3 root root 4096 Feb 16 14:19 directory

/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory:
total 12
drwx------ 3 root root 4096 Feb 16 14:19 .
drwx------ 3 root root 4096 Feb 16 14:19 ..
drwx------ 2 root root 4096 Feb 16 14:19 730b78a68299052b39d6f35bfb832661

/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory/730b78a68299052b39d6f35bfb832661:
total 20
drwx------ 2 root root 4096 Feb 16 14:19 .
drwx------ 3 root root 4096 Feb 16 14:19 ..
-rw-r--r-- 1 root root   96 Feb 16 14:20 meta.json
-r-------- 1 root root 1632 Feb 16 14:19 private_key.json
-rw-r--r-- 1 root root   79 Feb 16 14:19 regr.json

/etc/letsencrypt/archive:
total 12
drwx------ 3 root root 4096 Feb 16 14:20 .
drwxr-xr-x 5 root root 4096 Feb 16 14:19 ..
drwxr-xr-x 2 root root 4096 Apr 17 22:47 everynoise.com

/etc/letsencrypt/archive/everynoise.com:
total 40
drwxr-xr-x 2 root root 4096 Apr 17 22:47 .
drwx------ 3 root root 4096 Feb 16 14:20 ..
-rw-r--r-- 1 root root 1838 Feb 16 14:20 cert1.pem
-rw-r--r-- 1 root root 1842 Apr 17 22:47 cert2.pem
-rw-r--r-- 1 root root 1586 Feb 16 14:20 chain1.pem
-rw-r--r-- 1 root root 1586 Apr 17 22:47 chain2.pem
-rw-r--r-- 1 root root 3424 Feb 16 14:20 fullchain1.pem
-rw-r--r-- 1 root root 3428 Apr 17 22:47 fullchain2.pem
-rw------- 1 root root 1704 Feb 16 14:20 privkey1.pem
-rw------- 1 root root 1704 Apr 17 22:47 privkey2.pem

/etc/letsencrypt/csr:
total 16
drwxr-xr-x 2 root root 4096 Apr 17 22:47 .
drwxr-xr-x 9 root root 4096 Jun 12 00:03 ..
-rw-r--r-- 1 root root  924 Feb 16 14:20 0000_csr-certbot.pem
-rw-r--r-- 1 root root  924 Apr 17 22:47 0001_csr-certbot.pem

/etc/letsencrypt/keys:
total 16
drwx------ 2 root root 4096 Apr 17 22:47 .
drwxr-xr-x 9 root root 4096 Jun 12 00:03 ..
-rw------- 1 root root 1704 Feb 16 14:20 0000_key-certbot.pem
-rw------- 1 root root 1704 Apr 17 22:47 0001_key-certbot.pem

/etc/letsencrypt/live:
total 16
drwx------ 3 root root 4096 Feb 16 14:20 .
drwxr-xr-x 9 root root 4096 Jun 12 00:03 ..
-rw-r--r-- 1 root root  740 Feb 16 14:20 README
drwxr-xr-x 2 root root 4096 Apr 17 22:47 everynoise.com

/etc/letsencrypt/live/everynoise.com:
total 12
drwxr-xr-x 2 root root 4096 Apr 17 22:47 .
drwx------ 3 root root 4096 Feb 16 14:20 ..
-rw-r--r-- 1 root root  692 Feb 16 14:20 README
lrwxrwxrwx 1 root root   38 Apr 17 22:47 cert.pem -> ../../archive/everynoise.com/cert2.pem
lrwxrwxrwx 1 root root   39 Apr 17 22:47 chain.pem -> ../../archive/everynoise.com/chain2.pem
lrwxrwxrwx 1 root root   43 Apr 17 22:47 fullchain.pem -> ../../archive/everynoise.com/fullchain2.pem
lrwxrwxrwx 1 root root   41 Apr 17 22:47 privkey.pem -> ../../archive/everynoise.com/privkey2.pem

/etc/letsencrypt/renewal:
total 12
drwxr-xr-x 2 root root 4096 Apr 17 22:47 .
drwxr-xr-x 9 root root 4096 Jun 12 00:03 ..
-rw-r--r-- 1 root root  529 Apr 17 22:47 everynoise.com.conf

/etc/letsencrypt/renewal-hooks:
total 20
drwxr-xr-x 5 root root 4096 Feb 16 14:19 .
drwxr-xr-x 9 root root 4096 Jun 12 00:03 ..
drwxr-xr-x 2 root root 4096 Feb 16 14:19 deploy
drwxr-xr-x 2 root root 4096 Feb 16 14:19 post
drwxr-xr-x 2 root root 4096 Feb 16 14:19 pre

/etc/letsencrypt/renewal-hooks/deploy:
total 8
drwxr-xr-x 2 root root 4096 Feb 16 14:19 .
drwxr-xr-x 5 root root 4096 Feb 16 14:19 ..

/etc/letsencrypt/renewal-hooks/post:
total 8
drwxr-xr-x 2 root root 4096 Feb 16 14:19 .
drwxr-xr-x 5 root root 4096 Feb 16 14:19 ..

/etc/letsencrypt/renewal-hooks/pre:
total 8
drwxr-xr-x 2 root root 4096 Feb 16 14:19 .
drwxr-xr-x 5 root root 4096 Feb 16 14:19 ..

sudo ls -lRa /etc/apache2/sites-available

/etc/apache2/sites-available:
total 24
drwxr-xr-x 2 root root 4096 Apr 17 22:47 .
drwxr-xr-x 8 root root 4096 Jun 11 23:57 ..
-rw-r--r-- 1 root root 1886 Feb 16 14:20 000-default-le-ssl.conf
-rw-r--r-- 1 root root 1838 Feb 16 14:20 000-default.conf
-rw-r--r-- 1 root root 6338 Jun 16  2019 default-ssl.conf

glennm@everynoise-2:/etc/apache2$ sudo ls -lRa /etc/apache2/sites-enabled

/etc/apache2/sites-enabled:
total 8
drwxr-xr-x 2 root root 4096 Feb 16 14:20 .
drwxr-xr-x 8 root root 4096 Jun 11 23:57 ..
lrwxrwxrwx 1 root root   52 Feb 16 14:20 000-default-le-ssl.conf -> /etc/apache2/sites-available/000-default-le-ssl.conf
lrwxrwxrwx 1 root root   35 Jul 20  2020 000-default.conf -> ../sites-available/000-default.conf
lrwxrwxrwx 1 root root   35 Jul 20  2020 default-ssl.conf -> ../sites-available/default-ssl.conf

000-default-le-ssl.conf

<IfModule mod_ssl.c>
<VirtualHost *:443>
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        #ServerName www.example.com

        ServerName everynoise.com
        ServerAdmin glennm@spotify.com
        DocumentRoot /var/www/html

        <Directory "/var/www/html/">
                Options +ExecCGI -Indexes
                AddHandler cgi-script .cgi
                AllowOverride All
                Require all granted
                DirectoryIndex engenremap.html
        </Directory>

        <FilesMatch "\.py$">
                Require all denied
        </FilesMatch>

        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf


SSLCertificateFile /etc/letsencrypt/live/everynoise.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/everynoise.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>

000-default.conf

<VirtualHost *:80>
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        #ServerName www.example.com

        ServerName everynoise.com
        ServerAdmin glennm@spotify.com
        DocumentRoot /var/www/html

        <Directory "/var/www/html/">
                Options +ExecCGI -Indexes
                AddHandler cgi-script .cgi
                AllowOverride All
                Require all granted
                DirectoryIndex engenremap.html
        </Directory>

        <FilesMatch "\.py$">
                Require all denied
        </FilesMatch>

        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf
RewriteEngine on
RewriteCond %{SERVER_NAME} =everynoise.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

default-ssl.conf

<IfModule mod_ssl.c>
        <VirtualHost _default_:443>
                ServerAdmin webmaster@localhost

                DocumentRoot /var/www/html

                # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
                # error, crit, alert, emerg.
                # It is also possible to configure the loglevel for particular
                # modules, e.g.
                #LogLevel info ssl:warn

                ErrorLog ${APACHE_LOG_DIR}/error.log
                CustomLog ${APACHE_LOG_DIR}/access.log combined

                # For most configuration files from conf-available/, which are
                # enabled or disabled at a global level, it is possible to
                # include a line for only one particular virtual host. For example the
                # following line enables the CGI configuration for this host only
                # after it has been globally disabled with "a2disconf".
                #Include conf-available/serve-cgi-bin.conf

                #   SSL Engine Switch:
                #   Enable/Disable SSL for this virtual host.
                SSLEngine on

                #   A self-signed (snakeoil) certificate can be created by installing
                #   the ssl-cert package. See
                #   /usr/share/doc/apache2/README.Debian.gz for more info.
                #   If both key and certificate are stored in the same file, only the
                #   SSLCertificateFile directive is needed.
                SSLCertificateFile      /etc/ssl/certs/ssl-cert-snakeoil.pem
                SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key

                #   Server Certificate Chain:
                #   Point SSLCertificateChainFile at a file containing the
                #   concatenation of PEM encoded CA certificates which form the
                #   certificate chain for the server certificate. Alternatively
                #   the referenced file can be the same as SSLCertificateFile
                #   when the CA certificates are directly appended to the server
                #   certificate for convinience.
                #SSLCertificateChainFile /etc/apache2/ssl.crt/server-ca.crt

                #   Certificate Authority (CA):
                #   Set the CA certificate verification path where to find CA
                #   certificates for client authentication or alternatively one
                #   huge file containing all of them (file must be PEM encoded)
                #   Note: Inside SSLCACertificatePath you need hash symlinks
                #                to point to the certificate files. Use the provided
                #                Makefile to update the hash symlinks after changes.
                #SSLCACertificatePath /etc/ssl/certs/
                #SSLCACertificateFile /etc/apache2/ssl.crt/ca-bundle.crt

                #   Certificate Revocation Lists (CRL):
                #   Set the CA revocation path where to find CA CRLs for client
                #   authentication or alternatively one huge file containing all
                #   of them (file must be PEM encoded)
                #   Note: Inside SSLCARevocationPath you need hash symlinks provided
                #SSLCARevocationPath /etc/apache2/ssl.crl/
                #SSLCARevocationFile /etc/apache2/ssl.crl/ca-bundle.crl

                #   Client Authentication (Type):
                #   Client certificate verification type and depth.  Types are
                #   none, optional, require and optional_no_ca.  Depth is a
                #   number which specifies how deeply to verify the certificate
                #   issuer chain before deciding the certificate is not valid.
                #SSLVerifyClient require
                #SSLVerifyDepth  10

                #   SSL Engine Options:
                #   Set various options for the SSL engine.
                #   o FakeBasicAuth:
                #        Translate the client X.509 into a Basic Authorisation.  This means that
                #        the standard Auth/DBMAuth methods can be used for access control.  The
                #        user name is the `one line' version of the client's X.509 certificate.
                #        Note that no password is obtained from the user. Every entry in the user
                #        file needs this password: `xxj31ZMTZzkVA'.
                #   o ExportCertData:
                #        This exports two additional environment variables: SSL_CLIENT_CERT and
                #        SSL_SERVER_CERT. These contain the PEM-encoded certificates of the
                #        server (always existing) and the client (only existing when client
                #        authentication is used). This can be used to import the certificates
                #        into CGI scripts.
                #   o StdEnvVars:
                #        This exports the standard SSL/TLS related `SSL_*' environment variables.
                #        Per default this exportation is switched off for performance reasons,
                #        because the extraction step is an expensive operation and is usually
                #        useless for serving static content. So one usually enables the
                #        exportation for CGI and SSI requests only.
                #   o OptRenegotiate:
                #        This enables optimized SSL connection renegotiation handling when SSL
                #        directives are used in per-directory context.
                #SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
                <FilesMatch "\.(cgi|shtml|phtml|php)$">
                                SSLOptions +StdEnvVars
                </FilesMatch>
                <Directory /usr/lib/cgi-bin>
                                SSLOptions +StdEnvVars
                </Directory>

                #   SSL Protocol Adjustments:
                #   The safe and default but still SSL/TLS standard compliant shutdown
                #   approach is that mod_ssl sends the close notify alert but doesn't wait for
                #   the close notify alert from client. When you need a different shutdown
                #   approach you can use one of the following variables:
                #   o ssl-unclean-shutdown:
                #        This forces an unclean shutdown when the connection is closed, i.e. no
                #        SSL close notify alert is send or allowed to received.  This violates
                #        the SSL/TLS standard but is needed for some brain-dead browsers. Use
                #        this when you receive I/O errors because of the standard approach where
                #        mod_ssl sends the close notify alert.
                #   o ssl-accurate-shutdown:
                #        This forces an accurate shutdown when the connection is closed, i.e. a
                #        SSL close notify alert is send and mod_ssl waits for the close notify
                #        alert of the client. This is 100% SSL/TLS standard compliant, but in
                #        practice often causes hanging connections with brain-dead browsers. Use
                #        this only for browsers where you know that their SSL implementation
                #        works correctly.
                #   Notice: Most problems of broken clients are also related to the HTTP
                #   keep-alive facility, so you usually additionally want to disable
                #   keep-alive for those clients, too. Use variable "nokeepalive" for this.
                #   Similarly, one has to force some clients to use HTTP/1.0 to workaround
                #   their broken HTTP/1.1 implementation. Use variables "downgrade-1.0" and
                #   "force-response-1.0" for this.
                # BrowserMatch "MSIE [2-6]" \
                #               nokeepalive ssl-unclean-shutdown \
                #               downgrade-1.0 force-response-1.0

        </VirtualHost>
</IfModule>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
1 Like

Those all look pretty good. I would recommend disabling default-ssl.conf since isn't doing anything useful here.

sudo a2dissite default-ssl.conf

sudo apachectl -k graceful

You could have some old apache worker processes floating around that are serving the old certificate. Hopefully the reload command I gave you to run after disabling default-ssl.conf will help clear those out.

1 Like

-k is an illegal option?

1 Like

oh, -k restart, not reload

1 Like

Should be -k graceful. Sorry, nginx on the brain.

1 Like

restart is fine to use too

graceful is less disruptive, but maybe disruptive is helpful here

1 Like

This looks happier now. I will optimistically assume we fixed it until proven otherwise!

Thanks!

1 Like

Let me check. One moment... :slightly_smiling_face:

1 Like

Looks vastly better to me. :smiley:

Grade A now vs Grade T before

https://www.ssllabs.com/ssltest/analyze.html?d=everynoise.com

1 Like

Awesome. Thanks again!

2 Likes

You're very welcome. :blush:

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.