Renewed certificate reported as expired

Help
1

I run a small web and email server:
Debian 10.4
Apache 2.4.38-3+deb10u3
Postfix 3.4.10-0+deb10u1
Dovecot 2.3.4.1-5+deb10u2

The server hosts some domains. I use the certbot to create and renew the certificates. Apache, Postfix and Dovecot are configured to use the certificates in
/etc/letsencrypt/live/<domain>/

All was fine until the first renewal. Apache, Postfix and Dovecot have been restarted. The web sites provided by apache show the renewed certificate with the new expire date.
When I run openssl on the server, or when I check the site using online tools like
https://de.ssl-tools.net/mailservers
They show the certificate as expired! Also my mail client complains about expired certificates.
What can be wrong?

2

One thing that's wrong is that you didn't fill out the questionnaire which should have been presented to you when you started a thread in the Help section. Please fill out the following questions as best as you can:

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:

I ran this command:

It produced this output:

My web server is (include version):

The operating system my web server runs on is (include version):

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know):

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

3

My domain is: eehmke.de

I ran this command:

openssl s_client -connect mail.eehmke.de:465

It produced this output:
CONNECTED(00000005)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = eehmke.de
verify error:num=10:certificate has expired
notAfter=Jun 22 19:25:11 2020 GMT
verify return:1
depth=0 CN = eehmke.de
notAfter=Jun 22 19:25:11 2020 GMT
verify return:1
---
Certificate chain
 0 s:CN = eehmke.de
   i:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
 1 s:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
   i:O = Digital Signature Trust Co., CN = DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIF0DCCBLigAwIBAgISBAmmnV3kkkWfBfezF5xxAb1ZMA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0yMDAzMjQxOTI1MTFaFw0y
MDA2MjIxOTI1MTFaMBQxEjAQBgNVBAMTCWVlaG1rZS5kZTCCASIwDQYJKoZIhvcN
AQEBBQADggEPADCCAQoCggEBAJ7cWiPisAI4WX5ci3twtxj63cpqQaiMXDY5NBS/
2No19JyYFzTCoBQstWsJe2wAFp6YZ2GBrCTYQWXSbp3ZQrafve8/GCx2daYomlN5
UASyzm2SdBKRNvKixmsi5VwgycpzYGRmUsDA0OGGKxsDnlLaqLx7q21AVIXZaLSJ
N9nqmjH6tNtTjACRyt6TUo2zik609JdcETeCj6ExMavDCN0Jbnz17l6ObpQZ6d3w
QC6PrJJYTTmr9IOl6qKLvE3mjWn02urYk1sQInwTI9XkfbrEeifs8q382xu56Py2
uiSSCSnnt1Q7ttcbMJQFEBVm33C4x52YhHaqXnUaFT5REW8CAwEAAaOCAuQwggLg
MA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIw
DAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUYorrhkRhbCXH+kZT5I5Trsbc0jIwHwYD
VR0jBBgwFoAUqEpqYwR93brm0Tm3pkVl7/Oo7KEwbwYIKwYBBQUHAQEEYzBhMC4G
CCsGAQUFBzABhiJodHRwOi8vb2NzcC5pbnQteDMubGV0c2VuY3J5cHQub3JnMC8G
CCsGAQUFBzAChiNodHRwOi8vY2VydC5pbnQteDMubGV0c2VuY3J5cHQub3JnLzCB
mAYDVR0RBIGQMIGNgg5ibG9nLmVlaG1rZS5kZYIPY2xvdWQuZWVobWtlLmRlgg5k
YWVjLmVlaG1rZS5kZYIMZGIuZWVobWtlLmRlggllZWhta2UuZGWCDWZ0di5lZWht
a2UuZGWCDm1haWwuZWVobWtlLmRlghN0ZXN0Y2xvdWQuZWVobWtlLmRlgg13d3cu
ZWVobWtlLmRlMEwGA1UdIARFMEMwCAYGZ4EMAQIBMDcGCysGAQQBgt8TAQEBMCgw
JgYIKwYBBQUHAgEWGmh0dHA6Ly9jcHMubGV0c2VuY3J5cHQub3JnMIIBBQYKKwYB
BAHWeQIEAgSB9gSB8wDxAHcAb1N2rDHwMRnYmQCkURX/dxUcEdkCwQApBo2yCJo3
2RMAAAFxDjhvawAABAMASDBGAiEAnrckrVGmIkHbip0lVdoIVUQkDbfX/JAod0gX
ObTcZBoCIQDXMF999IfxATNX+D5Rq4qvDGmU45xS93PdcG7JGgm5wQB2ALIeBcyL
os2KIE6HZvkruYolIGdr2vpw57JJUy3vi5BeAAABcQ44bzUAAAQDAEcwRQIhANoj
dvPGr6QhVeHjMtLZ6gomHc69XyrKCR62Gpz3ALkYAiAXTduKx8/bcO7GnUdhsmQb
kraTL3j71hv7jdbOKYi23DANBgkqhkiG9w0BAQsFAAOCAQEAMVD2KvnQB/PCyhY4
dajMmRUw6Rt6+YZog2ZeSzpGTZag1zeEAaIAyIpeO9bgzfhfyiqtQh0eW0JMWkuO
1DYlcHwupV+dSaEjqy9QsDWztSgZv0szmYzqCqH/uSW6iQbl1L4hcr5JzJBVz2i1
vplKBZgGrJo0ZjhCpp5ZR6ycYKLX32vHmtUIbnPBKdYkmuU4Q3mtOMLGKZHV7iwk
PJjoCWurgbq4VJotfWpJBiv0ayz+4rwf7S2I+zzhHLdQI7aefSxgbp9le8dh0U9X
ssEL4ovVEqT18cOpgugz7BmA58ElyTCbXSkJ8GEI+HC3V0U2VSKPAxLnAEUKaQts
jJlM4g==
-----END CERTIFICATE-----
subject=CN = eehmke.de

issuer=C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3231 bytes and written 396 bytes
Verification error: certificate has expired
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 10 (certificate has expired)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 727D5D4E7F199169E857B5E674B5694474820B98D5F4C35B44329938955A4EEB
    Session-ID-ctx: 
    Resumption PSK: 6343C2299449E542844BE484CFB615FBF897A374C63192D8FE01FB63B89E77596F38611C9B14C98FC7BD7C141BBBE749
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - c9 47 be ef 8a ae e0 99-89 f1 29 2a 3c 8f 13 07   .G........)*<...
    0010 - 33 31 b9 60 97 86 75 f7-09 13 80 43 92 5e fa b0   31.`..u....C.^..
    0020 - 4b 9f 7f 3e 4b e3 ac a3-7f cb 97 2d 23 6d 9b 2f   K..>K......-#m./
    0030 - 27 f9 7c e9 98 03 28 63-8a 04 ae 78 98 75 d1 09   '.|...(c...x.u..
    0040 - 8a 23 14 40 d6 d4 c1 00-f7 12 52 1f 49 92 9a f5   .#.@......R.I...
    0050 - b3 9c 7a ec f2 9e de ac-00 de 64 53 5f 33 e0 1f   ..z.......dS_3..
    0060 - 7a d7 8f 96 9a 62 8a e5-d6 66 cf 40 77 69 1a 3f   z....b...f.@wi.?
    0070 - da 2d 5c 3c 22 55 4a d7-e3 76 6a a5 d6 09 63 12   .-\<"UJ..vj...c.
    0080 - de fd 5d dc 97 7b 9e 04-16 cd b4 71 18 b0 80 56   ..]..{.....q...V
    0090 - b3 20 07 ec 9b 75 14 41-98 78 71 e7 2d 97 be 21   . ...u.A.xq.-..!
    00a0 - 09 0b a6 89 2c 35 25 60-32 be 07 2a 9f 47 34 84   ....,5%`2..*.G4.
    00b0 - 16 04 23 32 2d fd fb 6f-2b 75 97 2a a5 8c 88 db   ..#2-..o+u.*....
    00c0 - 13 5a e5 62 33 c7 93 e7-aa 7d 49 0f 57 6a 3a 65   .Z.b3....}I.Wj:e
    00d0 - 22 b2 24 27 ff 67 ef fa-7d 03 34 b1 a2 ee 73 66   ".$'.g..}.4...sf

    Start Time: 1594304912
    Timeout   : 7200 (sec)
    Verify return code: 10 (certificate has expired)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
220 mail.eehmke.de ESMTP

My web server is (include version):
Apache 2.4.38-3+deb10u3

The operating system my web server runs on is (include version):
Debian 10.4

My hosting provider, if applicable, is:
Netcup

I can login to a root shell on my machine (yes or no, or I don’t know):
yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):
no, I have root access

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):
certbot 1.3.0

4

It only seems to be Postfix. Dovecot is using the most recent certificate.

Could you please show:

comm -23 <(postconf -n | sort) <(postconf -d | sort) | grep -E "^smtpd_"

5

comm -23 <(postconf -n) <(postconf -d) 2>&1 | grep -E “^smtpd_”

smtpd_banner = $myhostname ESMTP
smtpd_client_connection_count_limit = 25
smtpd_client_restrictions = permit_sasl_authenticated, check_client_access hash:/etc/postfix/whitelist, permit_mynetworks, reject_unauth_destination, reject_invalid_hostname, reject_unknown_client, reject_rbl_client sbl-xbl.spamhaus.org, permit
smtpd_data_restrictions = reject_unauth_pipelining, permit
smtpd_delay_reject = yes
smtpd_enforce_tls = no
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_sasl_authenticated, permit_mynetworks, check_helo_access hash:/etc/postfix/helo_access, reject_unauth_pipelining, reject_invalid_hostname, permit
smtpd_milters = unix:/spamass/spamass.sock, inet:localhost:12301
smtpd_recipient_limit = 1000
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, check_client_access hash:/etc/postfix/whitelist, reject_invalid_hostname, reject_unauth_destination, reject_unauth_pipelining, reject_non_fqdn_recipient, reject_non_fqdn_hostname, reject_non_fqdn_sender, reject_unknown_recipient_domain, reject_unknown_sender_domain, reject_unverified_recipient, reject_rbl_client sbl-xbl.spamhaus.org, check_policy_service unix:private/policy-spf, check_client_access hash:/etc/postfix/blacklist, check_policy_service inet:127.0.0.1:10023, permit
smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_invalid_hostname, reject_unauth_destination, reject_unauth_pipelining, reject_non_fqdn_recipient, reject_non_fqdn_hostname, reject_non_fqdn_sender, reject_unknown_recipient_domain, reject_unknown_sender_domain, reject_unverified_recipient, reject_rbl_client sbl-xbl.spamhaus.org
smtpd_sasl_auth_enable = yes
smtpd_sasl_authenticated_header = yes
smtpd_sasl_local_domain =
smtpd_sasl_path = private/auth
smtpd_sasl_security_options = noanonymous
smtpd_sasl_tls_security_options = $smtpd_sasl_security_options
smtpd_sasl_type = dovecot
smtpd_sender_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_non_fqdn_sender, reject_unknown_sender_domain, reject_unknown_address permit
smtpd_tls_auth_only = yes
smtpd_tls_chain_files = /etc/letsencrypt/live/www.eehmke.de/privkey.pem, /etc/letsencrypt/live/www.eehmke.de/fullchain.pem
smtpd_tls_loglevel = 1
smtpd_tls_protocols = !SSLv2, !SSLv3
smtpd_tls_received_header = yes
smtpd_tls_security_level = may
smtpd_use_tls = yes

6

Good, thanks.

And what’s the output of certbot certificates?

7

I replaced some subdomains by xxx etc.

sudo certbot certificates

Saving debug log to /var/log/letsencrypt/letsencrypt.log

Found the following certs:

Certificate Name: www.eehmke.de
Domains: eehmke.de blog.eehmke.de xxx.eehmke.de yyy.eehmke.de zzz.eehmke.de abc.eehmke.de mail.eehmke.de xyz.eehmke.de www.eehmke.de
Expiry Date: 2020-08-22 15:24:38+00:00 (VALID: 44 days)
Certificate Path: /etc/letsencrypt/live/www.eehmke.de/fullchain.pem
Private Key Path: /etc/letsencrypt/live/www.eehmke.de/privkey.pem

8

You know Let's Encrypt certificates are submitted to certificate logs on issuance, right? :wink:

In any case, I don't see any reason why Postfix shouldn't use your most recent certificate.

What do the Postfix logs say when you do a postfix reload ?

9

Just

Jul  9 17:03:37 eehmke postfix/postfix-script[13718]: refreshing the Postfix mail system
Jul  9 17:03:37 eehmke postfix/master[10404]: reload -- version 3.4.10, configuration /etc/postfix

No errors.

10

And openssl x509 -noout -text </etc/letsencrypt/live/www.eehmke.de/fullchain.pem really, actually, to be 200 % sure, shows the most recent certificate, right? I’m pretty much at a loss why Postfix isn’t using the correct file :pensive:

11 
# openssl x509 -noout -text </etc/letsencrypt/live/www.eehmke.de/fullchain.pem
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            03:c3:09:83:b1:5c:87:ee:1d:1a:8f:e2:18:45:36:b1:30:60
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
        Validity
            Not Before: May 24 15:24:38 2020 GMT
            Not After : Aug 22 15:24:38 2020 GMT
        Subject: CN = eehmke.de
...
12

Hi @eehmke

checking your domain you see the problem - https://check-your-website.server-daten.de/?q=eehmke.de#connections

Your main domain

2020-07-09.eehmke.de
2020-07-09.eehmke.de1149×253 12.3 KB

mail server (port 465) has the correct certificate:

CN=eehmke.de
	24.05.2020
	22.08.2020
expires in 44 days	blog.eehmke.de, cloud.eehmke.de, 
daec.eehmke.de, db.eehmke.de, eehmke.de, 
ftv.eehmke.de, mail.eehmke.de, 
testcloud.eehmke.de, www.eehmke.de - 9 entries

Your subdomain has the expired - see https://check-your-website.server-daten.de/?q=mail.eehmke.de#connections

CN=eehmke.de
	24.03.2020
	22.06.2020
17 days expired	blog.eehmke.de, cloud.eehmke.de, daec.eehmke.de, 
db.eehmke.de, eehmke.de, ftv.eehmke.de, mail.eehmke.de, 
testcloud.eehmke.de, www.eehmke.de - 9 entries

The ip addresses of main- and subdomain are the same.

But there are two different mail servers or one mail server with two different "vHosts" (don't know how that works with that MTA).

13

Hmm, the certificate is created for the main domain and all subdomains. How would there appear to be two mail servers? There are two vhosts indeed, for another domain.

14

Hm, indeed! When I change -servername mail.eehmke.de to -servername foo.eehmke.de or just eehmke.de, Postfix reports the correct certificate!

I didn’t even know Postfix supported virtualhosts for TLS? And why doesn’t postconf show multiple smtpd_tls_chain_files configuration options? :frowning:

Ah, I see, that is covered by tls_server_sni_maps, an option I’ve never heard of…

Perhaps you could show us:

comm -23 <(postconf -n | sort) <(postconf -d | sort) | grep -E "^tls_server_"

15

There is this line in main.cf:
tls_server_sni_maps = hash:/etc/postfix/vmail_ssl.map

That file contains the line:

mail.eehmke.de /etc/letsencrypt/live/www.eehmke.de/privkey.pem /etc/letsencrypt/live/www.eehmke.de/fullchain.pem

and the certificates for other domains.

16

I’m assuming the vmail_ssl.map has the correct spacing et cetera (probably removed by using the quote feature in stead of backticks).

Still at a loss here, sorry. Could it be that the certificate is somehow cached by using this hash table? And it’s a really, really good and sturdy cache unfortunately?

17

I followed this guide when I set up the vmail_ssl.map:

https://technikfreak.net/tipps-tricks/postfix-3-4-lets-encrypt-und-server-name-indication-sni/

18

Perhaps you’ll need to run postmap -F hash:/etc/postfix/sni after a renewal?

19

The hash only caches the names of the certificates, which have not changed

20

That’s what I figured from man postconf, but can’t hurt to try :grimacing: No other ideas left.