Certbot fails with SSLError / Max retries exceeded /directory after domain name and host name change

Yeah. Probably. Maybe updating the CA store would fix this. But supposedly this server was working before so idk.

Is this the really old R3 intermediate problem? I don't recall the details of that anymore it was so many threads ago. And, I have no idea where that would come from in this situation. That's why I wanted to see the actual certs just to check.

2 Likes

OP would need to add the -showcerts option to show us the entire chain. Although I think the chain would simply show the actual chain as send by the ACME API.

Right. Thanks. @revansx please add that to previous openssl command and show us entire output. Thanks

1 Like

as requested

openssl s_client -connect acme-v02.api.letsencrypt.org:443 -showcerts -servername acme-v02.api.letsencrypt.org

CONNECTED(00000003)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify error:num=2:unable to get issuer certificate
issuer= O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
issuer= C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=0 CN = acme-v02.api.letsencrypt.org
issuer= C = US, O = Let's Encrypt, CN = R3
verify return:1
---
Certificate chain
 0 s:CN = acme-v02.api.letsencrypt.org
   i:C = US, O = Let's Encrypt, CN = R3
-----BEGIN CERTIFICATE-----
MIIFxDCCBKygAwIBAgISA9hfcuI3eN+aBoonNSLFthf0MA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMzEwMjkxMTIyMTNaFw0yNDAxMjcxMTIyMTJaMCcxJTAjBgNVBAMT
HGFjbWUtdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcwggEiMA0GCSqGSIb3DQEBAQUA
A4IBDwAwggEKAoIBAQDgtVi3uy//gHqAgigmOPMVTXC4RDfMNwquT6WZPrBESF99
Wun66Z3emEgW7j1mSru/t5WY0oWWI6zefWIB0d/T8v70Z955iowYV5GjEkx/uC7R
EL7nBWtVqoXI8gklYVmbPm1VYVPzU6wVW1jrE2NwBozMRolD6w/DCf+t2gcTf6xh
W/SRjOS1pJfR3y/eEVQO4UMFGy9rTtfCaJ0iM3LNxCWdLKQ6ZBYBFhIK4GaNgfnF
IcH/D1D1oIzwjtHLltowQ58OPDhIquPEb1aFDbNXVarfsVDFbLZ1RF3Y5HqjS85T
3GApC09INsvH2r9iNm8oGlIeoNw+1RByOcY/lrVjAgMBAAGjggLdMIIC2TAOBgNV
HQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1Ud
EwEB/wQCMAAwHQYDVR0OBBYEFB1k/zZWdiIDhcnhkDI45bUEk/eBMB8GA1UdIwQY
MBaAFBQusxe3WFbLrlAJQOYfr52LFMLGMFUGCCsGAQUFBwEBBEkwRzAhBggrBgEF
BQcwAYYVaHR0cDovL3IzLm8ubGVuY3Iub3JnMCIGCCsGAQUFBzAChhZodHRwOi8v
cjMuaS5sZW5jci5vcmcvMIHjBgNVHREEgdswgdiCHmFjbWUtdjAyLTEuYXBpLmxl
dHNlbmNyeXB0Lm9yZ4IeYWNtZS12MDItMi5hcGkubGV0c2VuY3J5cHQub3Jngh5h
Y21lLXYwMi0zLmFwaS5sZXRzZW5jcnlwdC5vcmeCHmFjbWUtdjAyLTQuYXBpLmxl
dHNlbmNyeXB0Lm9yZ4IeYWNtZS12MDItNS5hcGkubGV0c2VuY3J5cHQub3Jnghxh
Y21lLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnghhpbmNpZGVudC5sZXRzZW5jcnlw
dC5vcmcwEwYDVR0gBAwwCjAIBgZngQwBAgEwggEGBgorBgEEAdZ5AgQCBIH3BIH0
APIAdwDatr9rP7W2Ip+bwrtca+hwkXFsu1GEhTS9pD0wSNf7qwAAAYt7Ynx3AAAE
AwBIMEYCIQCRg4KGWzDvkO31utznT8PFTDgK3Mpn4pRkbnJO1FU/CQIhALXSAdM9
KTwws0nHDzzwsGEEggakoGklLhSQjv8Belz7AHcAO1N3dT4tuYBOizBbBv5AO2fY
T8P0x70ADS1yb+H61BcAAAGLe2J8uAAABAMASDBGAiEAxm9d/hHiPwkPRhO5y/Ty
CrNaCKWnzqY9jrWfV+KNvuICIQDtspr6M2W7lhXQ1dfNzfLy7CRoD67dSnoxpc16
KSJerjANBgkqhkiG9w0BAQsFAAOCAQEAhtWT3IZvP7CG2SXyHz0FGfteKCba5VRv
f3+9Zt63+OvJKmrGYDAkodrUMvkhVpxTloGkUaPcStPG7K/QMy2yhisciv36p8pp
W2rMvn5/isgDbM8DGkyRU6jPqPRwH6c1vFR43Qc3NKCPUtryXMhrS5lTFKLDv7xx
EbkJ0V/4dVwIPOCErGXMg/cLrG6Jj7eBGAUTpeVGLj3RMIys1QchrLVOFC03Aqzm
LectuQDblb+qAB7EQVrfPljzPpc6dN8Hh/PterliayZBItauFau4eRi5QhErCdre
W3BlhT6ufamDtydJf83IhsPuRZTmKsAR7ZaKjNH+vnFCCjGT02Uk/A==
-----END CERTIFICATE-----
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
-----BEGIN CERTIFICATE-----
MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw
TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAw
WhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3Mg
RW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cP
R5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdx
sxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8Zutm
NHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxg
Z3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG
/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMC
AYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYB
Af8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaA
FHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcw
AoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRw
Oi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQB
gt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6W
PTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wl
ikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQz
CkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BIm
lJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4
avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2
yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1O
yK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90Ids
hCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+
HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6Zv
MldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqX
nLRbwHOoq7hHwg==
-----END CERTIFICATE-----
---
Server certificate
subject=CN = acme-v02.api.letsencrypt.org

issuer=C = US, O = Let's Encrypt, CN = R3

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3351 bytes and written 406 bytes
Verification error: unable to get issuer certificate
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 2 (unable to get issuer certificate)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 51293C4109E795DC6779D1568DFA49433CD608E5F9A070F127F5756E615439C2
    Session-ID-ctx: 
    Resumption PSK: 8599B3EA33A716048343ABA0B812632927A015463C82FCD93EBAE825D12D4962118710F1DCC1B94FF10CA49683B7D17F
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 86400 (seconds)
    TLS session ticket:
    0000 - 94 1e 03 1b 24 3b 1d 8a-dd 31 28 fd 2f 14 9b 24   ....$;...1(./..$
    0010 - 40 73 2a 65 f8 16 6e a7-9f 5d 79 83 77 77 d4 5a   @s*e..n..]y.ww.Z

    Start Time: 1701973185
    Timeout   : 7200 (sec)
    Verify return code: 2 (unable to get issuer certificate)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 3534266AD5F27ABD8309EFD0B15BE65C908D854ED2E47AAA6100ED7FCABCD5F6
    Session-ID-ctx: 
    Resumption PSK: 2F1F25402EC2B3F1F43F2595B288E6462BC7A0455A02C49330ABC43DFF7971156022B7270A04283287C3B675C16A14F2
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 86400 (seconds)
    TLS session ticket:
    0000 - 83 3f 1b 08 02 9f 78 01-81 4d bf 86 af ba dd 55   .?....x..M.....U
    0010 - ed 23 62 1d 16 26 a0 a9-41 0c 84 81 c8 1b f7 99   .#b..&..A.......

    Start Time: 1701973185
    Timeout   : 7200 (sec)
    Verify return code: 2 (unable to get issuer certificate)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK

...2 minute wait ...

closed

Thanks. Those are as expected.

So next step is update your CA cert store. Rocky is like RHEL - right?

What does this show?

cat /etc/pki/tls/cert.pem | grep -Ei 'ISRG|DST|Secti'
2 Likes

cat /etc/pki/tls/cert.pem | grep -Ei 'ISRG|DST|Secti'

# ISRG Root X1
yb/IQ639Lo7xr+L0mPoSHyDYwKcMhcWQ9DstliaxLL5Mq+ux0orJ23gTDx4JnW2P
# ISRG Root X1
# ISRG Root X2
n4rjyduYNM7YMxcoRvynyfDStNVNCXJJ+fKH46nafaF9a7I6JaltUkSs+L5u+9ym
N6a3vLqE6R5sGtRk2tRD/pOLS/IseRYQ1JMLiI+h2IYURpFHmygk71dSTlxCnKr3

It is odd to see two ISRG Root X1

If you browse that cert.pem file are the certs below those two comment lines the same?

2 Likes

Interesting in seeing the date of this file:
ls -l /etc/pki/tls/cert.pem

2 Likes

ls -l /etc/pki/tls/cert.pem

lrwxrwxrwx. 1 root root 49 Sep 19 16:57 /etc/pki/tls/cert.pem -> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

There are not

What shows?:
ls -l /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

1 Like

ls -l /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

-r--r--r--. 1 root root 227197 Dec 7 01:49 /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

Wow that seems brand new!
Can you upload that file here?

2 Likes

https://meza.wiki/WikiBlender/tls-ca-bundle.pem

1 Like

Well, now we know your CA store is definitely messed up.

The last two certs in that file are labelled R3 and ISRG Root X1

The first is an intermediate, not a root, and should not be there at all.

The second is mis-labelled and is actually the DST Root CA X3 cross-sign. While it is okay to be there other CA stores have since removed it given they have the real ISRG Root X1 itself.

I would remove them both or at least the R3 intermediate. Someone or something went out of their way to put them there as custom roots. So, you need to unwind that so they just don't get put back in at some future update.

The DST root may have been there at one point but it is now the last cert in the list. It never would have normally been the last which is why I say someone put it there intentionally.

3 Likes

Are you saying that if I remove the last two certs from that file then certbot might start working again?

Yes. You are having an unusual problem with certificate verification. The R3 intermediate should never be in your CA store. The CA store is a crucial element to certificate verification. So yes there's a good chance it is causing this problem.

4 Likes

[SOLUTION] It looks like that was indeed the thing that was keeping the "certbot certonly" command from running .. I removed those two certs from /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem and it ran and generated the cert - yay - thank you!

The new cert is now being used on my server https://meza.wiki

however now the browser now says that the cert is valid but the site is still untrusted.

Can you confirm this?
Any thoughts?

[POST SOLUTION NOTE - The issue after the cert was made and applied was a caching issue and vanished after a Ctrl-R on the browser]

You will have to provide more details. I also see the cert is fine.

Does your browser give a reason? Does it have anything to do with you having TLSv1 enabled on your server?

https://www.ssllabs.com/ssltest/analyze.html?d=meza.wiki&hideResults=on

3 Likes

Looks good from my end. Perhaps a caching issue.

2 Likes