Sophos UTM 9 ACME Client Missing Certs

UTM has the proper certs installed via UTM admin but for some reason the creating of account fails as client can't get the TOS due to certificate errors. Odd thing is I have another UTM with same certs installed and have no issues accessing the pages and creating an account..

My domain is: halsoftdev.com

I ran this command: wget https://acme-v02.api.letsencrypt.org/directory and curl -vvvv -I -L -k https://acme-v02.api.letsencrypt.org/directory

It produced this output:
WGET Output:
Connecting to acme-v02.api.letsencrypt.org|172.65.32.248|:443... connected.
ERROR: cannot verify acme-v02.api.letsencrypt.org's certificate, issued by `/C=US/O=Let's Encrypt/CN=R11':
unable to get issuer certificate
CURL Output:

  • Server certificate:
  •    subject: CN=acme-v02.api.letsencrypt.org                                                                                                                                                                                       
    
  •    start date: 2024-06-25 20:21:41 GMT                                                                                                                                                                                            
    
  •    expire date: 2024-09-23 20:21:40 GMT                                                                                                                                                                                           
    
  •    issuer: C=US; O=Let's Encrypt; CN=R10                                                                                                                                                                                          
    
  •    SSL certificate verify result: unable to get issuer certificate (2), continuing anyway. 
    

My web server is (include version): N/A
The operating system my web server runs on is (include version): UTM 9 - Latest Version
My hosting provider, if applicable, is: N/A
I can login to a root shell on my machine (yes or no, or I don't know): YES
I'm using a control panel to manage my site (no, or provide the name and version of the control panel): UTM 9 Administration Site

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): N/A - Using built in client in UTM

Which certs are those exactly?

2 Likes

These are the certs listed in the UTM certificate management page.
I have another UTM which lists the same certs and has no issues. I do have root access to terminal. Sophos UTM is based on a Sophos-hardened version of a SUSE Linux 11 kernel (SLES11).

E1
[C=US,­ O=Let's Encrypt,­ CN=E1]
Verification CA Download
Valid from: Sep 4 00:00:00 2020 GMT
Valid until: Sep 15 16:00:00 2025 GMT
Fingerprint: 09:­1E:­8E:­A1:­B2:­56:­A3:­12:­96:­2A:­F6:­C1:­40:­C0:­FB:­F0:­79:­A4:­07:­B3

E2
[C=US,­ O=Let's Encrypt,­ CN=E2]
Verification CA Download
Valid from: Sep 4 00:00:00 2020 GMT
Valid until: Sep 15 16:00:00 2025 GMT
Fingerprint: 9A:­40:­DB:­EB:­34:­7E:­86:­1D:­52:­3A:­70:­74:­36:­22:­5E:­16:­D5:­00:­01:­33

R3
[C=US,­ O=Let's Encrypt,­ CN=R3]
Verification CA Download
Valid from: Sep 4 00:00:00 2020 GMT
Valid until: Sep 15 16:00:00 2025 GMT
Fingerprint: A0:­53:­37:­5B:­FE:­84:­E8:­B7:­48:­78:­2C:­7C:­EE:­15:­82:­7A:­6A:­F5:­A4:­05

R4
[C=US,­ O=Let's Encrypt,­ CN=R4]
Verification CA Download
Valid from: Sep 4 00:00:00 2020 GMT
Valid until: Sep 15 16:00:00 2025 GMT
Fingerprint: 4D:­7F:­2D:­E6:­4A:­8E:­44:­39:­4F:­EC:­4A:­7D:­C8:­CD:­C4:­98:­23:­0D:­B8:­29

X1
[C=US,­ O=Internet Security Research Group,­ CN=ISRG Root X1]
Verification CA Download
Valid from: Jun 4 11:04:38 2015 GMT
Valid until: Jun 4 11:04:38 2035 GMT
Fingerprint: CA:­BD:­2A:­79:­A1:­07:­6A:­31:­F2:­1D:­25:­36:­35:­CB:­03:­9D:­43:­29:­A5:­E8

X2
[C=US,­ O=Internet Security Research Group,­ CN=ISRG Root X2]
Verification CA Download
Valid from: Sep 4 00:00:00 2020 GMT
Valid until: Sep 17 16:00:00 2040 GMT
Fingerprint: BD:­B1:­B9:­3C:­D5:­97:­8D:­45:­C6:­26:­14:­55:­F8:­DB:­95:­C7:­5A:­D1:­53:­AF

X2 Cross
[C=US,­ O=Internet Security Research Group,­ CN=ISRG Root X2]
Verification CA Download
Valid from: Sep 4 00:00:00 2020 GMT
Valid until: Sep 15 16:00:00 2025 GMT
Fingerprint: 15:­16:­82:­F5:­21:­8C:­0A:­51:­1C:­28:­F4:­06:­0A:­73:­B9:­CA:­78:­CE:­9A:­53

Well, those E1/E2/R3/R4 intermediates shouldn't be used as trust anchors to begin with and they've been retired. Having ISRG Root X1 and ISRG Root X2 as trust anchors should be enough.

Having a cross-signed X2 as trust anchor is useless too.

3 Likes

I removed them and tried the wget for the URL and this is the error:
ERROR: cannot verify acme-v02.api.letsencrypt.org's certificate, issued by `/C=US/O=Let's Encrypt/CN=R10':
unable to get issuer certificate

Well, the intermediate certificate is also being send by the ACME endpoint and it's signed by ISRG Root X1.

So it should work if your clients system has ISRG Root X1 in their root certificate store..

3 Likes

Does your curl -vv output show lines for CAfile and/or CApath like this? If so, what are they?

curl -vv https://acme-v02.api.letsencrypt.org/directory
*   Trying 2606:4700:60:0:f53d:5624:85c7:3a2c:443...
* Connected to acme-v02.api.letsencrypt.org (2606:4700:60:0:f53d:5624:85c7:3a2c) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
5 Likes

Here is the output. UTM is a hardened kernel.

  • Hostname was NOT found in DNS cache
  • Trying 172.65.32.248...
  • Connected to acme-v02.api.letsencrypt.org (172.65.32.248) port 443 (#0)
  • successfully set certificate verify locations:
  • CAfile: none
    CApath: /etc/ssl/certs/
  • SSLv2, Unknown (22):
  • SSLv3, TLS handshake, Client hello (1):
  • SSLv2, Unknown (22):
  • SSLv3, TLS handshake, Server hello (2):
  • SSLv2, Unknown (22):
  • SSLv3, TLS handshake, CERT (11):
  • SSLv2, Unknown (21):
  • SSLv3, TLS alert, Server hello (2):
  • SSL certificate problem: unable to get issuer certificate
  • Closing connection 0
    curl: (60) SSL certificate problem: unable to get issuer certificate
    More details here: curl - SSL CA Certificates

curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.

Does that path contain the relevant root certs?

2 Likes

Yes. You can see the ISRG certs there.
-rw-r--r-- 1 root root 1982 Jul 6 11:31 ISRG_Root_X1.pem
-rw-r--r-- 1 root root 835 Jul 6 11:31 ISRG_Root_X2.pem

Weird..

If I set random paths/files for --cacert and --capath (it's rather hard to mess up the verification actually), I'm getting a "unable to get local issuer certificate" error.. I find it kinda weird your error mesage misses the "local" part.

Did you run update-ca-certificates after you added the certs to /etc/ssl/certs/?

What does openssl s_client -connect acme-v02.api.letsencrypt.org:443 say?

2 Likes

UTL is a harden kernel and that CA update is handled by the admin pages.
This is the output from openssl:

openssl s_client -connect acme-v02.api.letsencrypt.org:443
CONNECTED(00000003)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify error:num=2:unable to get issuer certificate
issuer= O = Digital Signature Trust Co., CN = DST Root CA X3

Certificate chain
 0 s:/CN=acme-v02.api.letsencrypt.org
   i:/C=US/O=Let's Encrypt/CN=R11
 1 s:/C=US/O=Let's Encrypt/CN=R11
   i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1

Server certificate
-----BEGIN CERTIFICATE-----
MIIFxDCCBKygAwIBAgISAy9vsYJHcup8vGrztPFdvCujMA0GCSqGSIb3DQEBCwUA
MDMxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQwwCgYDVQQD
EwNSMTEwHhcNMjQwNjI2MjMxNzAwWhcNMjQwOTI0MjMxNjU5WjAnMSUwIwYDVQQD
ExxhY21lLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnMIIBIjANBgkqhkiG9w0BAQEF
AAOCAQ8AMIIBCgKCAQEA1WMc59azR1+OGx+if2mUV0gM5Dcm3RwrO82YpEUj0l4D
ZuU+PbiZMdHe3GWWeJ3RxzMXNuThhvg/uAq3czk1hoUAMC1jz04iMOblf0rK/neC
aSCLEdJQFt9AXAaBAHSdmaimFbytpVMQhIUpaYuNxPGtlSM2M0R6/MNWx36GAKm4
N9kTwZswKO4s4qP4r0l1AyGQeB0vYHwHKYeGK+XM6RzSQYR8w58CI12iG00hX5hd
gAzNNH2DR0K63lL+c8a9a4d1IDo8t5WJGv+CbITSmPp+wxxypW3Fxij96zNUxSru
YFoNdLNPUvK9eHpvWCud8Ev4REiGrZHKhOWjllI7hwIDAQABo4IC3DCCAtgwDgYD
VR0PAQH/BAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAMBgNV
HRMBAf8EAjAAMB0GA1UdDgQWBBTRUzJfBBIvV+kh1EmFvJcdV4wNPjAfBgNVHSME
GDAWgBTFz0ak6vTDwHpslcQtsF6SLybjuTBXBggrBgEFBQcBAQRLMEkwIgYIKwYB
BQUHMAGGFmh0dHA6Ly9yMTEuby5sZW5jci5vcmcwIwYIKwYBBQUHMAKGF2h0dHA6
Ly9yMTEuaS5sZW5jci5vcmcvMIHjBgNVHREEgdswgdiCHmFjbWUtdjAyLTEuYXBp
LmxldHNlbmNyeXB0Lm9yZ4IeYWNtZS12MDItMi5hcGkubGV0c2VuY3J5cHQub3Jn
gh5hY21lLXYwMi0zLmFwaS5sZXRzZW5jcnlwdC5vcmeCHmFjbWUtdjAyLTQuYXBp
LmxldHNlbmNyeXB0Lm9yZ4IeYWNtZS12MDItNS5hcGkubGV0c2VuY3J5cHQub3Jn
ghxhY21lLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnghhpbmNpZGVudC5sZXRzZW5j
cnlwdC5vcmcwEwYDVR0gBAwwCjAIBgZngQwBAgEwggEDBgorBgEEAdZ5AgQCBIH0
BIHxAO8AdQAZmBBxCfDWUi4wgNKeP2S7g24ozPkPUo7u385KPxa0ygAAAZBXDX34
AAAEAwBGMEQCIG9egf+izvmvdijtnoB59cN/ollJPMzQLNXCfyIiZoTHAiBs6cD0
gZUD/TgcWGf1IpTRHO6V71lUOj5jFjGsJKjxwAB2AO7N0GTV2xrOxVy3nbTNE6Iy
h0Z8vOzew1FIWUZxH7WbAAABkFcNfbkAAAQDAEcwRQIgUwpoLu4/OMKa1rAhA9n1
F8jGHHyFLKdGKct6rrOo7rsCIQDSKjgTnN0UatZwvAWkgQSzGWRI70JA2HScwUKf
k4KCkTANBgkqhkiG9w0BAQsFAAOCAQEATlIdSGjftH8xUxWTzA/KCh/0hNusPK4Z
UAfJYtuhJpHAweRvPfWJ3VvCxNiOi4Th6gpFabT01U94WNn6nrPgowoERAlHJ3RE
giC5ZjRa+QLULY6gbOGWGKtUIFmj/XUM29OC2CGHFhjB2hikvQdBLkmysIoasUCg
kWM/pyeSQQjfVH5oWw0ZFSnYjUwHaHjvsM8WfR2Oh83zykOerfx4da4g6dZUlVQM
toVcrPnhBB8EK4dINX6c7ZPvBCWwCbweS/lSaQ+szeuC/EuNW2jKGmmtjiDmfMGi
ixzL79gHPJG/TBf7Wx2OOj5nKzpOCvLshjZ6gfC07FKeAvrCmSYEEw==
-----END CERTIFICATE-----

subject=/CN=acme-v02.api.letsencrypt.org
issuer=/C=US/O=Let's Encrypt/CN=R11

No client certificate CA names sent
Peer signing digest: SHA256
Server Temp Key: ECDH, P-256, 256 bits

SSL handshake has read 3280 bytes and written
419 bytes

New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 48528372A7EFBE3021D99879C88B04F3200289ECF3E38E918EE93187FA4879C7
    Session-ID-ctx:
    Master-Key: F89CF6F464C4FA9408F8FC81278B5887DEFEFFF2FAB7C59430FB00B7E2C158C121CA6D3549B874FF502DF00EEF2CF12C
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1720281439
    Timeout   : 300 (sec)
    Verify return code: 2 (unable to get issuer certificate)

This situation smells like how cPanel uses a repository of intermediate certificates rather than using the one that's provided by the CA for installation, but that's in the opposite direction. :thinking:

2 Likes

This is weird. It looks like your openssl tries to use the ISRG Root X1 cross-signed-by DST Root CA X3? Where does that come from?

3 Likes

Good question. Wonder if there is a "support pack" that contains all the certs required. I think we are sneaking up on this. Thanks for the experienced eye on this. Where can I get a .pem of this?
Digital Signature Trust Co., CN = DST Root CA X3

You don't want and you don't need that root, it has been deprecated and is going to expire soon anyway.

You need to figure out where the ISRG Root X1 cross-signed-by DST Root CA X3 is in your system. I'm guessing you've got it added somewhere and OpenSSL is trying to use it for some reason. I don't know why it would be in OpenSSLs output otherwise, as it is not used by Let's Encrypt.

4 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.