Cmake.org certificate verification failure due to OCSP

OCSP: http://r3.o.lencr.org

  Certificate verification failed: The certificate is NOT trusted. The received OCSP status response is invalid.  Could not handshake: Error in the certificate verification. [IP: 66.194.253.25 443]

This server is used for cmake.org and apt.kitware.com cert. This is affecting a lot of CI builds right now

It looks like the OCSP response stapled by the cmake.org webserver is not good (OCSP Response Status: trylater (0x3)).

Do you run cmake.org? Restarting Apache might do the trick ... or even disabling stapling if the server can't get a response. Apache has some not-very-ideal OCSP stapling behavior and I think this is one of the bad things that can happen.

I think Let's Encrypt's OCSP servers are fine, I can get a good response for the cmake.org certificate.

5 Likes

I can confirm. For future reference, this is what I'm seeing as of now:

The cert currently served by cmake.org is this one (serial 03:ae:dd:1a:1c:57:a9:b8:65:73:84:a7:36:a5:ea:ef:14:81)

Requesting OCSP from Let's Encrypt works fine:

openssl ocsp -issuer r3.pem -serial 0x03aedd1a1c57a9b8657384a736a5eaef1481 -url http://r3.o.lencr.org -text

OCSP Request Data:
    Version: 1 (0x0)
    Requestor List:
        Certificate ID:
          Hash Algorithm: sha1
          Issuer Name Hash: 48DAC9A0FB2BD32D4FF0DE68D2F567B735F9B3C4
          Issuer Key Hash: 142EB317B75856CBAE500940E61FAF9D8B14C2C6
          Serial Number: 03AEDD1A1C57A9B8657384A736A5EAEF1481
    Request Extensions:
        OCSP Nonce:
            04106B0CA787992F682ED688CCF57F3669DE
OCSP Response Data:
    OCSP Response Status: successful (0x0)
    Response Type: Basic OCSP Response
    Version: 1 (0x0)
    Responder Id: C = US, O = Let's Encrypt, CN = R3
    Produced At: May  3 23:34:00 2023 GMT
    Responses:
    Certificate ID:
      Hash Algorithm: sha1
      Issuer Name Hash: 48DAC9A0FB2BD32D4FF0DE68D2F567B735F9B3C4
      Issuer Key Hash: 142EB317B75856CBAE500940E61FAF9D8B14C2C6
      Serial Number: 03AEDD1A1C57A9B8657384A736A5EAEF1481
    Cert Status: good
    This Update: May  3 23:00:00 2023 GMT
    Next Update: May 10 22:59:58 2023 GMT

    Signature Algorithm: sha256WithRSAEncryption
    Signature Value:
        70:35:bc:15:d7:38:7c:71:3d:87:83:91:92:a2:6d:6f:7b:a4:
        c0:68:cb:69:8f:b2:e6:01:9d:20:04:34:b3:5c:e4:fd:39:a8:
        4b:e5:dc:b2:26:08:bd:dc:99:6e:73:c3:20:66:0f:fe:ac:a8:
        da:31:a3:27:0b:86:c9:3c:14:2d:3a:05:4a:37:da:70:57:c0:
        68:40:2b:93:65:d4:b6:47:66:a8:a8:76:cb:4e:6b:5e:3f:21:
        26:7a:e5:39:e6:77:6f:34:e1:c3:e7:bd:45:85:64:f2:b4:23:
        1d:eb:9b:be:bb:a8:1f:b7:51:c9:71:81:8b:71:36:96:57:8a:
        50:fa:41:bb:32:65:08:23:d5:29:f3:6e:94:23:c3:77:07:6b:
        59:ff:1a:ea:8d:64:cf:ff:44:b3:67:25:2c:6a:89:b9:ea:ce:
        83:27:fc:b5:f6:f0:b4:c7:63:89:8e:92:2a:ec:ed:4b:ca:cf:
        ff:b2:56:0d:51:81:0d:f8:9e:ab:a1:6e:00:de:38:dc:1a:b8:
        a7:6a:eb:c1:04:d2:99:9a:4f:ec:2a:55:20:3c:53:d3:f3:f4:
        1f:d1:05:e8:4f:db:c8:65:0a:1c:2a:9e:25:58:a4:e5:48:11:
        81:5a:e8:72:11:6f:97:31:40:b0:62:34:32:00:dc:ad:8e:c8:
        fd:bb:f3:55
WARNING: no nonce in response
Response verify OK
0x03aedd1a1c57a9b8657384a736a5eaef1481: good
        This Update: May  3 23:00:00 2023 GMT
        Next Update: May 10 22:59:58 2023 GMT

The server's currently served cert + OCSP response:

openssl s_client -connect cmake.org:443 -status

CONNECTED(000001A0)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = cmake.org
verify return:1
OCSP response:
======================================
OCSP Response Data:
    OCSP Response Status: trylater (0x3)
======================================
---
Certificate chain
 0 s:CN = cmake.org
   i:C = US, O = Let's Encrypt, CN = R3
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: May  1 12:46:48 2023 GMT; NotAfter: Jul 30 12:46:47 2023 GMT
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Sep  4 00:00:00 2020 GMT; NotAfter: Sep 15 16:00:00 2025 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFKTCCBBGgAwIBAgISA67dGhxXqbhlc4SnNqXq7xSBMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMzA1MDExMjQ2NDhaFw0yMzA3MzAxMjQ2NDdaMBQxEjAQBgNVBAMT
CWNtYWtlLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJLaPyLU
8scvy5sKA30hM2w9FW6SAMx1L5+Ql37UHQmT2ZVkUdvtOqKL6b8sNldESPh1caNT
STqjTAgAynB5VGCvXmtHpyQzPO1NXoPAP1Vo2k48hn++QFva/o+E4Jlu0wpsyVAq
ElgHBagLo7mOF8rwkvoEGRc8C//q+VIZ68vusqKudh82Bm9wrtIpBt7zrqGRFPY5
uo1vDVsD76qkW7yDfWfyQ8SuKc3O9rvdZvqkuDzGDrvB+qP11teI3ORc0fUOxHxj
q7V+2Kwm4RM2oDOR+oGMGae+rDax1JmWpvBSWOXlrIOiUcdZUc36/pfg9uMmi2Zl
OHKvwPb4PVpTyGUCAwEAAaOCAlUwggJRMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUE
FjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQU
BeTnfKlbXleVVjc3qIGAXY6BLpkwHwYDVR0jBBgwFoAUFC6zF7dYVsuuUAlA5h+v
nYsUwsYwVQYIKwYBBQUHAQEESTBHMCEGCCsGAQUFBzABhhVodHRwOi8vcjMuby5s
ZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6Ly9yMy5pLmxlbmNyLm9yZy8wIwYD
VR0RBBwwGoIJY21ha2Uub3Jngg13d3cuY21ha2Uub3JnMEwGA1UdIARFMEMwCAYG
Z4EMAQIBMDcGCysGAQQBgt8TAQEBMCgwJgYIKwYBBQUHAgEWGmh0dHA6Ly9jcHMu
bGV0c2VuY3J5cHQub3JnMIIBBgYKKwYBBAHWeQIEAgSB9wSB9ADyAHcAtz77JN+c
Tbp18jnFulj0bF38Qs96nzXEnh0JgSXttJkAAAGH15DeRgAABAMASDBGAiEAkbYd
sbhrS1ZO2yN+wi77GXZhUxCEqABMOd5zoz1tgQgCIQD6SLiiyZvg21p8Q5LmgKpL
lb9kc1uIN+9nOom4pJpggwB3AK33vvp8/xDIi509nB4+GGq0Zyldz7EMJMqFhjTr
3IKKAAABh9eQ3nIAAAQDAEgwRgIhALTU572nEi+2Eqrjiha1Np8N0GyggYs/AW6r
HUxXvjhQAiEAzzGC7Flyz6gXR7HFpsWylLfTVwKB7dXdq20+FpAfhT0wDQYJKoZI
hvcNAQELBQADggEBAJC6p/BhANhJ+fIzRMUb/pmS1BNsin+noeFWC8ukMKnB6tOJ
bAwYA0n7F+NdUDrDpU67nlgKhmonjBxs+OsRpD3UT3AxgQrvWAZ1cln4BPbCuUlq
0KT70B3Q+nl6VcOhhQEo2uvl0wOCQcDd9+uadqZmEybdTiuBxgQ7VitnS59C+Qzc
gbek4us7g1OW9+OaLQ3YgLlCHcDglMyyXtlHSArKgKmcJcUPzSs6Gsh66URA3ACM
AhZzBHahAzdOEy96hJeyQyrniXTK/nmunmgT4pUOA++dQ2c2XbB4m/J/JmjP7PDK
omt+wZ5Zmc7EJNAolni/W6Pn8LbwdO+Fp46Nc2w=
-----END CERTIFICATE-----
subject=CN = cmake.org
issuer=C = US, O = Let's Encrypt, CN = R3
---
5 Likes

Thank you! I do not run CMake org but I'm affected by the issue. I'll take it to CMake maintainers

2 Likes

would that be considered a "429" error?

4 Likes

503, probably. The OCSP stapling implementation should retry.

But most don't. As you can see, most don't even look at what they got to see if it's valid before throwing away their still-probably-good, valid copy.

I've seen Apache return in the stapled response a DNS resolution failure string before...

6 Likes

RFC5685 defines the HTTP Status Code 429 as:

The 429 status code indicates that the user has sent too many
requests in a given amount of time ("rate limiting").

while RFC6960 defines the OCSP response code "tryLater" as:

In the event that the OCSP responder is operational but unable to
return a status for the requested certificate, the "tryLater"
response can be used to indicate that the service exists but is
temporarily unable to respond.

The latter is much more generic than the specific HTTP 429 status code. So in general, no I wouldn't consider them to be the same. OCSP tryLater can mean a variety of serverside issues, while 429 specifically means that a HTTP client should slow down.

6 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.