Nginx error: OCSP responder sent invalid "Content-Type" header

[mastercool]

I constantly get errors like this in my nginx errorlog:

2017/05/03 21:01:27 [error] 27909#27909: OCSP responder sent invalid "Content-Type" header: "text/html; charset=utf-8" while requesting certificate status, responder: ocsp.int-x3.letsencrypt.org

Sniffing with wireshark shows that the failed request tried to get the ocsp response from:

http://ocsp.int-x3.letsencrypt.org/MFMwUTBPME0wSzAJBgUrDgMCGgUABBR%2B5mrncpqz%2FPiiIGRsFqEtYHEIXQQUqEpqYwR93brm0Tm3pkVl7%2FOo7KECEgMv74ZjC%2F%2Fpco4lc9B59eZDSA%3D%3D

The response is an "HTTP/1.0 301 Moved Permanently" which redirects to this location via location header:

/MFMwUTBPME0wSzAJBgUrDgMCGgUABBR+5mrncpqz/PiiIGRsFqEtYHEIXQQUqEpqYwR93brm0Tm3pkVl7/Oo7KECEgMv74ZjC/pco4lc9B59eZDSA==

The supplied Content-Type header is "text/html; charset=utf-8", which makes nginx fail.

If I try this second request using wget I get an HTTP 400 error back.

My errorlog is constantly filling up and I suspect ocsp stapling will stop working for at least one of my domains, once the last valid response times out.

Please help!

[isk]

Can you give us the serial number of the certificate you are trying to validate?

[isk]

Actually, I found what I’m looking for in the URL.

The team is looking into an issue related to this right now, but it has to do with parsing the base64 hash in some rare cases being parsed incorrectly along the way to the OCSP responder. I’ll get back to you as there is more information.

[mastercool]

I managed to identify the certificate in question.
Here is the complete openssl output (maybe that helps):

Certificate:
Data:
Version: 3 (0x2)
Serial Number:
03:2f:ef:86:63:0b:ff:e9:72:8e:25:73:d0:79:f5:e6:43:48
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
Validity
Not Before: Apr 30 16:15:00 2017 GMT
Not After : Jul 29 16:15:00 2017 GMT
Subject: CN = hvggfm.kurswahl-online.de
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (384 bit)
pub:
04:e5:7a:78:53:92:69:fa:36:09:cf:0f:ec:67:30:
df:99:6c:f3:ed:4a:c8:9b:3b:be:22:43:af:05:d3:
4f:0c:59:a5:7e:e6:52:05:26:b2:60:05:a7:c5:51:
7f:c8:aa:3a:50:81:a7:93:16:1c:19:27:6f:f9:80:
4d:e6:89:48:dd:e9:32:25:e7:f6:a4:77:bd:59:04:
97:cd:42:8f:50:8c:c8:cb:83:51:c5:a1:48:c9:29:
13:95:6d:75:5d:c1:a3
ASN1 OID: secp384r1
NIST CURVE: P-384
X509v3 extensions:
X509v3 Key Usage: critical
Digital Signature
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Subject Key Identifier:
48:B8:36:20:74:94:91:CE:D3:0E:4D:8F:A8:BF:92:52:A8:F2:DD:DB
X509v3 Authority Key Identifier:
keyid:A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1

        Authority Information Access: 
            OCSP - URI:http://ocsp.int-x3.letsencrypt.org/
            CA Issuers - URI:http://cert.int-x3.letsencrypt.org/

        X509v3 Subject Alternative Name: 
            DNS:hvggfm.kurswahl-online.de
        TLS Feature: 
            status_request
        X509v3 Certificate Policies: 
            Policy: 2.23.140.1.2.1
            Policy: 1.3.6.1.4.1.44947.1.1.1
              CPS: http://cps.letsencrypt.org
              User Notice:
                Explicit Text: This Certificate may only be relied upon by Relying Parties and only in accordance with the Certificate Policy found at https://letsencrypt.org/repository/

Signature Algorithm: sha256WithRSAEncryption
     7b:e6:95:8a:16:7a:58:8c:c6:7c:ea:39:e7:53:01:9e:a7:e4:
     95:0b:a9:8f:b0:b6:c6:81:13:87:49:21:84:d4:9f:6e:c9:14:
     1b:b0:a1:08:c1:5c:70:23:43:15:8a:5e:05:7d:c2:9a:bd:c6:
     c5:e7:0a:3e:2d:c2:7d:93:f2:7f:fd:a0:f6:e6:0d:ca:37:66:
     a5:f8:c1:9f:2f:90:94:1c:38:a3:e9:a6:9c:2d:cd:52:59:2a:
     83:76:7a:7b:a9:c8:27:46:1c:0b:8c:32:3e:38:e1:87:b3:4b:
     e5:f1:1d:d2:11:ba:45:60:03:4d:9f:8b:08:fc:e1:69:92:ca:
     ec:84:59:a9:02:3f:bd:bb:50:41:24:08:14:5d:7f:ae:7e:f2:
     a2:ef:5c:5e:c0:a4:b4:3b:fd:bf:12:22:2d:7a:41:0b:51:a1:
     85:0a:aa:e8:d1:a9:d8:f3:ba:8d:5c:16:0a:ee:df:20:52:f2:
     37:4f:7a:e7:e0:ce:9f:ce:c0:f0:8f:87:54:11:0a:7b:19:0a:
     2d:8b:9b:00:47:57:1e:89:41:ef:0d:ab:57:1c:69:d2:4f:be:
     a0:a7:db:c3:1b:4e:48:4a:22:b0:0b:79:ce:fe:0c:2b:98:ab:
     d0:db:82:ec:c2:9e:d5:72:92:95:59:c5:d4:1f:bf:84:e8:45:
     ae:f8:c3:ba

[jsha]

Here’s the Boulder bug describing the problem: https://github.com/letsencrypt/boulder/issues/2728. In short, a small number of certificates produce an OCSP request with doubled slashes when base64-encoded. For those certificates, OCSP via POST works, but OCSP via GET currently fails. We’re working on getting a fix out. Thanks for posting!

[mastercool]

Thanks!

I’ll create a new certificate to work around this problem until the real fix is out