Status 400: Error unmarshaling certificate request

[tpenner]

I am getting Error unmarshaling certificate request with a HTTP status of 400 from my request to /acme/new-cert on the staging server.

Here is the unsigned CSR object:

{"resource":"new-cert","csr":"-----BEGIN CERTIFICATE REQUEST-----MIICtjCCAZ4CAQAwcTEcMBoGA1UEAwwTbWFjcHJvLnRwLnVzLjRkLmNvbTELMAkGA1UEBhMCVVMxETAPBgNVBAcMCFNhbiBKb3NlMRMwEQYDVQQIDApDYWxpZm9ybmlhMRwwGgYDVR0RDBNtYWNwcm8udHAudXMuNGQuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtrJv07xq9MlKiab7BZeJM6u8FO8quFH0UYIidfu0PVSSaH5RTPg6Nb5xw/2+V9RsveQZdNb3ViSI+4jRoVblIrf9UUe6OiVnDu2UyEPLIRfNHCXjIsxJ5jBSpS1FFN6K4gr3wzCwSr8gitQOWnRShel/DUZfgbaETShv9X/LFFPcbj5J8PrYPSiu67+J04sblTfHAJ+fjzJ4YwtLI//4OOYZMV+R2hflt+dcEx/+hF4DpBYrfNBhZBSLLwCKsbFz6nJYlcX4vVJt3rXySAR+2W3qEACxr6QZGsX1FDKM0A1AcfR14M15gUWoEMivXrgUAimvD8JNAzvGOEqWalxIhQIDAQABoAAwDQYJKoZIhvcNAQELBQADggEBAJJEB8rtEy/JxgVj+9Xl12aYhfI+EvsjCNA7US0P+Kl6W/jo1AKoHyE0UGkmKgbjPuy+uDCD60a7M4jRu5kIyY99ig2gIUbdwwXzeh/6zmtyMi/mgrkC31MjNTPvSElWPjP2NOmpQp5pRKx2mf94PedGqhkHzSNcDBXBsgZS8ws47Ai03xi5L0UuBByowRJxk9By146kdliYrbTEAGCC7YM5Rco2GTUzyIM5OpyHenrbCla+EtancNLVBX0u5UtKbA+nk78KlvKtLMr0T62pPiey9Hs1IurAwGtk+V2IifNlMgfXSsj/nhQyqnUlLfdxxGzH0snmik/jhl9TzktpmNE=-----END CERTIFICATE REQUEST-----"}

Here is the signed request that is being sent to /acme/new-cert on the staging server:

{"header":{"alg":"RS256","jwk":{"kty":"RSA","n":"rDBsgp0J3jNUF3zqYLcCH6Tt9abhoU8ZRUvLz6_JZFySRKVRs8ft8VnKQtsB4yUOan3UHGLqYsK0GMHW6ogruymZld4ZUnaqKM22-8CL97gzJC3LNQhVzQZJxcMOHXh-X-qYrDakqfkZC3tPBVsSg7PQeN-K-Pn2Rl7kPTrIFYULzNVFQvg5sFeEzXcDxrG1h7MycdqstJRZmyyT5auH36RXaDNA4yh6hDzleoMepkOV520Mgu8nMGkGoyyFY6WZFXZV7h6DVcrXu3JaFikR4vXwiwfiaqX0NivAeLA4CM0HhS18hFy1HHgDr_vw37YcEIbSNqUu9-zJEIW_VZhjDw","e":"AQAB"}},"protected":"eyJhbGciOiJSUzI1NiIsImp3ayI6eyJrdHkiOiJSU0EiLCJuIjoickRCc2dwMEozak5VRjN6cVlMY0NINlR0OWFiaG9VOFpSVXZMejZfSlpGeVNSS1ZSczhmdDhWbktRdHNCNHlVT2FuM1VIR0xxWXNLMEdNSFc2b2dydXltWmxkNFpVbmFxS00yMi04Q0w5N2d6SkMzTE5RaFZ6UVpKeGNNT0hYaC1YLXFZckRha3Fma1pDM3RQQlZzU2c3UFFlTi1LLVBuMlJsN2tQVHJJRllVTHpOVkZRdmc1c0ZlRXpYY0R4ckcxaDdNeWNkcXN0SlJabXl5VDVhdUgzNlJYYUROQTR5aDZoRHpsZW9NZXBrT1Y1MjBNZ3U4bk1Ha0dveXlGWTZXWkZYWlY3aDZEVmNyWHUzSmFGaWtSNHZYd2l3ZmlhcVgwTml2QWVMQTRDTTBIaFMxOGhGeTFISGdEcl92dzM3WWNFSWJTTnFVdTktekpFSVdfVlpoakR3IiwiZSI6IkFRQUIifSwibm9uY2UiOiJhaHZZMHhQVGlWS2J1ZV9wcWNQNFlHWUhsLXhxRFRHT3laaWZUZG1DTW5vIn0","payload":"eyJyZXNvdXJjZSI6Im5ldy1jZXJ0IiwiY3NyIjoiLS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS1NSUlDdGpDQ0FaNENBUUF3Y1RFY01Cb0dBMVVFQXd3VGJXRmpjSEp2TG5Sd0xuVnpMalJrTG1OdmJURUxNQWtHQTFVRUJoTUNWVk14RVRBUEJnTlZCQWNNQ0ZOaGJpQktiM05sTVJNd0VRWURWUVFJREFwRFlXeHBabTl5Ym1saE1Sd3dHZ1lEVlIwUkRCTnRZV053Y204dWRIQXVkWE11TkdRdVkyOXRNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXRySnYwN3hxOU1sS2lhYjdCWmVKTTZ1OEZPOHF1RkgwVVlJaWRmdTBQVlNTYUg1UlRQZzZOYjV4dy8yK1Y5UnN2ZVFaZE5iM1ZpU0krNGpSb1ZibElyZjlVVWU2T2lWbkR1MlV5RVBMSVJmTkhDWGpJc3hKNWpCU3BTMUZGTjZLNGdyM3d6Q3dTcjhnaXRRT1duUlNoZWwvRFVaZmdiYUVUU2h2OVgvTEZGUGNiajVKOFByWVBTaXU2NytKMDRzYmxUZkhBSitmanpKNFl3dExJLy80T09ZWk1WK1IyaGZsdCtkY0V4LytoRjREcEJZcmZOQmhaQlNMTHdDS3NiRno2bkpZbGNYNHZWSnQzclh5U0FSKzJXM3FFQUN4cjZRWkdzWDFGREtNMEExQWNmUjE0TTE1Z1VXb0VNaXZYcmdVQWltdkQ4Sk5BenZHT0VxV2FseEloUUlEQVFBQm9BQXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBSkpFQjhydEV5L0p4Z1ZqKzlYbDEyYVloZkkrRXZzakNOQTdVUzBQK0tsNlcvam8xQUtvSHlFMFVHa21LZ2JqUHV5K3VEQ0Q2MGE3TTRqUnU1a0l5WTk5aWcyZ0lVYmR3d1h6ZWgvNnptdHlNaS9tZ3JrQzMxTWpOVFB2U0VsV1BqUDJOT21wUXA1cFJLeDJtZjk0UGVkR3Foa0h6U05jREJYQnNnWlM4d3M0N0FpMDN4aTVMMFV1QkJ5b3dSSnhrOUJ5MTQ2a2RsaVlyYlRFQUdDQzdZTTVSY28yR1RVenlJTTVPcHlIZW5yYkNsYStFdGFuY05MVkJYMHU1VXRLYkErbms3OEtsdkt0TE1yMFQ2MnBQaWV5OUhzMUl1ckF3R3RrK1YySWlmTmxNZ2ZYU3NqL25oUXlxblVsTGZkeHhHekgwc25taWsvamhsOVR6a3RwbU5FPS0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLSJ9","signature":"cgpDGOeaWCijlyS7u2vtp424U1sLT3zBeU5K2u5XRNtxfiNmrNyE0L4hZ8FbuXG2pDDOZd5lV5W18JY-JDBEz72nQzQmxGb21qxgXcAumb8hAiBoayZpagAAliO6U-dj3xABR2hEIuXPQ_x7MZ5FbXjVZPhBz2ljOpSSDVuecY6PoKAvt8R2d9gL5mTqKcR8sX-9Z-b-RErasafTtOXzXSxPgkljC5PBBWw4jRMlj6szUR7BC7gwrJyI2mcVRFwjm0YH0aU7-MHugLSjW5AlVd-w_StF9cOHWGZ6Ih98hTh51kNJoywzSa0H_KXRKHeChqDxcxNeBKmEL2HrjBQ2QA"}

The response i get from the staging server is:

{
"type": "urn:acme:error:malformed",
"detail": "Error unmarshaling certificate request",
"status": 400
}

I read that this could be related to including the OrgName in the CSR, so I made sure to remove the organization name from the CSR but i still see this error.

I also read this could be related to omitting the subjectAltName in the CSR, so i made sure to include the subjectAltName in my CSR and have confirmed my CSR using certLogik’s online tool but i still see this error.

I am NOT using EC keys.

I am only specifying the following elements in my CSR:

CommonName = same FQDN used for the domain verification
Country = US
locality name = San Jose
State or Province = California
subjectAltName = same as CommonName

Could this be related to Boulder/issue#565? I am not (knowingly) setting any extensions as critical…

What else should i check?

[tialaramex]

I’m not sure it’s relevant to your problem but Let’s Encrypt can’t sign any of the details in your proposed subject other than the Common Name (assuming it matches a SAN DNS name). Public CAs are forbidden from signing things they can’t verify and Let’s Encrypt doesn’t verify anything except DNS names.

[tpenner]

are you saying i should remove country, state, and locality?

[tialaramex]

As I said I’m not at all sure it’s the cause of your problem, but I know Let’s Encrypt can’t sign those things, I think it will ignore them, but if you’re looking to rule things out…

[tpenner]

ok, thanks for the suggestion, i like narrowing things down and ruling stuff out.

I removed Country, Locality, and State.
Leaving only the CommonName and subjectAltName

But i still get the same error…

[schoen]

I believe according to the ACME spec your “csr” shouldn’t be a PEM CSR (with the -----BEGIN CERTIFICATE REQUEST----- stuff) and should be encoded with the JWK-style base64 instead of the traditional base64. Have you checked on those aspects?

[tpenner]

hmm, not in PEM format you say; i think you may be onto something here. sorry for overlooking that aspect of the spec. I am pretty sure my CSR is in PEM, not in DER format as the spec says it should be. I will look into converting it to DER and try again.

As for the JWK-style base base64url encoding, should i apply that to the CSR prior to signing the request?

[schoen]

Take a look at the ACME spec:

csr (required, string): A CSR encoding the parameters for the certificate being requested [RFC2986]. The CSR is sent in the Base64url-encoded version of the DER format. (Note: This field uses the same modified Base64 encoding rules used elsewhere in this document, so it is different from PEM.)

RFC 8555: Automatic Certificate Management Environment (ACME)

I think that's the issue here. (The use of base64 and its variants sure make these fields look like PEM data -- but they aren't!)

[schoen]

@tpenner

As for the JWK-style base base64url encoding, should i apply that to the CSR prior to signing the request?

Yes for JWK-level signing for the ACME protocol; no for CSR-internal signing when constructing the CSR object (which would probably be impossible anyway).

[tpenner]

I have been unable to determine if my CSR is PEM or DER format; the docs for the environment i am using show that the CSR is in PKCS format.

Also, my version of openssl is unable to validate the information from my CSR, even though i can validate it online at https://certlogik.com/decoder/

So I attempted to base64URLencode my CSR prior to signing the JWK, just to see if this has any effect…

Basically i encoded the csr property of this JSON:

{"resource":"new-cert","csr":"MIICyTCCAbECAQAwgYMxHDAaBgNVBAMME21hY3Byby50cC51cy40ZC5jb20xCzAJBgNVBAYTAlVTMREwDwYDVQQHDAhTYW4gSm9zZTETMBEGA1UECAwKQ2FsaWZvcm5pYTEcMBoGA1UdEQwTbWFjcHJvLnRwLnVzLjRkLmNvbTEQMA4GA1UECgwHNEQsIEluYzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALayb9O8avTJSomm+wWXiTOrvBTvKrhR9FGCInX7tD1Ukmh+UUz4OjW+ccP9vlfUbL3kGXTW91YkiPuI0aFW5SK3/VFHujolZw7tlMhDyyEXzRwl4yLMSeYwUqUtRRTeiuIK98MwsEq/IIrUDlp0UoXpfw1GX4G2hE0ob/V/yxRT3G4+SfD62D0oruu/idOLG5U3xwCfn48yeGMLSyP/+DjmGTFfkdoX5bfnXBMf/oReA6QWK3zQYWQUiy8AirGxc+pyWJXF+L1Sbd618kgEftlt6hAAsa+kGRrF9RQyjNANQHH0deDNeYFFqBDIr164FAIprw/CTQM7xjhKlmpcSIUCAwEAAaAAMA0GCSqGSIb3DQEBCwUAA4IBAQAuOIr81KvmsdkaKeni7j53ZDp0q17ieY7wk9IV6m2wyU0po5OUzaCJJ64N7YH8rCVe/114CZuMz1QX7jPuf7+cNrk2UGOdSu6oTb9YLj2Yxm1X+cEN2zIHGuD7Xu0Fjacq6owgHvcaJ/HcM8e97By6YOZ69+GqiVoeYKdW6+UZDMagxl62meb10FLP08ebXaVDHdaDpciVuZPeEtE8yxRaUb6rVk622Uo0X8lbfn48Xzix7iHgyN/05wIiitatnX7QEGAGvJZZ8C+YDpi9dDmTVr6U1Fl9J+YCbNihI3LRJyvMKxjnAMEnbKCvFtaNBP912+y+B4iB/5QrkgLZQi7V"}

So that it looks like this:

{"resource":"new-cert","csr":"TUlJQ3lUQ0NBYkVDQVFBd2dZTXhIREFhQmdOVkJBTU1FMjFoWTNCeWJ5NTBjQzUxY3k0MFpDNWpiMjB4Q3pBSkJnTlZCQVlUQWxWVE1SRXdEd1lEVlFRSERBaFRZVzRnU205elpURVRNQkVHQTFVRUNBd0tRMkZzYVdadmNtNXBZVEVjTUJvR0ExVWRFUXdUYldGamNISnZMblJ3TG5WekxqUmtMbU52YlRFUU1BNEdBMVVFQ2d3SE5FUXNJRWx1WXpDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTGF5YjlPOGF2VEpTb21tK3dXWGlUT3J2QlR2S3JoUjlGR0NJblg3dEQxVWttaCtVVXo0T2pXK2NjUDl2bGZVYkwza0dYVFc5MVlraVB1STBhRlc1U0szL1ZGSHVqb2xadzd0bE1oRHl5RVh6UndsNHlMTVNlWXdVcVV0UlJUZWl1SUs5OE13c0VxL0lJclVEbHAwVW9YcGZ3MUdYNEcyaEUwb2IvVi95eFJUM0c0K1NmRDYyRDBvcnV1L2lkT0xHNVUzeHdDZm40OHllR01MU3lQLytEam1HVEZma2RvWDViZm5YQk1mL29SZUE2UVdLM3pRWVdRVWl5OEFpckd4YytweVdKWEYrTDFTYmQ2MThrZ0VmdGx0NmhBQXNhK2tHUnJGOVJReWpOQU5RSEgwZGVETmVZRkZxQkRJcjE2NEZBSXBydy9DVFFNN3hqaEtsbXBjU0lVQ0F3RUFBYUFBTUEwR0NTcUdTSWIzRFFFQkN3VUFBNElCQVFBdU9JcjgxS3Ztc2RrYUtlbmk3ajUzWkRwMHExN2llWTd3azlJVjZtMnd5VTBwbzVPVXphQ0pKNjRON1lIOHJDVmUvMTE0Q1p1TXoxUVg3alB1ZjcrY05yazJVR09kU3U2b1RiOVlMajJZeG0xWCtjRU4yeklIR3VEN1h1MEZqYWNxNm93Z0h2Y2FKL0hjTThlOTdCeTZZT1o2OStHcWlWb2VZS2RXNitVWkRNYWd4bDYybWViMTBGTFAwOGViWGFWREhkYURwY2lWdVpQZUV0RTh5eFJhVWI2clZrNjIyVW8wWDhsYmZuNDhYeml4N2lIZ3lOLzA1d0lpaXRhdG5YN1FFR0FHdkpaWjhDK1lEcGk5ZERtVFZyNlUxRmw5SitZQ2JOaWhJM0xSSnl2TUt4am5BTUVuYktDdkZ0YU5CUDkxMit5K0I0aUIvNVFya2dMWlFpN1Y”}

Which is then signed in a JWK style request like i did for the validation processes.

After doing this I now get the following error:

{
  "type": "urn:acme:error:malformed",
  "detail": "Error parsing certificate request. Extensions in the CSR marked critical can cause this error: https://github.com/letsencrypt/boulder/issues/565",
  "status": 400
}

So this makes me wonder - which of these two is expected (base64urlencoded(csr) or just csr) inside of the JSON object.

I am still searching for a solution to this CSR issue.

[schoen]

Your CSR was in PEM format, not DER format. A CSR whose byte values are all alphanumeric plus ASCII punctuation – like yours beginning in “MIIC” above – is going to be PEM-encoded.

What you should probably do given the representation of the CSR you have is akin to base64urlencode(base64decode(csr)).

[tpenner]

Thank you so much @schoen this is precisely what i needed!