Certbot "Exiting abnormally" if CSR uses PrivKey with ec parameters encoded (=param_enc explicit)

lssl · August 10, 2016, 1:54pm

Using manual LE cert generation

    certbot-auto certonly \
    --server https://acme-staging.api.letsencrypt.org/directory \
    --standalone --standalone-supported-challenges tls-sni-01 \
    --csr=./example.com.csr.der ...

If the CSR has been generated with a separately-seeded privkey, with NO explicit params encoded in the key

    openssl ecparam -name secp384r1 -param_enc explicit -outform pem -out ./params.pem
    openssl ecparam -genkey -name secp384r1 -noout -in ./params.pem -out ./privkey.pem

generating a CSR as

    Certificate Request:
        Data:
            Version: 0 (0x0)
            Subject: C=US, ST=NewYork, L=NewYork, O=example.com, OU=example.com, CN=example.com/emailAddress=noreply@example.com
            Subject Public Key Info:
                Public Key Algorithm: id-ecPublicKey
                    Public-Key: (384 bit)
                    pub: 
                        ...
                    ASN1 OID: secp384r1
                    NIST CURVE: P-384
            Attributes:
            Requested Extensions:
                X509v3 Subject Alternative Name: 
                    DNS:example.com
        Signature Algorithm: ecdsa-with-SHA512
             ...

then the LE cert generation is successful

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at
   /tmp/example.com/0001_chain.pem.
   Your cert will expire on 2016-11-06. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot-auto
   again. To non-interactively renew *all* of your certificates, run
   "certbot-auto renew"
 - If you like Certbot, please consider supporting our work by:

But if the CSR has been generated with explicit parameter encoding, and self-seeding,

    openssl ecparam -genkey -name secp384r1 -noout -out ./privkey.pem \
     -param_enc explicit \

so that the generated CSR is

    Certificate Request:
        Data:
            Version: 0 (0x0)
            Subject: C=US, ST=NewYork, L=NewYork, O=example.com, OU=example.com, CN=example.com/emailAddress=noreply@example.com
            Subject Public Key Info:
                Public Key Algorithm: id-ecPublicKey
                    Public-Key: (384 bit)
                    pub: 
                        ...
                    Field Type: prime-field
                    Prime:
                        00:...
                    A:   
                        00:...
                    B:   
                        00:...
                    Generator (uncompressed):
                        04:...
                    Order: 
                        00:...
                    Cofactor:  1 (0x1)
                    Seed:
                        a4:...
            Attributes:
            Requested Extensions:
                X509v3 Subject Alternative Name:
                    DNS:example.com
        Signature Algorithm: ecdsa-with-SHA512
             ...

an error’s returned

    An unexpected error occurred:
    The request message was malformed :: Error parsing certificate request. Extensions in the CSR marked critical can cause this error: https://github.com/letsencrypt/boulder/issues/565
    Please see the logfiles in /var/log/letsencrypt for more details.

where

cat /var/log/letsencrypt/letsencrypt.log

     ...
     2016-08-08 16:33:04,568:DEBUG:acme.client:Received response <Response [400]> (headers: {'Content-Length': '217', 'Boulder-Request-Id': '4...', 'Expires': 'Wed, 08 Aug 2016 16:33:05 GMT', 'Server': 'nginx', 'Connection': 'close', 'Cache-Control': 'max-age=0, no-cache, no-store', 'Pragma': 'no-cache', 'Boulder-Requester': '2...', 'Date': 'Wed, 10 Aug 2016 16:33:05 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': '...'}): '{\n  "type": "urn:acme:error:malformed",\n  "detail": "Error parsing certificate request. Extensions in the CSR marked critical can cause this error: https://github.com/letsencrypt/boulder/issues/565",\n  "status": 400\n}'
    2016-08-10 16:33:04,569:DEBUG:certbot.main:Exiting abnormally:
    Traceback (most recent call last):
      File "/root/.local/share/letsencrypt/bin/letsencrypt", line 11, in <module>
        sys.exit(main())
      File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/certbot/main.py", line 744, in main
        return config.func(config, plugins)
      File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/certbot/main.py", line 558, in obtain_cert
        _csr_obtain_cert(config, le_client)
      File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/certbot/main.py", line 527, in _csr_obtain_cert
        certr, chain = le_client.obtain_certificate_from_csr(config.domains, csr, typ)
      File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/certbot/client.py", line 228, in obtain_certificate_from_csr
        authzr)
      File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/acme/client.py", line 319, in request_issuance
        headers={'Accept': content_type})
      File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/acme/client.py", line 656, in post
        return self._check_response(response, content_type=content_type)
      File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/acme/client.py", line 572, in _check_response
        raise messages.Error.from_json(jobj)
    Error: urn:acme:error:malformed :: The request message was malformed :: Error parsing certificate request. Extensions in the CSR marked critical can cause this error: https://github.com/letsencrypt/boulder/issues/565

The referenced bug

**_Golang errors on extensions marked critical #565_**
[https://github.com/letsencrypt/boulder/issues/565](https://github.com/letsencrypt/boulder/issues/565)

addresses parsing problems, but is Golang, not python related, and doesn’t mention the params encoding specifically.

Is there a specific reason that privkey parameter encoding is DISALLOWED in LE, or is this (still) a bug?

cool110 · August 10, 2016, 3:48pm

The CA/B Baseline Requirements state that ECC certs must use the NIST P‐256, P‐384, or P‐521 curves.

lssl · August 10, 2016, 4:02pm

The CA/B Baseline Requirements state that ECC certs must use the NIST P‐256, P‐384, or P‐521 curves.

Ok, and what point are you making?

As you can see from my OP

-name secp384r1

where

openssl ecparam -list_curves | grep secp384r1
  secp384r1 : NIST/SECG curve over a 384 bit prime field

and at https://www.ietf.org/rfc/rfc5480.txt

2.1.1.1.  Named Curve

   ...
The NIST-named curves are:
   ...
     -- Note that [FIPS186-3] refers to secp192r1 as P-192, secp224r1 as
     -- P-224, secp256r1 as P-256, secp384r1 as P-384, and secp521r1 as
     -- P-521.
   ...

i.e. 'secp384r1' == 'P-384'

cpu · August 10, 2016, 4:27pm

This seems like it might be an issue with the upstream Golang x509 package that Boulder uses to process CSRs.

I created a CSR (test.ecparam.csr) following your instructions and used it with this test program:

package main

import (
	"crypto/x509"
	"fmt"
	"io/ioutil"
)

func main() {

	bytes, _ := ioutil.ReadFile("test.ecparam.csr")

	req, err := x509.ParseCertificateRequest(bytes)

	if err != nil {
		fmt.Printf("Err: %s\n", err)
	}

	fmt.Printf("Req: %#v\n", req)
}

On go version go1.6.2 linux/amd64 this produces:

Err: asn1: structure error: tags don't match (16 vs {class:0 tag:13 length:45 isCompound:true}) {optional:false explicit:false application:false defaultValue:<nil> tag:<nil> stringType:0 timeType:0 set:false omitEmpty:false} certificateRequest @2
Req: (*x509.CertificateRequest)(nil)

lssl · August 10, 2016, 4:30pm

@cpu

This seems like it might be an issue with the upstream Golang x509 package that Boulder uses to process CSRs.

So the python error from logs is simply a red-herring here?

cpu · August 10, 2016, 4:37pm

I agree the text is a little bit vague/misleading. Certbot is informing you of the error response from Boulder, generated because it was unable to parse the CSR.

The upstream x509 library does not support explicit parameter encoding and regardless of whether it is added there we likely won’t support it in Boulder.

Is there a specific use case that is pushing you towards using explicit encoding that can’t be met in another fashion?

lssl · August 10, 2016, 4:52pm

@cpu

I agree the text is a little bit vague/misleading.

I'd suggest it's a bit more than that ... there's simply no indication of it NOT being a python problem. In any case, the 'log output' is generally not helpful -- other than, of course, the "maybe go see this bug" test.

Is there a specific use case that is pushing you towards using explicit encoding

The use case is simply that defined by my standard workflow/process which includes

no separate ec params generation and storage
parameter encoding in the key to accommodate Openssl v < (iirc) 1.0.2 curve usage.

I'm already having to modify my processes to accommodate LE since neither EC automation nor parallel-algo cert generation are currently available via certbot, and have to be done manually.

that can't be met in another fashion?

ec param inclusion in the key is perfectly legitimate usage, and should be supported, or at the very least not cause breakage of this sort.

of course just about anything can be worked-around. That's not IMO that's not a valid rationale for unnecessary breakage.

cpu · August 10, 2016, 5:08pm

You could file a feature request with the Golang project. Support in the x509 package would be a prerequisite for any discussion on support from the Boulder side.

I'm sorry we can't support your existing workflow without modification.

TCM · August 10, 2016, 5:15pm

That's just not true. From the very first line of the log it's clear that it's a server problem:

2016-08-08 16:33:04,568:DEBUG:acme.client:Received response <Response [400]> (headers: {'Content-Length': '217', 'Boulder-Request-Id': '4...', 'Expires': 'Wed, 08 Aug 2016 16:33:05 GMT', 'Server': 'nginx', 'Connection': 'close', 'Cache-Control': 'max-age=0, no-cache, no-store', 'Pragma': 'no-cache', 'Boulder-Requester': '2...', 'Date': 'Wed, 10 Aug 2016 16:33:05 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': '...'}): '{\n  "type": "urn:acme:error:malformed",\n  "detail": "Error parsing certificate request. Extensions in the CSR marked critical can cause this error: https://github.com/letsencrypt/boulder/issues/565",\n  "status": 400\n}'

lssl · August 10, 2016, 5:40pm

You could file a feature request with the Golang project

github.com/golang/go

crypto/x509: missing support for EC parameter encoding

opened 05:39PM - 10 Aug 16 UTC

closed 09:47PM - 10 Aug 16 UTC

ghost

NeedsDecision FrozenDueToAge

LetsEncrypt's "Boulder" server, the backend for processing CSRs, apparently uses… ``` go1.6.2 linux/amd64 ``` and specifically ``` the upstream Golang x509 package ... to process CSRs. ``` If a CSR submitted to LetsEncrypt service references a PrivKey that has ec parameters explicitly encoded, e.g., ``` openssl ecparam \ -genkey \ -param_enc explicit \ ... ``` , perfectly valid under Openssl, the Boulder server process fails with a server error ``` Err: asn1: structure error: tags don't match (16 vs {class:0 tag:13 length:45 isCompound:true}) {optional:false explicit:false application:false defaultValue:<nil> tag:<nil> stringType:0 timeType:0 set:false omitEmpty:false} certificateRequest @2 Req: (*x509.CertificateRequest)(nil) ``` This is claimed as a result of GoLang's X509 pkg lacking support ``` The upstream x509 library does not support explicit parameter encoding ``` Despite their befuddling insistence that ``` regardless of whether it is added there we likely won't support it in Boulder. ``` it'll be useful for GoLang's X509 pkg to implement feature parity with Openssl in supporting the explicit parameter encoding More detail's provided in the original bug post there **_Certbot “Exiting abnormally” if CSR uses PrivKey with ec parameters encoded (=param_enc explicit)_** [https://community.letsencrypt.org/t/certbot-exiting-abnormally-if-csr-uses-privkey-with-ec-parameters-encoded-param-enc-explicit/18814](https://community.letsencrypt.org/t/certbot-exiting-abnormally-if-csr-uses-privkey-with-ec-parameters-encoded-param-enc-explicit/18814) I can provide additional info here as requested

From the very first line of the log it's clear that it's a server problem:

If it was 'clear' what the problem was, it wouldn't be a question. We'll just have to disagree on what 'clear' is.

jsha · August 10, 2016, 6:01pm

Sounds like this (native Certbot support) is the best path forward, and the work is already in progress. Thanks for the input!

lssl · August 10, 2016, 6:07pm

Sounds like this (native Certbot support) is the best path forward

Depends on what that native certbot support will do. That's unclear,

and

the work is already in progress.

has no current ETA

TCM · August 12, 2016, 9:15pm

If you know anything about HTTP, "Response 400" is self-explanatory. So there's the possibility that it's crystal clear but still a question, in which case you shouldn't be making such definitive statements:

system · September 11, 2016, 9:15pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Is the private key generated by certbot? Help	5	931	July 12, 2023
Certbot command you use to renew or generate cert by passing my own CSR? Help	4	11548	July 3, 2020
Certbot-auto why generate multiple certificate server key folder Server	20	5135	February 20, 2019
Creating parallel ECDSA & RSA keys -- same CSR, just with add'l cmmd line params? or NEW, re-keyed csr? Issuance Tech	5	3744	September 5, 2016
Certificate with must-staple extension Help	15	1349	March 22, 2022

Certbot "Exiting abnormally" if CSR uses PrivKey with ec parameters encoded (=param_enc explicit)

Related topics