Thanks for the tip. I was already safeguarding against potential serial conflicts between CAs on my data-storage, but did not think about an account mismatch affecting the result.
5 Likes
New “replaces” field in the Order object: We have extended the Order object to include a "replaces" field. This field identifies the certificate being replaced by a new order. Our goal is to use this field to exempt renewals within ARI suggested renewal windows from rate limits, thereby encouraging further ARI adoption.
I like this idea in practice but not if it could lead directly to a failure that would otherwise be caught by just "starting again", I don't use replaces in Implemented HTTP-01 with ARI Extension in Javascript I am just choosing a random time within the window to create a new certificate.
2 Likes
Copying my comment from the github issue here, so that folks here aren't led astray:
The library is returning the whole authorityKeyIdentifier extension. ARI wants the keyIdentifier field of that extension. The proper thing to do is to parse those bytes as ASN.1 and access the field with tag 0. Simply discarding the first four bytes is likely to work most of the time in practice, but is by no means guaranteed.
5 Likes
FYI, the relevant Python code for this is (including a fallback to @orangepizza's hack if the asn1 package is not installed:
The "easier" way to parse all this information is to construct an OCSP response and copy stuff over, but that requires having access to the Intermediate certificate handy. If you just have the Leaf Certificate, it's a decoding party.
try:
# "asn1", # https://github.com/andrivet/python-asn1 == pypi/ansi
import asn1
except ImportError:
asn1 = None
for i in range(0, cert.get_extension_count()):
_ext = cert.get_extension(i)
if _ext.get_short_name() == b"authorityKeyIdentifier":
# strip the first 4 bytes BECAUSE (certbot pr info below)
# by nature of asn1 encoding single member sequence
# we can strip first 4 bytes to get akid
# seq/len/octetstring/len
if asn1:
log.debug("asn1 available")
_akid = _ext.get_data()
# build a decoder
decoder = asn1.Decoder()
# decode the payload
decoder.start(_akid)
_decoded_a = decoder.read() # tag + payload
# decode the inner payload
decoder.start(_decoded_a[1])
_decoded_b = decoder.read() # tag + payload
akid = _decoded_b[1]
else:
log.debug("asn1 unavailable; using hack")
akid = _ext.get_data()[4:]
break
5 Likes
For what it's worth, that approach still has a bug. It's certainly better than just dropping the first four bytes -- if one of the extension's fields was surprisingly long, then the length indicator might be more than one byte long, and this new approach handles that correctly. But if the Authority Key Identifier extension contains one or more of its other possible fields (authorityCertIssuer or authorityCertSerialNumber) instead of the keyIdentifier field, then this method may still return the wrong thing. Ideally you'd use this approach, but after getting _decoded_b you'd check that the tag there is 0x80 (context-specific field zero). In the WebPKI, that extension should only ever have the keyIdentifier field, but thorough/defensive programming is always valuable.
5 Likes
It's worth a lot. Thanks for picking that up!
5 Likes
Yeah I was saying in my post yesterday that those 4 bytes is actually a sequence with the information about the length of the AKI field, but I edited it out.
It should be safe to remove those 4 bytes unless the length of the AKI changes significantly and I don't see any reason why It would.
I have shown a clear example of how to manually parse the sequence correctly with decodeAKI with the sequence being talked about being called seq7
If you are parsing ASN.1 notation correctly then 0x30 in the first position is always a sequence and the next byte or bytes is the length, In this implementation of readASN1Length you pass the offset of the next byte after a TAG and it returns the length of the data and the lengthOfLength which is the number of bytes to skip
BTW is there defined way to guess if error returned for order is about ARI and likely succeed if requested without one?
3 Likes
Section 5 of the current draft-07 requires that servers must respond with an HTTP 409 (Conflict) error if the cert specified by replaces has already been replaced. But it also talks about rejecting requests for other reasons that may be implementation specific. I've seen Boulder return HTTP 404 when the cert specified couldn't be found.
But short of interpreting the text of the error body, I don't think there's going to be a definitive list of error codes that indicate you should re-try without the replaces field. My client is currently only looking for 409 and 404 errors and then retrying without the replaces field regardless of the error body. If I come across any additional errors in the future, I may just change things so that any non-success response gets retried without the replaces field.
At the end of the day, avoiding rate limits is nice, but not at the expense of forcing administrative intervention.
5 Likes
I've tried to design server-ssl in a way where you should never hit the rate limit, I think using the ARI Window without replaces is sufficient enough to avoid traffic spikes and stuff.
I check the window often but only actually generate certificates when its time to do so.
and another one is requesting account it not one requested old certificate: think most other errors can be avoided by requesting ARI info of old certificate from the CA and only use ARI if server finds it, this one can't as ARI doesn't talk about what account it requested by. I think most clients likely retry any order with ARI without it at any error, but that isn't good status que.
2 Likes
Most clients manage the "account" internally and this is not likely to change but I have thought about that.
You could use openssl to check the public key of the certificate signing request against the certificate, but that isn't ARI
1 Like
You can check this with openssl or by comparing the public keys some other way, eg.
# Step 1: Extract public key from CSR
openssl req -in your_csr_file.csr -noout -pubkey -out public_key.pem
# Step 2: Extract public key from certificate
openssl x509 -in your_certificate.crt -noout -pubkey -out cert_public_key.pem
# Step 3: Compare the two public keys
diff public_key.pem cert_public_key.pem
If you see no output from the diff command, it means the public keys match, and the certificate corresponds to the CSR. If there are differences, the accounts do not match.
that's about certificate's key, certificate doesn't say what acme account was requested it
3 Likes
take this example, one set is your "account" and the other set is the keys you signed the CSR with, you should keep track of which "account" used which "signing keys" so you can connect the dots so to speak.
I agree with you, maybe some kind of thumbprint could be added, just trying to be helpful.
To me, those are really weird names..
1 Like
One can track a SPKI Hash to quickly cross-reference all these related keys and applications.
AccountKey and PrivateKey SPKIs are for the key itself.
CSR and Cert SPKIs are for the PrivateKey.
5 Likes
I've done some testing and figured I'd circle back on this for the benefit of others.
First, I constructed an edge-case cert, attached, that roughly mimics the AppendixA cert. The important bit is that it contains BOTH the authorityCertIssuer and authorityCertSerialNumber fields. The Python crypto library requires BOTH to be present when constructing a cert; I'm not sure if that is RFC compliant or not. I was hoping to test against all possible permutations, but this will suffice. I suggest others test against this cert, or build your own!
Learnings:
1- I'm not sure if this is due to implementation details of Python's Cryptography or the ASN1 package, but I have NOT been able to get a malformed value in there; the code above does seem to always be correct. I haven't had much time to traceback/debug into those packages.
2- Simply hacking away the first 4 bytes will indeed break. I'll note this in the certbot PR
3- regexing the data out of openssl textual output also does seem to work well across a handful of versions I have. The keyid seems to always be first. I need to test against more versions though.
-----BEGIN CERTIFICATE-----
MIIC3zCCAcegAwIBAgIFAIdlQyEwDQYJKoZIhvcNAQELBQAwFTETMBEGA1UEAwwK
RXhhbXBsZSBDQTAeFw01MDAxMDEwMDAwMDBaFw01MDAxMDEwMDAwMDBaMBYxFDAS
BgNVBAMMC2V4YW1wbGUuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC
AQEAy3VnrpBZT8VcByJKAu4zIrAVu14cRlipEuQMHSe9Psa7vvXzoJQD6Uc4/mwz
o7m+3yaRM8eSQWSc4AraBfqkRfSNP+wwygVqfbHcWOe9/v3uai1IfCKT+WryszP3
ahN/ceMqRc1axukk7bO5KeLlj4/wUGgq+zYhAF4ZJcolwwD4M8qUTgplKWLzhpZz
BbJlvQCi2sAkQDH0d7hice56BBY2X7d35T65MAeeVKFxXOWN59HMgitLCDK0E9Mf
GJJHriSB3iW39zNiasyTupZFhrNS8G/5NGd0oqAlkkj/Jb7cX9w0RfCJw5sEfpL6
lDtqgF0VyfRhSYdEzYAMoSaeGwIDAQABozUwMzAxBgNVHSMEKjAogBRpiFtrh0ZA
QeGze4R7oK4s3gHI1KENhgtFWEFNUExFLkNPTYIBATANBgkqhkiG9w0BAQsFAAOC
AQEAchL9etTHAaaJ1h1xqbIR7dL7QcHodSL8tGgsWww0/g+XqcRy/Y9TJV1zLhJs
UnrPC7mWRGsn+aTD/VBJwd+K6i7gbQ5WDvY+AClFUTJNtwOxT+hZxm0KdkWXfRJc
ciejEynaxypoZy4i73uM9inPea8mDKwdOD3ZTDAJqVCBGyJN1BRqVCte8aWNA/ls
htw6KK8Uusn6mGBuEw+dYHhN/T+1ouKiYYYAoeDU7wq+qKdNb1wcBeQGOCro2C9K
4izzQ1FXpYc+wh8MceTtYDIrI+Wszhwb/S/1K5uD1sJ9F033gzNj8e61dMQtdPr9
xcE1fgyXJnOFsRliMCw6R7IsDA==
-----END CERTIFICATE-----
3 Likes
Yes, the way the AuthorityKeyIdentifier extension works is that you either present a keyIdentifier, or you present both a authorityCertIssuer and an authorityCertSerialNumber. Basically, you're either uniquely identifying your issuer by key (since a key is the real CA, and may be represented by multiple certs like cross-signs), or you're uniquely identifying your issuer by name+number.
2 Likes
authorityKeyIdentifier field has the OID of 2.5.29.35
so long as you extract by OID you should always get the correct value
Here is an example of how to reliably extract the AKI from a certificate that has been turned into a buffer
export function decodeAKI(certBuffer) {
if (certBuffer[0] != TAGS.SEQUENCE) { // Cert Starts with SEQUENCE
return undefined;
}
let offset = 1;
const seq1 = readASN1Length(certBuffer, offset); // Certificate SEQUENCE (3 elem)
if (seq1 === undefined) {
return undefined;
}
offset += seq1.lengthOfLength + 2;
const seq2 = readASN1Length(certBuffer, offset); // TBSCertificate SEQUENCE (8 elem)
if (seq2 === undefined) {
return undefined;
}
offset += seq2.lengthOfLength + 2;
if (certBuffer[offset - 1] !== TAGS.CONTEXT_SPECIFIC_ZERO) { // version [0] (1 elem)
return undefined;
}
offset += certBuffer[offset] + 2;
if (certBuffer[offset - 1] != 0x02) {
return undefined;
}
offset += certBuffer[offset] + 2;
while (certBuffer[offset - 1] !== TAGS.CONTEXT_SPECIFIC_THREE) { // skip to extensions [3] (1 elem)
const skipSequences = readASN1Length(certBuffer, offset);
if (skipSequences === undefined) {
return undefined;
}
offset += skipSequences.length + skipSequences.lengthOfLength + 1;
}
const seq5 = readASN1Length(certBuffer, offset); // extensions [3] (1 elem)
if (seq5 === undefined) {
return undefined;
}
offset += seq5.lengthOfLength + 2;
const seq6 = readASN1Length(certBuffer, offset); // Extensions SEQUENCE (9 elem)
if (seq6 === undefined) {
return undefined;
}
offset += seq6.lengthOfLength + 2;
let inner1;
while (true) { // List of Extension SEQUENCE (3 elem)
const seq7 = readASN1Length(certBuffer, offset);
if (seq7 === undefined) {
return undefined;
}
inner1 = certBuffer.slice(offset + 1, offset + 1 + seq7.length);
if (walkExtensions(inner1)) { // find extnID OBJECT IDENTIFIER 2.5.29.35 authorityKeyIdentifier (X.509 extension)
offset += seq7.lengthOfLength + 2;
break;
}
else {
offset += seq7.length + seq7.lengthOfLength + 1;
}
}
let offset1 = 0;
offset1 += inner1[1] + 2;
const slice1 = inner1.slice(offset1 + 2); // extnValue OCTET STRING (24 byte) 30168014FC46D101435FBB7BA63D3068AE11BAE0BC6DC9D3
offset1 = 0;
const seq7 = readASN1Length(slice1, offset1); // SEQUENCE (1 elem)
if (seq7 === undefined) {
return undefined;
}
offset1 += seq7.lengthOfLength + 2;
const inner2 = slice1.slice(offset1 + 1, offset1 + 1 + seq7.length + 2); // [0] (20 byte) FC46D101435FBB7BA63D3068AE11BAE0BC6DC9D3
return inner2.toString('hex');
function walkExtensions(inner) {
const seq7 = readASN1Length(inner, 1);
const inner1 = certBuffer.slice(offset + 1, offset + 1 + seq7.length + 2);
const v = bytesToOID(inner1);
return v === "2.5.29.35";
}
}
export function readASN1Length(buffer, offset) {
if (offset >= buffer.length) {
return undefined;
}
const lengthByte = buffer[offset];
if (lengthByte < 0x80) {
return { length: lengthByte, lengthOfLength: 1 };
}
const lengthOfLength = lengthByte & 0x7F;
if (lengthOfLength === 0) {
return undefined;
}
if (offset + lengthOfLength >= buffer.length) {
return undefined;
}
let length = 0;
for (let i = 1; i <= lengthOfLength; i++) {
length = (length << 8) | buffer[offset + i];
}
return { length: length, lengthOfLength: lengthOfLength };
}
export function pemToBuffer(pemCertificate) { // turn cert into buffer
const base64Cert = pemCertificate
.replace(/-----BEGIN CERTIFICATE-----/g, '')
.replace(/-----END CERTIFICATE-----/g, '')
.replace(/\s+/g, '');
return Buffer.from(base64Cert, 'base64');
}