I was routinely checking CT logs for a couple of my domains where I stumbled accross multiple issuances which didn’t seem right. Every single domain had two certificates issued on the same day.
Further investigation revealed that both certificates share the same serial number and both are practically the same certificate. I noticed the only difference was one being a precertificate and the other one a leaf certificate.
What is the difference between the two?
Are precertificates required to be published to CT logs?
Does this only happen with crt.sh? Is it a problem with them?
These SCTs as they are abbreviated, can be included in the TLS handshake, in a stapled OCSP response or, as Let’s Encrypt uses them, into the final certificate itself.
For the SCT to be included in the final certificate, the CA (in casu Let’s Encrypt), has to send a “pre-certificate” to the log first. Then, when it receives the SCT, it can include this SCT in the final certificate. But it has to send the final certificate to the log too. So you’ll see it twice.
Your first and third question were already answered, so I'll take your second one:
For you, yes they are required. There's no way for you to get a certificate from Let's Encrypt (and most other certificate authorities) without them.
For Let's Encrypt, no, they don't necessarily have to submit them. But if they didn't, everyone would be forced to set up OCSP stapling or compile and install an extra extension into their Apache and nginx servers to comply with Google Chrome's new Certificate Transparency requirements, or their sites would not work in recent versions of Google Chrome.
That's a lot of work, which is why most CAs, including Let's Encrypt, have gone the precertificate/SCT route, so end users don't have to do anything they didn't do before to comply with these new requirements.
Also have a look at censys.io as that is the one I ended up standardizing my searches on. It has a good API and I believe they don't include records with pre-certificates
Alternatively, (root as well as intermediate) certificate authorities
may submit a certificate to logs prior to issuance. To do so, the CA
submits a Precertificate that the log can use to create an entry that
will be valid against the issued certificate. The Precertificate is
constructed from the certificate to be issued by adding a special
critical poison extension (OID 1.3.6.1.4.1.11129.2.4.3, whose
extnValue OCTET STRING contains ASN.1 NULL data (0x05 0x00)) to the
end-entity TBSCertificate (this extension is to ensure that the
Precertificate cannot be validated by a standard X.509v3 client)