Trying to Understand the Google Transparency Report

Help
1

My domain is: igw.news
There are 9 others with certificates on the same server. IP: 5.101.140.50

I ran this command:
https://transparencyreport.google.com/https/certificates?cert_search_auth=&cert_search_cert=&cert_search=include_expired:true;include_subdomains:false;domain:igw.news&lu=cert_search

It produced this output:

GoogleTransReport
GoogleTransReport1331×767 22.9 KB

My web server is (include version):
Server version: Apache/2.4.41 (Ubuntu)
Server built: 2021-10-14T16:24:43

The operating system my web server runs on is (include version):
Ubuntu 20.04.3 LTS (GNU/Linux 5.4.0-90-generic x86_64)

My hosting provider, if applicable, is:
Dedicated Server on UKServers

I can login to a root shell on my machine (yes or no, or I don't know):
Yes, through SSH

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
No.

The version of my client is (e.g. output of c or certbot-auto --version if you're using Certbot):
certbot 1.21.0

So the Google Search certificates by hostname on igw.news shows 4 rows of infomation.
2 are dated 15th Nov. I think I created them with the certbot.

2 are dated 12th Nov. and these are for igw.news and *.igw.news
I am not sure if I did these - maybe I did.

I think I only have one certificate for igw.news on my server which
covers igw.news and www.igw.news

However it seems that there is a certificate for ALL subdomains ?
Is it valid ?
Does this mean that if I create `health.igw.news then I have a certificate that
covers this as well?

So, the First row detail gives this:

Subject  CN=igw.news
Serial Number 3:54:31:85:8E:C3:04:92:A1:4A:A9:3F:CE:81:00:D6:E5:12
Issuer  C=US, O=Let's Encrypt, CN=R3
Validity 15 Nov 2021 — 13 Feb 2022
DNS Names igw.news, www.igw.news

The Second row detail gives this:

Subject  CN=*.igw.news
Serial Number 4:20:6E:E9:0D:4E:48:6D:E3:77:3F:95:66:02:90:7C:BF:5E
Issuer  C=US, O=Let's Encrypt, CN=R3
Validity 12 Nov 2021 — 10 Feb 2022
DNS Names *.igw.news,  igw.news

The Third row detail gives this:

( seems to be a copy of the Second row info ... same Serial number )

Subject  CN=*.igw.news
Serial Number 4:20:6E:E9:0D:4E:48:6D:E3:77:3F:95:66:02:90:7C:BF:5E
Issuer  C=US, O=Let's Encrypt, CN=R3
Validity 12 Nov 2021 — 10 Feb 2022
DNS Names *.igw.news,  igw.news

The Fourth row detail gives this:
( seems to be a copy of the First row info ... same Serial number )

Subject  CN=igw.news
Serial Number 3:54:31:85:8E:C3:04:92:A1:4A:A9:3F:CE:81:00:D6:E5:12
Issuer  C=US, O=Let's Encrypt, CN=R3
Validity 15 Nov 2021 — 13 Feb 2022
DNS Names igw.news, www.igw.news

So, I seem to have TWO certificates with different serial numbers?

In my /etc/letsencrypt/live/ directory, I only have one directory for igw.news
and of course it only contain one private key.

So I am not sure about this at all.

Any help would be very much appreciated.

2 Likes
2

The duplicate certificates you've found are called precertificates.

You can also search CT Logs on crt.sh, which gives you nice detailed reports for every certificate.

See for example these links:

https://crt.sh/?q=igw.news

https://crt.sh/?id=5612251133

https://crt.sh/?id=5612243334

As you can see on crt.sh, the certificate with Serial Number 03:54:31:85:8e:c3:04:92:a1:4a:a9:3f:ce:81:00:d6:e5:12 is listed as a "precertificate". Same for one of the other two.

So, what are those?

Let's Encrypt is - like other Certificate Authorities (CAs) - required to log all of their issued (leaf) certificates into a Certificate Transparency (CT) Log. Those store the certificates and they are the (primary) data source used by the Google Report and crt.sh.

When browsers like Google Chrome connect to your website, they want a proof that your certificate has been dutifully logged into a CT log. To have this proof readily available, we use a technology called Signed Certificate Timestamp (SCT). Those are send together with the certificate, so that Chrome has immediate proof that the cert has been logged. SCTs can either be send via standalone TLS extensions, or are directly embedded into the certificate. Let's Encrypt does the latter, because that is much easier for subscribers - they don't have to worry about CT at all, because everything is already in the certificate.

However, in order to aquire such a SCT, you need to submit the certificate to the log first. However, if you want to have your SCT in the certificate, that's a chicken or the egg problem: You need the certificate before you get the certificate.

The solution for this is the precertificate. When Let's Encrypt issues a certificate, it first issues a "precert" to the log and then receives the SCTs back. Let's Encrypt then creates the final certificate - including the SCT - and sends that back to you.

There's no requirement to additionally log the final certificate, but its usually logged anyway. Hence you see all your certificates twice.

(And yes, apart from the precerts there are two certificates, one including a wildcard)

5 Likes
3

Personally, I find crt.sh more clear: crt.sh | igw.news

It shows 2 certificates, the same as Google, but Google showed the two certificates as 4. Perhaps it also shows pre-certificates? I dunno, I always use the "deduplicate" option on crt.sh to only show one of the pre- cq. real certificates.

Correct, a wildcard certificate with the * as leftmost DNS label is valid for all single label subdomains. E.g., the certificate for *.example.com is valid for bar.example.com but not for foo.bar.example.com.

However, your site currently uses the non-wildcard certificate. And you can't use a certificate from a certificate log, as you also require the corresponding private key. If you don't have the private key, the certificate itself is useless. You can check which certificates are available in certbot by running the following command:

certbot certificates

Although the fact you only have one directory for igw.news in /etc/letsencrypt/live/ suggests you don't have the wildcard on your system.

6 Likes
4

@Dave.Sintra Hello again Dave :slight_smile:

You may find the wildcard cert in your /etc/letsencrypt/archive/igw.news folder. The cert "file" in the /live folder is a symlink to the most recent cert in /archive. Each time Certbot issues a new cert it adds it to /archive and then updates the symlink in /live.

It is best to have your Apache conf pointing to /live so it always has the most recent. In special cases and temporarily you could point Apache to something in /archive but in your case it would probably be best to just create a new wildcard if that is the direction you want to go.

5 Likes
5

Thanks SO much for your explanations.
Now I have a much better understanding.

I have run certbot certificates
and I get this output ...

Found the following certs:
  Certificate Name: bot10x.com
    Serial Number: 3ed0836288cd1a68d655bb2beb628e8b354
    Key Type: RSA
    Domains: bot10x.com
    Expiry Date: 2022-02-13 17:56:36+00:00 (VALID: 87 days)
    Certificate Path: /etc/letsencrypt/live/bot10x.com/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/bot10x.com/privkey.pem
  Certificate Name: chitchatmedia.net
    Serial Number: 3ab84253de5eaa3296903a22acc24b9b00f
    Key Type: RSA
    Domains: chitchatmedia.net
    Expiry Date: 2022-02-13 18:12:04+00:00 (VALID: 87 days)
    Certificate Path: /etc/letsencrypt/live/chitchatmedia.net/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/chitchatmedia.net/privkey.pem
  Certificate Name: cybxpert.com
    Serial Number: 4e0c55a03ea8bc7e592371bebe3670e2778
    Key Type: RSA
    Domains: cybxpert.com
    Expiry Date: 2022-02-13 17:58:53+00:00 (VALID: 87 days)
    Certificate Path: /etc/letsencrypt/live/cybxpert.com/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/cybxpert.com/privkey.pem
  Certificate Name: expressresponse.net
    Serial Number: 3534fe48d0716a2f299811f85ffd7411a50
    Key Type: RSA
    Domains: expressresponse.net
    Expiry Date: 2022-02-13 17:59:30+00:00 (VALID: 87 days)
    Certificate Path: /etc/letsencrypt/live/expressresponse.net/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/expressresponse.net/privkey.pem
  Certificate Name: gldn.page
    Serial Number: 4d7937c5b11872eef45b3e2665e2c701c54
    Key Type: RSA
    Domains: gldn.page www.gldn.page
    Expiry Date: 2022-02-13 07:26:06+00:00 (VALID: 87 days)
    Certificate Path: /etc/letsencrypt/live/gldn.page/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/gldn.page/privkey.pem
  Certificate Name: igw.news
    Serial Number: 35431858ec30492a14aa93fce8100d6e512
    Key Type: RSA
    Domains: igw.news www.igw.news
    Expiry Date: 2022-02-13 12:25:37+00:00 (VALID: 87 days)
    Certificate Path: /etc/letsencrypt/live/igw.news/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/igw.news/privkey.pem
  Certificate Name: pdg.reviews
    Serial Number: 33b6012b1986c22837f19fa1f9a52cb49f7
    Key Type: RSA
    Domains: pdg.reviews
    Expiry Date: 2022-02-13 18:00:09+00:00 (VALID: 87 days)
    Certificate Path: /etc/letsencrypt/live/pdg.reviews/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/pdg.reviews/privkey.pem
  Certificate Name: reviewed.page
    Serial Number: 363ce925f18a660c43691082b999018f1ac
    Key Type: RSA
    Domains: reviewed.page
    Expiry Date: 2022-02-13 17:55:44+00:00 (VALID: 87 days)
    Certificate Path: /etc/letsencrypt/live/reviewed.page/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/reviewed.page/privkey.pem
  Certificate Name: ukncsa.com
    Serial Number: 34784a8abdb52a94ae49bd4b92da5cb2a64
    Key Type: RSA
    Domains: ukncsa.com
    Expiry Date: 2022-02-13 18:12:42+00:00 (VALID: 87 days)
    Certificate Path: /etc/letsencrypt/live/ukncsa.com/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/ukncsa.com/privkey.pem
  Certificate Name: www.reviewed.page
    Serial Number: 3824bf32f419833336ccf1c35d8cdf1483a
    Key Type: RSA
    Domains: www.reviewed.page reviewed.page
    Expiry Date: 2022-02-13 17:52:08+00:00 (VALID: 87 days)
    Certificate Path: /etc/letsencrypt/live/www.reviewed.page/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/www.reviewed.page/privkey.pem
  Certificate Name: yel.page
    Serial Number: 4a4113e7954c3ebd81b9176d9f80381d58b
    Key Type: RSA
    Domains: yel.page
    Expiry Date: 2022-02-13 18:02:13+00:00 (VALID: 87 days)
    Certificate Path: /etc/letsencrypt/live/yel.page/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/yel.page/privkey.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
root@Ubuntu20:~#

I have summarised the domains covered

Domains: bot10x.com
Domains: chitchatmedia.net
Domains: cybxpert.com
Domains: expressresponse.net
Domains: gldn.page www.gldn.page
Domains: igw.news www.igw.news
Domains: pdg.reviews
Domains: reviewed.page 
Domains: ukncsa.com
Domains: www.reviewed.page reviewed.page 
Domains: yel.page

It seems that reviewed.page got done twice and I DO
have 2 certificates for this as there are 2 entries in the
directory /etc/letsencrypt/live/
www.reviewed.page
and reviewed.page
(and they have different privkey.prm files)

It also seems that www.bot10x.com
is NOT covered by the bot10x.com certificate as are several of the others.

What can I do about this ?

Would it not be best to have the certificates covering both the domain and the www.domain ?

How do I fix this?

Again, thanks for all your help.
Dave

PS, I checked out https://crt.sh/?q=igw.news

and noticed that although there are 4 lines on info, the Public Keys
are the same in each case - so only ONE certificate I guess.

1 Like
6

Yes, it would. To correct just issue a new cert with both names. What command did you use to create them originally? There are various ways - it would be easier to give example if you show what you've done.

4 Likes
7

Hi,

I used ...
certbot --apache

and then it gave me selections like this ...


Saving debug log to /var/log/letsencrypt/letsencrypt.log

Which names would you like to activate HTTPS for?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: bot10x.com
2: www.bot10x.com
3: chitchatmedia.net
4: www.chitchatmedia.net
5: cybxpert.com
6: www.cybxpert.com
7: expressresponse.net
8: www.expressresponse.net
9: gldn.page
10: www.gldn.page
11: igw.news
12: www.igw.news
13: pdg.reviews
14: www.pdg.reviews
15: reviewed.page
16: www.reviewed.page
17: smartbiz.pro
18: www.smartbiz.pro
19: ukncsa.com
20: www.ukncsa.com
21: yel.page
22: www.yel.page
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel):

I then selected the pair like 1,2 in order to have a certificate
for both the domain and www.domain.

If I do that again, will it overwrite the existing certificates ?

ALSO, how would I get a certificate that covers all subdomains with a wildcard
eg *.igw.news ?

Thanks.

2 Likes
8

Yes, select the pair and it will in practice overwrite the certs. It actually adds the new cert to the /archive folder and updates the symlink in the /live folder to point to that but the result to Apache is the same. The name in the /live and /archive folders relate to the first name you choose (more details here).

If you then have a "leftover" folder name and cert you won't need use sudo certbot delete --cert-name X naming the folder of the one no longer needed. Your Apache conf should be updated to use the combined cert before that.

Given your current list of names I do not see much value in a wildcard. Refer back to the post from osiris. Someone made a wildcard cert of *.igw.news on Oct12. Two things ... one is you or someone already knows how to do that. More importantly, that wildcard covers names like blog.igw.news, mail.igw.news, anyname.igw.news but does not cover any of your other domain names.

That said, you could have one cert with all of your names in the same cert - just select more than just two names when prompted. Adjust your Apache conf to refer to this new composite cert which will be in the folder named with the first cert in sequence (see link in this post first paragraph). Whether that is more convenient is up to you. I am just describing an option. If you do this you would delete all the cert folders you already made and just use this latest folder for each Apache VirtualHost certificate files.

I hope this is cogent :slight_smile:

PS: I was surprised to see you using the --apache installer. Your prior apache samples in the other post did not have any lines saying "managed by certbot" which is what usually happens when you do that.

4 Likes
9

You could also flex your certbot skills and use --cert-name to force a given name onto the new cert.
Like: --cert-name ALL-Sites [doesn't have to be in FQDN format]
As well as certonly if you really want to get under the hood and make the Apache SSL file changes manually. And if you are feeling extremely daring, you could strip the entire certbot process down to only "manual"ly defined steps and even use custom challenge file handling.

As for wildcard certificates, they require DNS-01 authentication.
Which requires making DNS entries in the corresponding DNS zone (can be delegated via CNAME).
And to automate that process, you should use a DNS Service Provider (DSP) that supports DNS zone updates via API. And use an ACME client that has a DNS plugin that supports that DSP.
[again, CNAMEs can be used to delegate records to another DNZ zone served by another DSP]

4 Likes
10

Thanks very much for all the advice.

Probably I did create the *.igw.news cert without realizing !!!

On one of my domains ... gld.page, I think I DO need a wild card certificate.

I want to install "wordpress multisite" on my gldn.pages domain. This enables multiple websites on a single WordPress installation using sub-domains.

It says "To use a subdomain configuration, you must have a wildcard entry in your DNS. To create a wildcard DNS record, enter * in the name field and enter your server IP address in the value field in your DNS manager" which is pretty straightforward.

I guess I can then issue a new certificate to cover all the sub-domains for *.gldn.page
I certainly would not want to issue a new cert every time I create a new subdomain blog, so a wild card in this case would make sense

I think I am quite happy to have all the domains on different certificates.

I will definitely be looking in to those other commands to clear up orphaned folders or certs.

Thanks again !

2 Likes
11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.