Let's Encrypt Certificate is reported EXPIRED on Android 10 Private DNS Feature

Help
1

==== Early Notice ====

All domains contained are having Client Request Logging enabled, if you do NOT want to share your public IP, please do not try to visit the domain/link.

==== End of Early Notice ====
Hi, I didn't notice that the Let's Encrypt DST Root CA X3 will expire at Sep 30 14:01:15 2021 GMT, which is exactly last night (Indonesian Time).

I run a custom poisoned DNS server so that I can poison my devices to avoid getting spammed with a inappropriate advertisement in my phone home launcher and notification bar.

Anything has been perfectly running fine over like 2-3 months, however last night without any changes done to my server, my phone refuses to connect into my DNS without a specific reason or warning.

For now, I temporarily disabled my phone Private DNS ( requires TLS ), and run the normal basic un-encrypted TCP connection so that I'm able to atleast have internet to check my server control panel.

To my surprise, my TirtaDNS panel has been reporting that a client (in this case my phone) are throwing a lot of ERR_SSL_SSLV3_ALERT_CERTIFICATE_EXPIRED, around 5.2k of such expired failures in the past night.

After reading a bit on ServerFault, it looks like Let's Encrypt root recently expired and most client should change into the new one, as my laptop and other of my devices did nicely.

When using browser on the phone with a bypass DNS options enabled, I can still access my site which means that my browser does already have the new Let's Encrypt new root certificate, however.... it looks like the Android Private DNS
feature seems to be not able to automatically switch.

At first, I thought it's my certificate that's expiring, but I had a automatic renewal configured already to run certbot, well but I decided to renew a new certificate anyway as I was thinking it might solve the issue as the error reported by the phone are ALERT_CERTIFICATE_EXPIRED, turns out it didn't.

So my question is, is there any version of the new ISRG Root X1 and the R3 certificate which we can install on the android trusted certificate list manually ?

I also just recently just enabled Nginx exposing on my panel, so that we can now try to access https://tirtadns.tirtagt.xyz/ and it'll server the same certificate that the TirtaDNS uses.

I also try to let openssl connect and test the configuration by using :
openssl s_client -connect tirtadns.tirtagt.xyz:853 -servername tirtadns.tirtagt.xyz

  • Note that DNS over TLS port is 853

and here's part of the output :

Certificate chain
 0 s:CN = tirtadns.tirtagt.xyz
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   i:O = Digital Signature Trust Co., CN = DST Root CA X3

subject=CN = tirtadns.tirtagt.xyz

issuer=C = US, O = Let's Encrypt, CN = R3

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 4583 bytes and written 392 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

which I believe means that the TirtaDNS is already serving the correct ISRG Root X1 and R3, but it's just my phone that didn't trust it.

My domain is: tirtadns.tirtagt.xyz
My phone is : Infinix Zero 8
Phone Version : Android 10 Build X687-H851D-Q-OP-210705v513

I ran this command: No Command

It produced this output: No command

My web server is (include version): Custom + Nginx v1.18.0

The operating system my web server runs on is (include version): Ubuntu 20.04

My hosting provider, if applicable, is: OVH VPS

I can login to a root shell on my machine (yes or no, or I don't know): Yes, Full Root

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): Yes, but actually no, I used my own customized panel

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 0.40.0

2

Hi there,
Your server is correctly serving Let's Encrypt's recommended certificate chain, which looks like this:

tirtadns.tirtagt.xyz < R3 < ISRG Root X1 < DST Root CA X3

It's that last certificate (DST Root) that has recently expired, however as most modern devices also trust the ISRG Root, serving a certificate chain with an expired certificate after a trusted one should not be an issue.

From what you say, it looks like your phone however may have an issue when this chain is used for DNS over TLS, but not general HTTPS web browsing.

If you only access this server from relatively-modern devices, you can use Let's Encrypt's alternate chain which doesn't include the expired DST Root.

To do this, you will need to upgrade certbot to at least version 1.12.0, and then add the command line option --preferred-chain "ISRG Root X1" when requesting a certificate.

Let me know how it goes :slight_smile:

6 Likes
3

Thank you for the response.

I'll quickly upgrade certbot and use the --preferred-chain "ISRG Root X1" switch when requesting the cert from certbot, and report back here.

1 Like
4

@Tugzrida Thank you very much introducing the --preferred-chain "ISRG Root X1".

Now my phone accepts the certificate, and is connected to my tirtadns.tirtagt.xyz.
I tested it on my other devices, they took a little time to revalidate, however it's all now connected correctly with my TirtaDNS !

That parameter seems to resolve my issue, thank you very much once again @Tugzrida :grinning_face_with_smiling_eyes: :+1:

2 Likes
5

Wonderful! I'm glad I could help :grinning_face_with_smiling_eyes:

2 Likes
6

I was thinking that I need to turn off the TirtaDNS on my phone, and get spammed by an inappropriate advertisement every minutes !

Turns out, I can enjoy my TirtaDNS again :slight_smile:
Cheers !

2 Likes
7

Any idea how to deal with this within plesk? A lot of APIs and Postman don't works with the cert in the chain.....

8

Do you have access to the server SSH ? I think the plesk control panel didn't offer a way to include the parameter.

9

Sure, I have full root access

10

That's great, I never used plesk though so I'm not quite sure about how to do that on plesk

However, basically what I did is that :

  • I deleted the current certificate using certbot delete
  • I then regenerate the new certificate using certbot certonly --manual `--preferred-chain "ISRG Root X1"
  • I configure my TirtaDNS and Nginx server to use the new certificate instead.

and after that, it just works !

1 Like
11

If you don't want to re-issue a new cert until it expires, you can also re-use the current cert simply by removing the cross-signed "ISRG Root X1" certificate from the full chain served by your server.

Just update your ACME client config to request the non-default chain on next invocation.

This issue affects Private DNS client on both Android 10 and Android 11.

I've raised this issue on day 1 in Google IssueTracker here:
https://issuetracker.google.com/issues/201661947

2 Likes
12

I was also having issues with my android device rejecting the certificate presented to the "private dns" feature in android.

I am using a oneplus 9 pro, with android 11.
EDIT: It was also an issue on a samsung phone and a huawei phone running android 10

It took me a LONG time to find this post, so i will add some words to help someone else searching for it

lets encrypt android private dns over tls cannot connect expired root ca

1 Like
13

Yep, I hope more and more people seen this thread, we got the solution here !

14

Hello. I am having the same issue on Android 10 devices connecting to my API. I am using PLESK but on a Windows server. How do I fix this issue? I have deleted the existing certificate and issued another Lets Encrypt cert but I still get the cert expired on 10/18. The cert is valid until Jan 2022.

15

Are your new certificate are on the main chain or the "ISRG Root X1" ?

Had you tried the above suggestion yet ? I believe Plesk doesn't allow you to switch the certificate chain though.

Pada tanggal 19 Okt 2021 11.59 PM, Roger via Let's Encrypt Community Support letsencrypt@discoursemail.com menulis:

16

I would not know where to look to determine if it is on the main chain. I have search around in can find no ISRG Root X1 option.

17

You'll need to have a direct access to the server via SSH or other similar way for manual management, most control panel won't have the option to switch the chain.

As I did above, I generated the certificate manually while passing some argument into the certbot script.

18

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.