Feedback on raising Certificates per Registered Domain to enable peer-to-peer networking

Issuance Policy
1

I work on the libp2p project (libp2p.io) which is an open source networking stack meant to make it easier for developers to build networked applications, and particular ones that operate in a peer-to-peer or distributed manner. Some of the most well known users of libp2p include: IPFS (https://ipfs.tech/), Ethereum (https://ethereum.org/) and Filecoin (https://filecoin.io/) among many others both big and small.

My team (Interplanetary Shipyard) would like to launch a service that allows users to register for DNS names like <peer identifier>.libp2p.direct and enable them to get wildcard certificates from Let’s Encrypt for *.<peer identifier>.libp2p.direct via DNS01 ACME challenges. This would enable users to generate addresses like 1-2-3-4.<peer id>.libp2p.direct, which would resolve via DNS to 1.2.3.4 , and have those addresses usable in web browsers and other user agents that require CA signed certificates.

There are hundreds of thousands of libp2p nodes in the wild and while many wouldn’t need a certificate (e.g. they don’t have public IP addresses anyway and would have to deal with DNS rebinding protections) a whole lot would. IPFS’ Amino DHT alone has tens of thousands of publicly reachable server nodes Amino DHT | ProbeLab.

The Ask

Having read through the documentation it seems like we would qualify for increased rate-limits for the libp2p.direct domain name (i.e. the Certificates per Registered Domain limit). At the moment raising the limit to 5k-10k certificates per week seems sufficient. However, this may need to increase to >10k over time. However, I wanted to engage with the community here to see if there are any issues for us to be concerned about before applying for the raised limit.

More information about the problem we’re trying to solve and why having raised limits for certificates seems like the best way forward as well as an analysis of prior art, risks and explored alternatives is below:

Problem statement

libp2p (libp2p.io) is an open source networking stack meant to make it easier for developers to build networked applications, and particular ones that operate in a peer-to-peer or distributed manner. Some of the most well known users of libp2p include: IPFS (https://ipfs.tech/), Ethereum (https://ethereum.org/) and Filecoin (https://filecoin.io/) among many others both big and small.

Since the Web platform is a major way in which developers deliver applications to their users it’s important for the libp2p project to be able to make applications usable in web browsers like Chromium-variants, Firefox, and Safari. Given that browsers now lean heavily on Secure Contexts (MDN, W3C) it’s important that any communications that a web browser needs to make occur within a secure context. At the moment the ways to do this are: Have a CA signed certificates, use WebRTC with pre-determined certifcate hashes, use WebTransport with pre-determined certificates hashes.

Unfortunately, the most widely used browsers have missing features or bugs that using make WebRTC and WebTransport painful or unusable (some examples include ‣ and ‣, but there are more). While we’ve done work with groups like Igalia to work with browser vendors to remediate issues it appears to be a long road ahead until they’re reliably usable by developers.

As we continue working with browser vendors to improve the state of usable peer-to-peer communications in browsers having the ability for libp2p developers to leverage more hardened protocol implementations like HTTPS and Secure WebSockets would make their experience much easier.

While Let’s Encrypt currently makes it easy for anyone to obtain a certificate given a registered domain many users don’t have registered domains and don’t want to go through the process of obtaining one just to start using their application.

Goals

It should be possible for all common browser nodes to be able to communicate with non-browser nodes that are reachable with public IP addresses without users being required to buy or setup a domain name.

Non-goals

Ability for browser nodes to be able to connect to nodes with non-public IP addresses (e.g. behind NATs). This is out of scope due to the amount of effort required to deal with browser vendors to make WebRTC better and/or enable other protocols (e.g. WebTransport) to do holepunching. For the time be

Proposed solution

  1. Shipyard owns the domain name libp2p.direct
  2. Users’ applications will automatically be able to set their DNS01 ACME challenges for .libp2p.direct with Shipyard (note: a libp2p peerID is essentially a hash of a public key and so control can be verified statelessly)
  3. Users will be able to automatically get certificates for their domain via DNS01 ACME challenges to Let’s Encrypt for *.<peerID>.libp2p.direct
  4. This requires Let’s Encrypt to be ok with the number of certificates we will be registering. Given that this mimics the arrangement they have with Plex for plex.direct this is hopefully not a problem.
  5. Shipyard will have a DNS resolver will resolve all IPs of the form 1-2-3-4.<peerID>.libp2p.direct to an A record for 1.2.3.4 and xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxx.<peerID>.libp2p.direct to a AAAA record for the corresponding ipv6 address
  6. Applications will be able to disseminate domain names that will be usable in browsers with protocols like HTTPS and Secure WebSockets without the user needing to set anything up

Risks

  • In setting up a service for domain registration and setting the DNS01 ACME challenges there will be a need to handle takedown notices for abusers of a given domain name
    • Shipyard already has experience with this kind of work as a result of running the ipfs.io public gateway
  • The volume of peers wanting certificates grows high enough to cause problems for Shipyard’s service or for Let’s Encrypt
    • Shipyard: Given that most of the server-side work is stateless this shouldn’t be an issue for quite a while and if it is then Shipyard will need to setup some sort of rate-limiting / spam prevention
    • Let’s Encrypt: Given that Plex is already handling >4.5M certs of a similar nature for Plex it may not be an issue for a while. However, if it were Let’s Encrypt could ask Shipyard to institute rate-limits on setting DNS01 ACME challenges for new certificate issuances which would alleviate the pressure.

Prior Art

  • Plex: They are helping users do DNS01 with Let’s Encrypt to get users certificates for *.<identifier>.plex.direct and sets up DNS to return an A record for 1-2-3-4.<identifier>.plex.direct to 1.2.3.4. To handle this they have increased limits with Let’s Encrypt
  • pubtls.org: Was a project to give people <hash-of-publickey>.pubtls.org domains and let them set DNS records for themselves.
  • Tailscale
    • They are helping their users get custom subdomains and Let’s Encrypt certificates for them. To handle this they have increased limits with Let’s Encrypt

Alternative solutions

  • Work with browser vendors to make it possible to use self-signed certificates when the certificate is known ahead of time → In progress but quite slow
  • Propose a CA like Let’s Encrypt that they offer a generic service that could likely handle what libp2p and Plex both need (e.g. LE runs a domain name like letsencrypt.direct and give users certificates for *.<hash-of-public-key>.letsencrypt.direct when they prove they have the corresponding private) → Would be convenient if they are interested, but want the increased burden on Let’s Encrypt to be as low as possible.
    • If there’s interest we’d certainly be interested to use it or to have our open service be a prototype for what LE might want to run in the future
  • Use something like 1-2-3-4-<peerID>.libp2p.direct rather than 1-2-3-4.<peerID>.libp2p.direct → this means more certificates have to be issued since a given peer may have multiple IP addresses
  • Use something like a certificate hash instead of a peerID (e.g. to not encourage long-lived public-key based addresses that are painful to rotate) → potentially doable, although it seems would require more state on behalf of Shipyard’s side

Implementation and rollout plan

  • If handling of the large number of certificates is approved by Let’s Encrypt, Shipyard would setup the DNS resolver and prototype the challenge registration protocol in go-libp2p (GitHub - libp2p/go-libp2p: libp2p implementation in Go) before specifying it more publicly and rolling it out
  • It would likely start as experimental in go-libp2p for Secure WebSockets for more widespread use and testing and then a similar capability to support HTTPS would be added. It would likely roll out within kubo (GitHub - ipfs/kubo: An IPFS implementation in Go) and the IPFS ecosystem before being more widely adopted and deployed
  • Get libp2p.direct onto the Public Suffix List

Success Criteria

  • A large enough percentage of non-NAT’d IPFS nodes are directly reachable from common web browsers such that they realistically can directly retrieve data from hosting nodes without needing a fallback/proxy.
  • Adoption across other libp2p-based projects
  • Sufficient adoption that will bolster our case to browser vendors that there should be native support for connecting to user applications even when the users haven’t registered for a domain name (e.g. via some self-certifying address).

FAQ

  • What would a certificate like *.<peerID>.libp2p.direct mean for a consuming party?
    • It would be asserting that the controller of a given peer identity (e.g. private key) is also the controller of the given subdomain
    • For something like https://1-2-3-4.<peerID>.libp2p.direct which has DNS returning an A record for 1.2.3.4 a consumer trusting the CA + domain owner of libp2p.direct can be sure that the address is either controlled by the owner of the corresponding private key or will result in a certificate error.
3 Likes
2

Maybe add libp2p.direct into https://publicsuffix.org/ ?

6 Likes
3

The public suffix list would be required for security reasons anyway. Please note that "rate limits for Let's Encrypt" is NOT a valid reason to include a domain to the PSL. But security reasons are. And as a side effect, the rate limits are affected too.

7 Likes
4

I suspect you're responding to @orangepizza's suggestion, but as mentioned it seems better to wait for requesting PSL inclusion until after we've received a rate-limit increase from Let's Encrypt, so it's clear that inclusion is for security purposes.

5

The PSL inclusion process has no insight into whether or not you have LE rate limit overrides.

Also, if you get LE "Certificates Per Registered Domain" rate limit overrides for libp2p.direct, and then get libp2p.direct onto the PSL, those overrides will immediately cease to be effective (because it will no longer count as a Registered Domain), so getting the override will have largely been a waste of your time and ours.

As Osiris says, I suggest you get this domain added to the PSL as quickly as possible, because you need to have cookie and cross-origin domain separation between the people you are allowing to register subdomains.

11 Likes
6

so getting the override will have largely been a waste of your time and ours.

Thanks for the advice :pray:. I was mostly following what Tailscale did (PSL PR Add ts.net and beta.tailscale.net domains by danderson · Pull Request #1453 · publicsuffix/list · GitHub), but of course we have no interest in wasting your time or ours.

We’ve put up a PR to the PSL at add libp2p.direct by aschmahmann · Pull Request #2084 · publicsuffix/list · GitHub. While we are waiting for that PR to get merged are there any other concerns or asks for us to take care of?

7 Likes
7

Thanks @aarongable for the advice on getting added to the PSL, our PR is now merged and we should be all squared away there. Is there anything else you need on our end or should we be all set?

Aside from the email we have included in the PSL is there anything you need from us in the event that you want to reach out due to something like a spike in certificate issuances causing a problem? Given that we're utilizing Let's Encrypt's public infrastructure I want to make sure we're being respectful consumers of that resource.

Thanks again :pray:

7 Likes
8

Congrats on getting your PSL PR merged!

It will take a little bit of time for that to flow downstream to us (it has to pass through one intermediate repository first, and then we have to update our copy of the PSL, and then we have to deploy that updated code) but hopefully it won't take too long.

I do have a few small suggestions that will help things go smoothly:

  • Ensure that the ACME client you ship attempts to renew at random or jittered times (e.g. not at midnight every night) to avoid contributing to periodic load spikes
  • Ensure that the ACME client you ship supports ARI, so that it can handle automatic replacement without your users needing to intervene in case there is a mass revocation event
  • Ensure that the ACME client you ship can auto-update without your users' intervention, so that future improvements (like support for short-lived certs) can be delivered to your users directly

And finally, remember that we're a public-benefit non-profit, so you can always donate to support the work we do and help keep us runnning: Donate - Let's Encrypt

11 Likes
9

@aschmahmann You can follow the above steps by checking, in sequence:

Once Boulder runs a commit that includes the PSL update, you're good to go.

6 Likes
10

@aarongable the code landed in publicsuffix-go (Updated list from source (#1027) · weppos/publicsuffix-go@a8ed110 · GitHub).

Any chance the dependency could be updated in boulder? Happy to submit the PR if it'll make things easier.

Edit: PR is at Update public suffix list by aschmahmann · Pull Request #7672 · letsencrypt/boulder · GitHub

3 Likes
11

Thanks for creating the PR; it's been merged and should be deployed Thursday of this week.

5 Likes
12

Thanks you all for the help :pray:, things look good. Looking forward to posting back once things are up and running for any interested parties to try out.

4 Likes
14

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.