Feedback needed for our new account "pausing" feature and self-service "unpause" portal

Translations is on our roadmap but the initial version will not be.

7 Likes

Uhm, do we anticipate a need to test for this while providing support?

4 Likes

I still don't see the security implications. Worst thing that could happen is that someone could try to get a certificate again, i.e. go back to the status quo. So even if the URL got out, no real harm could be done, and if the problem isn't solved, the identifier will be paused again shortly after.

Surely someone could also try to automate the process, but to what purpose? It still won't get them a valid certificate, so why would someone automate the unpausing instead of fixing the problem?

1 Like

Pure malevolence?

:smiling_imp:

(I see this type of move as akin to using --force-renewal in certbot)

:musical_score: If at first you don't succeed, try the same thing again and again and again... (ad nauseum)

or

I've seen crazier things.

(Sometimes I feel like Docker containers were created for trial-and-error insanity.)

4 Likes

As I understand it, the point is to save Boulder the trouble of going through the whole validation process just to say "no". In order for this to make any sense, the "pause" checks and management have to be much lighter on resources than the actual process. This is why I even suggested only changing the URLs after they have been used. Otherwise you will have to check for expiration, generate new ones, update the database, etc. If it's to complicated, you might as well just stick to the status quo.

I don't buy it. Who is smart enough to code a bot if he's not even able to delete an outdated renewal configuration file? Might as well just launch a DDOS attack. At least there's scripts for that out there. :wink:

Again, the system resources you invest in checking for all the possible ways in which someone could try to circumvent the process (without benefit), bot detection, captchas, etc., could turn out more than just doing a few more real domain validations. If those became a problem, there must be an incredible amout of them, because the workload of validating a single domain will be negligible.

1 Like

It's not about intelligence. It's about motivation. :wink:

The effort to which some people will go to do the wrong thing can be truly astounding.

6 Likes

Here is my one cent. Just pause the account-identifier key with the appropriate error message including a pointer to this forum. What is the quantity of the un-pause requests you are expecting? If not much, you can un-pause them manually. Delay the design, decision and implementation of the self-service un-pause until you see that the quantity of unpausing requests on this forum gets overwhelmed.

EDIT: start pausing the account-identifier keys that are failing since many-many years. Wait a bit for feedback, and gradually reduce the expected failure time period until it fits your needs.

3 Likes

If we paused the set of account+identifier pairs that we currently intend to manually pause in the first wave, and only 0.1% of those requested an unpause, we'd still be dealing with thousands of manual unpause requests.

Manual never works at our scale.

The nice thing about our system is that the URLs are never stored in our database. Each URL contains a signed JWT, so the unpausijg system can tell that it's a genuine unpause request without having to consult the database at all. And yes, rejecting a new-order request due to some identifiers being paused is a couple orders of magnitude cheaper than letting it through and then attempting but failing validation.

10 Likes

I think you are overestimating the unpause requests. I guess it must be at least two order less. But being on the safe side, just pause 100 times less in the first wave, and wait a week for feedback.

EDIT: start with something small. Be ready to rollback.
You also need feedback from the user:
What is up now? Were you satisfied with the same regular error messages and no certificate in the past couple of years, and suddenly a different error message wakes you up?

3 Likes

Wandering off topic but one day I'd love to be able to enter my email into a letsencrypt.org page and be sent a magic sign in link (as per the jwt) so I can see stuff associated with that (email) account:

  • list of active associated public account keys (for that email address) and when last used
  • successful and failing cert orders (including the zombies for pause/unpause)
  • current (approx) rate limit levels for each account associated with that email

As much as a diagnostic tools as anything else, and not massively powerful (e.g. cannot revoke etc). I believe most of that data would already be present but not surfaced to the user and it would likely only appear to a small subset of users, but currently it's easy to use satya@microsoft.com as your email address on an LE account and if I was him I'd like to know (for instance)! [or I might want to proactively pause certain identifiers associated with my account]

The reason I'm suggesting such a thing is so as to have half an eye on a future user portal as features like this get built.

8 Likes

Anyone can use any email address when they register an account, so I'm not sure how meaningful it would be.

This is probably more EAB territory, which would add another layer of complexity (not to mention how to handle all of the existing accounts). EAB would give you a centralized external account login to see whatever information associated with you.

2 Likes

Yeah I guess my point was that anyone can use any email address, but the email address owner could have some control over that, or at least see what's associated with their address.

4 Likes

My mistake, I took some of these screenshots at different times during the development. Both the log line and form itself will contain the same set of paused names. For instance, if the order contains 10 names and 4 of them are paused, the log line and the unpause form will contain the same 4 names.

Small correction: The first fifteen paused identifiers present in the NewOrder request that returned the log line. For instance, if the order contains 20 names and 19 of them are paused, the log line and the unpause form will contain the first 15 paused names returned by the database.

I hope this clarifies the implementation and demonstration.

10 Likes

This is why I suggest a simpler version that only offers "unpause" functionality. That would allow avoiding a JWT token, and could just be handled through a random b64 string (8-12 chars long) used as an account identifier.

In my experience with commercial websites and open source projects, all end user demographics have (endless) problems with copy/paste of long multi-line URLs.

6 Likes

I know this is probably off-the-wall, but what about just presenting a static link to a simple webpage with a short form where one enters the domain names to be unpaused, one per line? A captcha could eventually be put on that page if there is too much automated interaction.

The idea here is that there's no need to personalize the functionality. This would allow for a simple, efficient list of paused identifiers to be utilized on the backend.

Edit: This doesn't track well with domain ownership changes.

3 Likes

I think they want to be able to pause name per account, in that if a name is working on one account but not on another they only want to impact that broken account. And in the case of multiple broken clients for the same name, only unpause one broken client at a time.

6 Likes

Suppose it does depend upon how you want to slice-and-dice it. Pausing per account does solve the domain-ownership-change problem.

3 Likes

The JWT allows us some niceties though. We configure both RFC 7519 registered claims and custom claims in the WFE2, sign them, transmit, and finally validate those claims in the SFE which gives us a level of confidence that the JWT wasn't tampered in-flight. We're using go-jose's JWT implementation which also enforces expiration based on a claim field so that old URLs become irredeemable.

Example of those claims can be found here.

W.r.t the long URLs in this the year of the Linux desktop 2024, most people should have a mouse and be able to right click copy/paste?

5 Likes

The set of "system administrators" which leave a broken system running for years, and then suddenly want to try to get it working again, may have a significant overlap with the set of people still muddling around trying to figure out how to use the basics of their terminal emulator software and Linux. There's more than one post on this forum of someone confused when they're running certbot in manual mode, are trying to copy the token so they hit Ctrl+C, and then have no idea what to do when the program quits on them instead of copying the token.

I personally think that the longer URL may probably be worth the tradeoff, but it certainly is a tradeoff and is going to make things harder on some people.

8 Likes

This version -- where a short random string somehow carries the necessary information, rather than encoding it directly in the URL -- requires using a database to map that short random string to the relevant information. We could go that way in the future, but we don't believe the extra implementation and deployment complexity is worth it to give people an easier-to-copy URL. Obviously we can revisit that idea in the future, but this stateless system carries a lot of value for us right now, so we plan to keep the JWK essentially as-is.

8 Likes