Let's make Let's Encrypt easy and simple

At least in the old days (which make me misty just thinking about them), there was a manual. Now people just refer to a non-existent set of docs by saying that for any platform, googling it brings up 10 how-tos. Which is not true, but certainly takes the phenomenon to the next level. In a generation, I wonder what the excuse will be.

1 Like

Sorry in advance for the huge post.

Don't apologise! It took me an embarrassing amount of reading before I felt confident I knew the difference between a protocol and a cipher, and how the key relates (or doesn't) to both. Throw in a bunch of acronyms and it's not surprising most people misunderstand. I still make mistakes that cause eyes to roll in the forums.

Yeah, there is a large amount of assumed knowledge on the part of most documentation. It's not all unreasonable - if you're capable of setting up a server and using the command line, the documentation is actually pretty good.

The problem arises when the majority of people aren't running their own server and they've never used a command line. Hosting services are common, and there are plenty of people using Windows as a server and have never had to use the command line. I'm not surprised there's currently no easy step by step guides that work for everybody since the possible combinations of situations is astronomical.

Of course, as the available clients mature they'll be able to fill in more and more blanks for the user, give better error/feedback messages, and be generally more robust. But even then, the user will have to know what clients are available for their systems, and not be required to clone git repositories and the like (or even know what git it).

I guess I'm saying that Let's Encrypt (the CA) is barely 6 months old. It only came out of closed beta testing at the beginning of the year, and the documentation has improved remarkably in that time. The new certbot website is excellent - you choose your OS and web server, and it provides pretty simple instructions (at least it does for FreeBSD!)

Of course, it provides very little help if something goes wrong, but as @pfg said earlier, the questions being asked in this forum are a very small percentage compared to the number of certificates successfully issued. As the clients mature, less and less will go wrong.

I also think we need to avoid the Nirvana Fallacy, and get rid of the idea we need 100% perfect guides for everybody. Cars are pretty simple to drive, but there are still people who struggle to get a driver licence!

Success will grow! The official client is still in beta, it can only get better. LE is already pretty successful and software and documentation is constantly being developed and improved upon. I haven't seen anything that makes me think there are problems that could overwhelm us. As long as fewer and fewer users are having issues (as a proportion of the overall userbase), we're heading in the right direction.

Probably because many of them aren't using linux and have never seen a command line. Most of the people I see asking questions in this forum are using hosting services and don't actually run their own servers. Those that do will often finish the thread with a "Fixed! I simply forgot to ..."

Me too :innocent: Don't let the naysayers discourage you! But at the same time, don't think the docs or the implementation is worse than it is! Pfg is right, very few people have trouble, and of those that do (e.g. in these forums), most ultimately get the results they're after.

Uh, I'm not why the answer is important. Pfg's point was that the issuance numbers aren't inflated by repeated attempts or renewals, 90% are people successfully obtaining the certs they're after the first time (which is the goal - don't get distracted!) Whether they use those certs for good or evil is irrelevant, just like nobody worries whether somebody is using their driver's licence to drive the getaway car in a robbery.

I'm not sure what the issue with that is - the majority of people using LE don't know what a CSR is, much less want to use their own. Those that what to generate their own and use it aren't going to have issues simply using a CSR flag with the client. This is a small fringe case that doesn't really need to be disproven. If you can generate a CSR with openssl, then you can use a CSR flag with certbot.

That's less and less of an issue over time. I've never used "git clone" in my life, yet I've just renewed my multi-SAN cert that I obtained months ago. Git is great if you're on an unsupported system, but LE clients are increasingly available as packaged apps. I'm on FreeBSD, and I can choose between the official client and a third party client directly from Ports/Pkg. Linux systems are the same, with multiple clients available through whatever package management systems they use.

Although you're right, that "git clone" line probably shouldn't be there, it's a hangover from the early days before certbot was easily available. "Git clone" will work on just about any *nix system, so nowadays it's basically a fallback position.

That's a bit disingenuous. Yes, for the majority of people they do actually work that easily. The many difficulties reported in the community forums are tiny in comparison to the millions of certificates successfully issued. That's why pfg quoted you the numbers with 90% being new domains - that's 90% of millions of successfully issued certificates, which is the goal.

Now of course, that number may have been higher if the client and docs were better, but that's a work in progress (on a project that's only been publicly available for 6 months!)

No, that plugin is part of certbot, you have nothing to obtain. At this point it's hard to tell if you're playing Devil's advocate or not!

You seem to have worked yourself into a mindset of this all being too hard. It's really not. The documentation is not as unclear as you're presenting and the clients (official and third party) work well. Much of what you're saying is very vague and non-specific. I know you're not a document writer but if there's a specific issue with the docs then let us know.

Shoen and other maintainers have been very receptive to people asking for clarification or amendments to the documentation. Documentation is improved when people ask for specific omissions to be filled in or for unclear writing to be clarified. You just have to ask when you find it.

Bugger! I'm really sorry to hear that.

I'm trying to respond to your posts respectfully and considerately because I can see you're not just complaining, you're actively trying to contribute. I really appreciate that. I hope all the negativity in this thread doesn't discourage you too much.

1 Like

DarkSteve, Thank you for your positive and supporting post. It has done much to make me feel more optimistic.

Perhaps I should clarify (again) that my main complaint is the poor quality of our Start Here page, as compared to that of many other exciting projects. I’ve given a list of these above, but please add Caddy to that list, as it uses LE as part of a webserver, and its documentation is so simple and clear that it inspires one to click “Download” and try it out. LE does not currently do so, in my opinion.

You are welcome to disagree with my opinion, and feel instead that our initial documentation is welcoming, accurate, and sets the stage for a satisfying new user experience. As I said, that opinion can be tested, using test subjects or automated surveys.

But I do want to respond (again) to your assumptions about our issuance statistics:

…the issuance numbers aren’t inflated by repeated attempts or renewals, 90% are people successfully obtaining the certs they’re after the first time (which is the goal - don’t get distracted!) Whether they use those certs for good or evil is irrelevant…

…the majority of people using LE don’t know what a CSR is, much less want to use their own. Those that what to generate their own and use it aren’t going to have issues simply using a CSR flag with the client. This is a small fringe case that doesn’t really need to be disproven.

I’m sure you know that spam and viruses are a major hazard for most users of the Internet, requiring specialized protective software that doesn’t work perfectly. But you don’t seem aware that people whose job is distributing viruses and spam use a large number of domain names. While some viruses are typically stored on an insecure WordPress page on a plumbing supply website, where they can be hidden easily from the owner of the site, others (especially when spam is involved) benefit from a dedicated website, parked under a very large number of domain names.

Spammers automatically allocate many IPs and domain names, sometimes supported by cooperative paid or bribed Registrars and other infrastructure in their countries.

If spammers were to feel that the future of the Web may involve the elimination of HTTP websites, then they may have a great interest in allocating a great number of HTTPS domain names now, in advance of the need. That would involve obtaining many certificates.

Spammers are very smart people (or their paid developers are very smart). They know that patterns of repeated attempts, renewals, etc., would reveal their presence in the LE client stream. They have long experience (in spamming) in how to avoid patterns (to foil SpamAssassin and security people alike).

This is the background information that I thought was obvious when I objected to making assumptions about hundreds of thousands of successful and apparently first-time certificate issuances. Unless a “honeypot” is set up to catch malicious users, or other very subtle security research is done, we have no idea what percentage of LE certificates are obtained by highly skilled and motivated malicious users, as compared to naive users who actually are following the Start Here page, and then getting any questions successfully answered in just minutes, as has been claimed.

It is not the fact that these people are motivated by greed that is the issue here at all. It is the simple possibility that the certificate statistics may not mean what everyone posting here thinks they mean.

1 Like

Back again so soon?

All Let’s Encrypt’s certificates are public documents, and they’re published automatically to the CT logs, that’s why we know independently how many there are. You’ve made some wild general assertions about these certificates but they don’t seem to be backed up by any actual facts.

1 Like

There is no such thing as an “HTTPS domain name”. There are domain names, for which one or more TLS certificates may or may not have been issued. In order to have LE issue a cert for a domain, the domain must exist and have authoritatively-published DNS records, and you must demonstrate control over that domain by (1) placing a cryptographically-determined piece of content at a specified URL on that domain, (2) creating a self-signed certificate for a cryptographically-determined subdomain, and serving that on your site, or (3) updating DNS records for that domain to include a cryptographically-determined token. So, if the scammers/spammers are going to try to occupy a large portion of the encrypted web, they must first own a lot of domains.

And once they own a domain, yes, they can get a cert for it. Why not? Your repeated reference to spammers and scammers is confusing, since it has no apparent connection to “mak[ing] Let’s Encrypt easy and simple”. It sounds, in fact, a lot like FUD, and if you hadn’t otherwise shown that you’re seriously interested in the project I’d just consider it such. But what are you really trying to suggest? The best I can piece together your post, it sounds like you’re saying that the issuance numbers are misleading, because some significant portion of the issued certs have been issued to spammers/scammers. Is that it? And if so, do you have any evidence to support it? And if not, please say plainly what you’re suggesting, and why.

I’m not especially interested in what spammers/scammers “may be” doing–they “may be” fornicating with dogs in rural South America, they “may be” perfecting cold fusion, or they “may be” teenagers living in their mothers’ basements and raking in their ill-gotten gains. What they “may be” doing has no bearing on this discussion. But if you have evidence of what they are doing, and it’s relevant to Let’s Encrypt, please share it.

You’ve made some good points here, some questionable ones, and some outright wrong ones (like the statement that you need a github account to use the git clone command). Mixing in borderline-FUD just weakens the rest of your points.

3 Likes

I agree here. Although it does give some information, it lacks clarity regarding third party clients and it’s actually hard to find documentation for the official client. I know certbot is being handled by the EFF, but we could better reference it’s documentation (especially in a section called “Getting Started”).

I disagree here. Let’s Encrypt is a CA, not a downloadable program. As part of it’s beta testing and initial release, it had an (underdeveloped) client to give people a chance to use the service. Now that the client is actually developed by a third party (the EFF) I don’t think LE is in the position to have a simple “download” link.

That said, LE should really update the Getting Started page to reflect the plethora of clients available and platforms supported, as well as a clear link to certbot that indicates it’s the officially recommended client for *nix systems. It should also reference a user’s first steps if they’re using a hosting service and don’t have command line access.

I’m aware of all of that, I’m just not sure why you think it matters.

First, Let’s Encrypt’s priority is securing websites, they’ve specifically said mail servers are not part of their mission statement. Secondly, although their certificates can be used for mail servers, it’s irrelevant - mail servers don’t need a CA signed cert to use encryption. They’re happy enough using opportunistic encryption (I know, I run one).

Lastly, whether mail from domain “viagra4U” is transported from MTA to MTA via LE secured TLS or not has absolutely no bearing on whether spam filters detect it. None. In fact, by the time the spam filter sees the email, it’s well past the MTA to MTA connection.

That’s why I said “don’t get distracted! Whether they use those certs for good or evil is irrelevant!”

And Let’s Encrypt uses Google’s blacklist to ensure they don’t provide certificates for possible malware domains (such as windowsnotificationcentre.com or itunes.tk). This has previously been discussed at length.

No, their opportunistic business people like any other. Low hanging fruit pays better than jumping through hoops for that extra 10c. To get around LE’s limits requires effort and virtually no payoff. Why not send spam opportunistically encrypted rather than from some random domain secured by LE? It doesn’t make sense. This really is nothing to lose sleep over, or even be mildly concerned about.

And the background information I thought was obvious was how mail works :wink:

Well, if they’re spammers, I can guarantee you they’re not highly skilled!

Sorry to harp on this, but there is absolutely no reason to think there is any significant number of malicious sites in that 90%. I know you say we simply don’t know unless research is done, but there is simply no gain to obtaining millions of LE certificates for use in spam. None. There’s actually a loss to anybody that does it. That’s time and effort with no real-world benefit.

Now, if you’re talking about malicious websites, LE uses Google’s blacklist to ensure they’re not legitimising that kind of malware (as I linked above).

Seriously, relax. It’s not just that there is no evidence of abuse either way, it’s that there is no evidence of abuse and reason against it.

Failing that, you could do a little research yourself and go through the issued certificates. One of LE’s key principles is transparency, so you can see every single certificate they’ve issued (as stated by the LE site).

It’s all good, it really is!

1 Like

I haven’t read every single post here, but I do agree with the basic issues the OP brought up. when I first started to use LE, I went to another site to get let’s encrypt installed and working correctly because their guides were so much better than what LE offered at the time.

have the docs improved on LE since then? I’m not sure…I haven’t needed to look since my initial install.

1 Like

danb35, I know that there is no such thing as an “HTTPS domain name”. That was a fast way to say “a domain name intended for use with modern HTTPS-level security to elevate maliciousness to a new leve”. I think I will just stick with everything I said. It’s not FUD, it’s a possibility that should be considered. As to the rest, tl;dr, sorry. I have spent too much time here.

1 Like

andrewjs18, Please pretend you are a new visitor interested in using LE. Read the home page, click the Get Started link. You will get to the start page. Please glance over the start page and let us know if you find it simple (looks like it will work) and inspiring (makes you want to try it out).

1 Like

That’s… really unfortunate. This is your topic, your thread, but you can’t be bothered to read the responses some of us have put a lot of time into?

I think this is a symptom of why a lot of people are having trouble - they want the outcome (secured website) but it’s too much effort if they have to do anything or understand anything.

Misconceptions about how mail works are common, which is why I went to the effort of explaining it for you. I was trying to put your mind at ease, but you don’t want to be at ease. Andrewjs18 displayed the same attitude - it was too much effort for him to read the posts in a thread he thinks is important.

There’s only so much the LE community can do to hold your hand. I’m really disappointed in your response :disappointed:

1 Like

DarkSteve, Well, if you’re going to respond like this, then I have to respond right back. Even if I have other things I need to do.

“I think this is a symptom of why a lot of people are having trouble…”

Nonsense. People report trouble here because they try to follow the scant documentation here and it doesn’t work for them. Insulting these people as not wanting to spend effort is not supported by the hours of work they frequently report. I don’t mind your fair criticisms of me, I’m not in the audience of LE. I’m a retired software engineer who has seen problems with documentation and knows how they can kill a project.

The goal of LE is to secure the Web. There is no reason that owners of domain names must endure pain in using LE when LE claims to eliminate the usual pain of obtaining and installing certificates.

I don’t think I have any misconceptions about how email works, and my discussions here do not focus on email, other than to mention spammers (those who scrape the Web for addresses, and send out random GETs or POSTs in the hope of successful Javascript, database, and other injections).

If it is really true that random domains from the transparency lists don’t resolve to countries known to harbor these criminals, then I am wrong about this one point, and there are lots of people out there currently using LE successfully. In spite of the bad documentation.

What’s this about “holding my hand”? Am I asking basic questions that should be answered by my reading the documentation? No, I’m complaining very specifically about our poor documentation.

And, yes, I have limited time. Let’s find some documentation folks to fix our documentation, and stop fighting battles over words.

If you have an important point in your long recent post that you really would like to hear my response to, quote it and I’ll respond. But make sure it is a valid point, not your belief or opinion presented as a fact. Almost all of my postings in this thread is opinion and marked as such.

1 Like

It only bears serious consideration if there’s evidence that it’s happening. Thus far you’ve provided none, and shown no inclination to provide any. To be of any relevance, the evidence would need to show that some significant portion of the certs issued by Let’s Encrypt to date have been to spammers/scammers.

I’m sorry you don’t feel it’s worth your time to discuss an issue you raised. If you’re trying to not look like a troll, this isn’t a way to do it.

1 Like

@DarkSteve, why do you sound angry with me? if let’s encrypt wants people to use their product to help move encryption forward at a greater pace, then their software and documentation needs to be easy to use…it is not. I think that’s why this thread was created. there have been similar issues raised in the past about the friendly nature of trying to use let’s encrypt…

wanting people to search for threads on a forum to help get software installed isn’t what I’d consider making the software easy to use.

edited to add: I definitely appreciate the time and effort going into the project, for sure.

1 Like

It is. Most of the time. But as usual, there are a lot of different possibilities in server configuration. Most of the time, you’re set within a few minutes. But the documentation can’t cover every single different server configuration. So yes, for some people it’s gonna be a little more different. But can you blame LE/EFF for that? Should they cover e-ver-y sin-gle server config? Or is a coverage of 99% enough? You can also argue people are responsible for their own non-standard server configuration…

1 Like

Osiris, I agree that neither LE nor EFF should address every possible configuration of every possible webserver. But there should be detailed documentation explaining the principles of making LE/EFF work in any situation. Instead of promising “no pain”, LE/EFF should promise something more reasonable and achievable.

This is not like a programming language, which can be expected to work immediately upon download. The documentation should be much more detailed, not just grandiose promises. This is all my opinion.

1 Like

I definitely agree with you that every single configuration cannot possibly be supported or written out in a guide because that list would never end.

all that said though, I just went through the ‘getting started’ page a minute ago and it’s vastly improved since I last looked at it. kudos to the team for that.

I think the doc could be improved by explaining in more detail what’s going on with the commands being used - switches, the different types of ways to obtain certs, which iteration of the command you should use if your site is behind a proxy like cloudflare or incapsula, how to set up a crontab to automatically run the renewal script, etc…that all helps to keep people on THIS site rather than looking at other sites for guides.

2 Likes

Just check it out by yourself. BTW: Could you name me this “harbor countries”?
If looking through the whole CT log is too much for you, look at the stats. Have a look at the most commonly-used TLDs and ask yourself: Are .com, .net and .de such harbour countries for criminals?

That’s not spamming, that’s web scraping or vulnerability scanning done by bots.

Still don’t know what criminals/spammers/whatever have to do with making a simple web UI, but well… Rather get on-topic now. :smiley:

1 Like

Well, after a few weeks - life interfered - and two more days of Google, trial and error, and frustration, I seem to have a secure web site. Or at least a secure Apache placeholder page.

Or at least an Apache placeholder page that this says is secure, but which Firefox thinks is not due to “mixed content.”

Thank to the authors of the twenty-seven different web pages and how-tos that I have bookmarked for future reference.

I’ll wrap this up with a some specific observations. I’ve worked in a few highly technical situations, including training and supporting end users, and even drafting documentation, so feel I can speak out on this.

  1. This is important stuff, not some half-baked Wordpress plugin that rotates cat pictures. If the certificate is not configured right the consequences can be large and dangerous.

  2. To the vast majority of end users there is no distinction between “LetsEncrypt” and “Certbot.” The LE pages imply that the latter is a function of the former, and any ordinary person will assume that they are one project.

End users don’t need or want to be lectured on the distinction. They just want to make the thing work.

  1. It’s pretty obvious that LE/CB expect a moderately high level of knowledge, and some specific levels of user access. These things (as has been pointed out) need to be front and centre on the “Get Started” page. (You need This software installed; this level of access to your server; and the skills to do these tasks.)

If the folks behind LE/CB don’t want to support less skilled users they should post a big, nasty warning to scare them off.

  1. Regardless of who LE/CB wants as end users of their products, there has got be a single source for reliable, authoritative information. In a crunch it should be possible to get most common questions answered on either the LE or CB web sites.

Trolling though dozens of Google results is almost always a recipe for disaster.

What I will suggest, based on a lot of years of different projects that are structurally similar, is this:

Whoever is making decisions at the topmost regions of LetsEncrypt and/or CertBot need to sit down and decide who their intended user base is.

It doesn’t honestly matter whether they want to serve the high priests of server land, or the guy with a five page GoDaddy web site on shared hosting, but it does matter that, having made that decision, they ensure that resources are available to draft the best possible support materials, suitable for the skills and knowledge level of the intended audience.

Until then there’s a strong indication that language like “It’s free, automated, and open” and “automates away the pain and lets site operators turn on and manage HTTPS with simple commands” should be removed because it seems to misrepresent what is being offered.

And one final note. “Well, it worked for me,” is not, and never will be an adequate response to anything.

3 Likes

Appalbarry, Thank you. Finally, a real person who really tried it all out and tells it like it is. I like that you write near the top of this thread that people are currently expecting a “one-click” solution, based on the home page, and not finding it. I hope that whoever can edit the home and startup pages will do a complete rewrite or find someone who can. In my opinion, we either need to tone down our promises to fit the current software, or improve the software to fulfill our promises. If it is supposed to just work ‘out of the box’, and doesn’t, how can this be kept a secret? I don’t see those central to this project doing experiments to measure or demonstrate usability. I see only grand claims and beliefs. Then they blame me (busy with my own life and nonprofit organization) for not being “specific” in my criticisms. I’ve been specific in almost every posting of mine on this thread. People should really listen, early, to those who reveal flaws, not sweep them (the people or the flaws) under a rug. Again, just my opinion.

1 Like

Your mixed content warning is unrelated to Let’s Encrypt. It means the site mixes secure and insecure content, typically because it uses images from a non-HTTPS website. The test you linked from SSL Labs has diagnose that your secure web server is set up correctly, which is the part Let’s Encrypt helped with. The good news is that “mixed content” is an issue you don’t need to ask about here, it’s a general web dev type question, one anybody with any sort of secure server might have just like if you couldn’t figure out how to add an image to a page, or were trying to debug a PHP script that doesn’t work.

Breaking out “certbot” the client and Let’s Encrypt the service was done deliberately, albeit belatedly. It doesn’t help to try to squish them back together now. LOTS of people are successfully using other Let’s Encrypt clients, questions about that are on-topic here, but wouldn’t be covered in a FAQ about certbot. I think you’d have less confusion if that distinction had been in place from day one, not more.

1 Like