Looks like something is wrong with how Let's Encrypt issues certs. I have used NPM, Nginx, Caddy, etc. on a range of domains that are working, have been known to be working, or have been created within the last 24 hours. My issuer (Cloudflare) is also working fine. All services I have used to request a new cert have been met with an error from Let's Encrypt.
Welcome to the Let's Encrypt Community! 
There was about a 3 hour window today when Let's Encrypt was performing maintenance on their production servers. Are you still seeing issues?
1 Like
Yes I am
Is there any information you can post (e.g. logs, domain name(s)) to help us look into this?
2 Likes
We'll need more info than that to help you. Please answer as much as you can from the form you should have been shown when posting in Help section. I re-posted that form below.
I am puzzled by your comments. NPM itself is not an ACME Client so cannot request certs. It uses acme.sh or Certbot for that. nginx itself is also not an ACME Client. Cloudflare does not issue certs although it can acquire them on your behalf when enabling Universal SSL.
Let's Encrypt typically issues over 5 million certs per day so a problem from one person is likely something unique to their situation.
=============================================
Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.
My domain is:
I ran this command:
It produced this output:
My web server is (include version):
The operating system my web server runs on is (include version):
My hosting provider, if applicable, is:
I can login to a root shell on my machine (yes or no, or I don't know):
I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
3 Likes
I am puzzled by your comments. NPM itself is not an ACME Client so cannot request certs. It uses acme.sh or Certbot for that. nginx itself is also not an ACME Client. Cloudflare does not issue certs although it can acquire them on your behalf when enabling Universal SSL.
I'm just using those as examples that integrate Let's Encrypt. I've used all three before to fetch a new certificate.
=============================================
My domain is: I had to go through a weeks worth of messages to remove my domain from here as I don't want the extra traffic invited by publishing this. I have also tested two different subdomains just in case, one a day old, the other months old.
I ran this command: I have used NPM and Caddy (sudo systemctl restart caddy)
It produced this output:
NPM
Deleting file: /data/nginx/proxy_host/30.conf
Deleting file: /data/nginx/proxy_host/30.conf.err
Could not delete file: {
"errno": -2,
"code": "ENOENT",
"syscall": "unlink",
"path": "/data/nginx/proxy_host/30.conf.err"
}
CMD: /usr/sbin/nginx -t -g "error_log off;"
Reloading Nginx
CMD: /usr/sbin/nginx -s reload
Requesting Let'sEncrypt certificates for Cert #9: poopoohead.example.com
Command: certbot certonly --config '/etc/letsencrypt.ini' --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-9" --agree-tos --authenticator webroot --email 'example@gmail.com' --preferred-challenges "dns,http" --domains "poopoohead.example.com"
CMD: certbot certonly --config '/etc/letsencrypt.ini' --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-9" --agree-tos --authenticator webroot --email 'example@gmail.com' --preferred-challenges "dns,http" --domains "poopoohead.example.com"
Deleting file: /data/nginx/temp/letsencrypt_9.conf
CMD: /usr/sbin/nginx -t -g "error_log off;"
Reloading Nginx
CMD: /usr/sbin/nginx -s reload
Saving debug log to /tmp/letsencrypt-log/letsencrypt.log
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/letsencrypt-log/letsencrypt.log or re-run Certbot with -v for more details.
Caddy
{"level":"info","ts":1741395630.970068,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"example.yot>
{"level":"info","ts":1741395630.9708796,"logger":"http","msg":"using ACME account","account_id":"https://acme-staging->
{"level":"info","ts":1741395631.4061992,"msg":"trying to solve challenge","identifier":"test.example.com","chall>
{"level":"error","ts":1741395631.7708013,"msg":"challenge failed","identifier":"test.example.com","challenge_typ>
{"level":"error","ts":1741395631.770926,"msg":"validating authorization","identifier":"test.example.com","proble>
{"level":"info","ts":1741395632.9070494,"msg":"trying to solve challenge","identifier":"test.example.com","chall>
{"level":"error","ts":1741395633.2778335,"msg":"challenge failed","identifier":"test.example.com","challenge_typ>
{"level":"error","ts":1741395633.2779365,"msg":"validating authorization","identifier":"test.example.com","probl>
{"level":"error","ts":1741395633.2780135,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifi>
{"level":"error","ts":1741395633.2780528,"logger":"tls.obtain","msg":"will retry","error":"[test.example.com] Ob>
The operating system my web server runs on is (include version):
Distributor ID: Ubuntu
Description: Ubuntu 24.04.2 LTS
Release: 24.04
Codename: noble
My hosting provider, if applicable, is:
Cloudflare
I can login to a root shell on my machine (yes or no, or I don't know):
Yes
I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
No
The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot 2.9.0
Every domain name in a certificate is published in public Certificate Transparency (CT) logs. Both good and bad actors scan those. It is not possible to keep your domain name hidden from bots and scanners. For that matter, the entire universe of IPv4 addresses can also readily be scanned (and are) directly.
NPM is notorious for not showing useful error messages. And, nothing you extracted from them says anything useful. Since it is using Certbot you could look into the log file whose name it gives. Look for something like
Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: www.example.com
Type: connection
Detail: 11.11.11.11: Fetching http://www.example.com/.well-known/acme-challenge/sampleToken:
Connection refused
The "Connection Refused" is the reason for that failure. We often see "timeout" or some http error code like 404, 403 or similar. Those are helpful bits of info.
As for Caddy, none of that log is very helpful either. The right-most bits are truncated.
There is almost certainly a general comms or server config problem on your end. With more info maybe we can help locate it.
For Caddy, it should be easy and perhaps asking on their community forum might give you rapid answer.
2 Likes
There's no obvious problems on the side of Let's Encrypt. Validation issuance volume and error rates are normal.
6 Likes
Every domain name in a certificate is published in public Certificate Transparency (CT) logs. Both good and bad actors scan those. It is not possible to keep your domain name hidden from bots and scanners.
I understand this, however I still have my reservations.
nothing you extracted from them says anything useful. Since it is using Certbot you could look into the log file whose name it gives.
I checked through all of my logs and even did a purge to re-run it again (still getting internal error from Let's Encrypt). The log files that are created have nothing down in them. The only thing I see while running NPM is that it fails at Let's Encrypt.
As for Caddy, none of that log is very helpful either.
This is all Caddy gives me, again, failing at Let's Encrypt
There is almost certainly a general comms or server config problem on your end. With more info maybe we can help locate it.
The info I provided was deemed not "useful". What more info do you need and where would it come from? I provided what logs and errors I'm getting on my end.
Can you upload the entire Certbot log? You'll need to copy it to a .txt file first. An earlier post of yours showed its name: Saving debug log to /tmp/letsencrypt-log/letsencrypt.log
The Caddy log might be useful but the part you posted was cutoff on the right margin. The useful part of those log lines are missing. Besides, the Caddy Community forum is a better place to seek advice on Caddy setup issues.
As for NPM, I can nearly guarantee you will be seeing some sort of timeout or possibly "404" or "403" error. All of which point to an NPM or comms config problem. Getting NPM sorted out requires expertise to work through all the "moving parts" it configures.
Sorting out a comms problem requires specific info from you like the domain name. We can't possibly test and debug a connection problem without knowing what Let's Encrypt is connecting to.
The Certbot request setup by NPM used the HTTP Challenge. That needs a request of the below format to reach your domain and return the token setup by NPM/Certbot. To DIY this try a curl request from the public internet and see what happens. You should be able to reach the server that NPM is configuring for this. Normally this test gets an HTTP 404 error. If you see timeout or response headers from the "wrong" server that helps locate the problem.
curl -i http://example.com/.well-known/acme-challenge/Test404
Also try this test site: https://letsdebug.net
As for Caddy, I believe it uses a TLS-ALPN Challenge by default which is slightly different but still involves an LE Server sending requests to your system and getting the right response. I am not expert enough at Caddy to help debug problems with it. We don't see them much here as they have a nice community of their own. https://caddy.community/
2 Likes
I checked the logs and log directory for NPM. There is literally, and I mean literally, nothing in the log file. 0 lines.
My mistake, didn't see that. Here's the full output from Caddy:
linux systemd[1]: Started caddy.service - Caddy.
{"level":"info","ts":1741466454.7633498,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/var/lib/caddy/.local/share/caddy","instance":"b5f7b91d-8da0-4c3f-b59d-40ee6f955a65","try_again":1741552854.7633479,"try_again_in":86399.99999951}
{"level":"info","ts":1741466454.763471,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1741466454.7637062,"logger":"tls.obtain","msg":"acquiring lock","identifier":"peepeepoopoo.example.com"}
{"level":"info","ts":1741466454.764322,"logger":"tls.obtain","msg":"lock acquired","identifier":"peepeepoopoo.example.com"}
{"level":"info","ts":1741466454.7643917,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"peepeepoopoo.example.com"}
{"level":"info","ts":1741466454.7647614,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["peepeepoopoo.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"example@gmail.com"}
{"level":"info","ts":1741466454.7647734,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["peepeepoopoo.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"example@gmail.com"}
{"level":"info","ts":1741466454.7647815,"logger":"tls.issuance.acme","msg":"using ACME account","account_id":"https://acme-v02.api.letsencrypt.org/acme/acct/2270381986","account_contact":["mailto:example@gmail.com"]}
{"level":"info","ts":1741466455.2140088,"msg":"trying to solve challenge","identifier":"peepeepoopoo.example.com","challenge_type":"http-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
Checking the journal for Caddy also gives no errors.
They sent me here.
It fails, saying there is in fact an A record for it and showing my public address, but claims port 80 is not being reach by Let's Encrypt. I've checked my entire server, router, and ISP:
- My server is listening and can be reached on port 80
- My router has port 80 forwarded to the server
- My ISP does not block any ports
The only errors I'm getting are from Let's Encrypt.
You'll need to ask about that on the NPM github then.
The Let's Debug test site checks connectivity two ways. The first is by sending requests from its own test server (somewhere in Europe). The second is by running a test using the Let's Encrypt staging server. This second test should fail in a specific way. If it fails other ways it indicates a problem on your end.
Use the "verbose" link and show the output lines that should look like I posted earlier that you should see from Certbot. They will be similar for the Let's Debug test.
From the public internet? What does a test site like below show for connectivity?
Be sure to click the "HTTP" button. A ping or port test is not sufficient for HTTP connections.
5 Likes
As for the Caddy log. Note I am not a Caddy expert. But, this is the last line:
{"level":"info","ts":1741466455.2140088,"msg":"trying to solve challenge","identifier":"peepeepoopoo.example.com","challenge_type":"http-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
Looks like Caddy is setting up a cert request using an HTTP Challenge. But, it does not report the result of that. The LE Server issues pretty useful error messages for failed challenges. It is up to the requesting ACME Client (Caddy) to report them.
What is the link for your post at the Caddy Community?
2 Likes
So... the server runs two tests and should fail the second... what's the reasoning behind this? What benefit does a failed second test provide to someone attempting to get a cert from Let's Encrypt? This doesn't make any sense.
What other ways? How can I see if it's on my end of Let's Encrypt, Caddy, and NPM all point to Let's Encrypt being the issue? This doesn't make any since.
You only said the NPM was pointing to Certbot, but you also said NPM is giving "useless" information and I needed to go the NPM's github for help. The only thing you said to run with certbot is to check it's version. Nothing is reporting like the example you gave above, and even so, you did not clarify what sepcifically you wanted from Certbot, only that I could "look into the log file whose name it gives", which, again, you said to post on NPM to figure out why there's nothing there.
My other services are reachable, but only from a cert with Cloudflare or using a Cloudflare Tunnel. I can't test this when the services are not reachable since Let's Encrypt refuses to issues certs of which I can test this with.
Again, the only thing I get from the Let's Debug site is "a request to this address over port 80 did not succeed. Your web server must have at least one working IPv4 or IPv6 address." which is not correct. I can reach my services right now from anywhere and any device over the public internet. Let's Encrypt is having issues reaching my services.
I feel as if there's confusion on this too. I am using Cloudflare right now to handle DNS since Let's Encrypt is failing to issue certs. I have switch Cloudflare to DNS only when attempting to get a cert from Let's Encrypt; LE fails, even though the A record is present and check-host.net can see it.
The Let's Debug site obviously cannot successfully get a certificate from your site, as it doesn't have any method of successfully passing the challenge (as it doesn't magically have access to your server, luckily..). Thus any issuance attempt from Let's Encrypt ultimately fails. However, issuance can fail in many different ways and only one of those failures is expected. If any error besides the expected failure mode presents itself, it will show on the Let's Debug output. However, the expected failure mode will not.
2 Likes
Thank you for phrasing this in a better way. I understood from previous headaches when LE went down a while back that the testing site had an expected error, but the phrasing made me things had changed. Too much misinformation out there right now....
Let's Debug runs two tests. It is a debugging test site. Each test proves different things.
Much of my comments won't make sense because I have to talk generalities. If you provided a domain name, which we have asked for multiple times, I could be more specific.
No, I said NPM was using Certbot as the ACME Client to request a cert. NPM configures your system and Certbot to coordinate the cert request and response.
I never said that.
Yes, if the log file that Certbot produced is empty then something went wrong with how NPM manages your system. You'll need to ask them about that.
An HTTP Challenge uses HTTP. No cert is involved.
That's a new wrinkle. Would have been helpful to know that earlier.
I am going to have to see some proof of that. Was that other test site able to reach you using HTTP test?
3 Likes
My guess would be that cloudflares proxying is preventing http domain validation from working against any of your services. If you settings are strict enough then http just doesn't happen. Trying switching off DNS proxying in cloudflare for a domain and see if you certificate requests work again.
2 Likes
Adding to what @webprofusion correctly said, turning off proxying (setting to gray cloud) has the added benefit of bypassing Cloudflare WAF rules, which in themselves can interfere with ACME HTTP-01 challenges.
1 Like