Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems: ... Connection refused

Hi all,

I had certbot running as standalone. Renewals in the past did not cause any problem but this time I'm screwed and have no idea how to handle this.

In the past it was all pretty easy. I have an nginx on my pi. In the past I turned it off for renewals, started certbot and all was good.

I wonder if sudo apt --purge remove certbot, erase /etc/letsencrypt by hand and sudo apt install certbot is a wise thing. As the outdated certificates still work I'd like to have an experienced user have a look ...

Any hints are highly appreciated!

root@pi# certbot --version

certbot 2.1.0

root@pi# netstat -tunlp

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:5232            0.0.0.0:*               LISTEN      1752/docker-proxy   
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      32394/cupsd         
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      800/sshd: /usr/sbin 
tcp        0      0 0.0.0.0:53              0.0.0.0:*               LISTEN      898/pihole-FTL      
tcp        0      0 127.0.0.1:42291         0.0.0.0:*               LISTEN      783/containerd      
tcp        0      0 0.0.0.0:888             0.0.0.0:*               LISTEN      898/pihole-FTL      
tcp        0      0 0.0.0.0:5000            0.0.0.0:*               LISTEN      1586/docker-proxy   
tcp        0      0 127.0.0.1:6011          0.0.0.0:*               LISTEN      82406/sshd: pi@pts/ 
tcp        0      0 127.0.0.1:6010          0.0.0.0:*               LISTEN      82358/sshd: pi@pts/ 
tcp        0      0 127.0.0.1:6012          0.0.0.0:*               LISTEN      82431/sshd: pi@pts/ 
udp        0      0 0.0.0.0:53              0.0.0.0:*                           898/pihole-FTL      
udp        0      0 0.0.0.0:123             0.0.0.0:*                           898/pihole-FTL      
udp        0      0 0.0.0.0:5353            0.0.0.0:*                           579/avahi-daemon: r 
udp        0      0 0.0.0.0:59035           0.0.0.0:*                           579/avahi-daemon: r

root@pi# certbot certonly -v -d dav.knowscore.de -d knowscore.de -d knowscore.social -d transl.knowscore.de -d www.knowscore.de -d knowscore.social -d gottteam.social -d hammeln.social -m java@wispa.de --standalone

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Requesting a certificate for dav.knowscore.de and 6 more domains
Performing the following challenges:
http-01 challenge for dav.knowscore.de
http-01 challenge for gottteam.social
http-01 challenge for hammeln.social
http-01 challenge for knowscore.de
http-01 challenge for knowscore.social
http-01 challenge for transl.knowscore.de
http-01 challenge for www.knowscore.de
Waiting for verification...
Challenge failed for domain dav.knowscore.de
Challenge failed for domain gottteam.social
Challenge failed for domain hammeln.social
Challenge failed for domain knowscore.de
Challenge failed for domain knowscore.social
Challenge failed for domain transl.knowscore.de
Challenge failed for domain www.knowscore.de
http-01 challenge for dav.knowscore.de
http-01 challenge for gottteam.social
http-01 challenge for hammeln.social
http-01 challenge for knowscore.de
http-01 challenge for knowscore.social
http-01 challenge for transl.knowscore.de
http-01 challenge for www.knowscore.de

Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems:
  Domain: dav.knowscore.de
  Type:   connection
  Detail: 91.10.78.139: Fetching http://dav.knowscore.de/.well-known/acme-challenge/rtcsIZmETfoLIhEpu75SzTVfYpVs47JmtKhO6zOzl-s: Connection refused

  Domain: gottteam.social
  Type:   connection
  Detail: 91.10.78.139: Fetching http://gottteam.social/.well-known/acme-challenge/f_U-TCe8hSY61G8rhVpotErAnv28LXwYwRc4y6XEXrs: Connection refused

  Domain: hammeln.social
  Type:   connection
  Detail: 91.10.78.139: Fetching http://hammeln.social/.well-known/acme-challenge/iPkX-8eiSJWiYbBJeWT7bhbXo6IC-3qX5yxG5IePurM: Connection refused

  Domain: knowscore.de
  Type:   connection
  Detail: 91.10.78.139: Fetching http://knowscore.de/.well-known/acme-challenge/Wrro5L4GUZxwlWqvF5d3-l3b3sPSqqdCKU1qFbq5xM8: Connection refused

  Domain: knowscore.social
  Type:   connection
  Detail: 91.10.78.139: Fetching http://knowscore.social/.well-known/acme-challenge/zwwuXkI9IgAp74Ukvvs5PdRXmawLWBRIr-0e_8bDaY8: Connection refused

  Domain: transl.knowscore.de
  Type:   connection
  Detail: 91.10.78.139: Fetching http://transl.knowscore.de/.well-known/acme-challenge/TmfsRA3KFeELayeSUIbnxySdvnxpoWPghT42oKqqKFU: Connection refused

  Domain: www.knowscore.de
  Type:   connection
  Detail: 91.10.78.139: Fetching http://www.knowscore.de/.well-known/acme-challenge/j8bjw00Vbcc7KfDp3gfVS1tBMTwEVpHyTEjwsa8rauk: Connection refused

Hint: The Certificate Authority failed to download the challenge files from the temporary standalone webserver started by Certbot on port 80. Ensure that the listed domains point to this machine and that it can accept inbound connections from the internet.

Cleaning up challenges
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

root@pi# more /var/log/letsencrypt/letsencrypt.log

2025-03-24 12:24:20,563:DEBUG:certbot._internal.main:certbot version: 2.1.0
2025-03-24 12:24:20,564:DEBUG:certbot._internal.main:Location of certbot entry point: /usr/bin/certbot
2025-03-24 12:24:20,564:DEBUG:certbot._internal.main:Arguments: ['-v', '-d', 'dav.knowscore.de', '-d', 'knowscore.de', '-d', 'knowscore.social', '-d', 'transl.knowscore.de', '-d', 'www.knowscore.de', '-d', 'knowscore.social', '-d', 'gottteam.social', '-d', 'hammeln.social', '-m', 'java@w
ispa.de', '--standalone']
2025-03-24 12:24:20,565:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2025-03-24 12:24:20,585:DEBUG:certbot._internal.log:Root logging level set at 20
2025-03-24 12:24:20,586:DEBUG:certbot._internal.plugins.selection:Requested authenticator standalone and installer None
2025-03-24 12:24:20,587:DEBUG:certbot._internal.plugins.selection:Single candidate plugin: * standalone
Description: Spin up a temporary webserver
Interfaces: Authenticator, Plugin
Entry point: standalone = certbot._internal.plugins.standalone:Authenticator
Initialized: <certbot._internal.plugins.standalone.Authenticator object at 0x7f7b94a510>
Prep: True
2025-03-24 12:24:20,588:DEBUG:certbot._internal.plugins.selection:Selected authenticator <certbot._internal.plugins.standalone.Authenticator object at 0x7f7b94a510> and installer None
2025-03-24 12:24:20,588:INFO:certbot._internal.plugins.selection:Plugins selected: Authenticator standalone, Installer None
2025-03-24 12:24:20,901:DEBUG:certbot._internal.main:Picked account: <Account(RegistrationResource(body=Registration(key=None, contact=(), agreement=None, status=None, terms_of_service_agreed=None, only_return_existing=None, external_account_binding=None), uri='https://acme-v02.api.letse
ncrypt.org/acme/acct/1419318876', new_authzr_uri=None, terms_of_service=None), 9d2d0c0fe73246e47855b4f56ee11981, Meta(creation_dt=datetime.datetime(2023, 11, 17, 14, 25, 30, tzinfo=<UTC>), creation_host='pi5', register_to_eff=None))>
2025-03-24 12:24:20,903:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2025-03-24 12:24:20,907:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org:443
2025-03-24 12:24:21,457:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 1042
2025-03-24 12:24:21,458:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Mon, 24 Mar 2025 11:24:21 GMT
Content-Type: application/json
Content-Length: 1042
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "profiles": {
      "classic": "https://letsencrypt.org/docs/profiles#classic",
      "shortlived": "https://letsencrypt.org/docs/profiles#shortlived (not yet generally available)",
      "tlsserver": "https://letsencrypt.org/docs/profiles#tlsserver (not yet generally available)"
    },
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.5-February-24-2025.pdf",
    "website": "https://letsencrypt.org"
  },
  "newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",
  "renewalInfo": "https://acme-v02.api.letsencrypt.org/draft-ietf-acme-ari-03/renewalInfo",
  "revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert",
  "wnV72HSxvbk": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417"
}
2025-03-24 12:24:21,459:DEBUG:certbot._internal.display.obj:Notifying user: Requesting a certificate for dav.knowscore.de and 6 more domains
2025-03-24 12:24:21,465:DEBUG:certbot.crypto_util:Generating ECDSA key (2048 bits): /etc/letsencrypt/keys/0003_key-certbot.pem
2025-03-24 12:24:21,471:DEBUG:certbot.crypto_util:Creating CSR: /etc/letsencrypt/csr/0003_csr-certbot.pem
2025-03-24 12:24:21,474:DEBUG:acme.client:Requesting fresh nonce
2025-03-24 12:24:21,474:DEBUG:acme.client:Sending HEAD request to https://acme-v02.api.letsencrypt.org/acme/new-nonce.
2025-03-24 12:24:21,657:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "HEAD /acme/new-nonce HTTP/1.1" 200 0
2025-03-24 12:24:21,657:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Mon, 24 Mar 2025 11:24:21 GMT
Connection: keep-alive

root@pi# cat /var/log/letsencrypt/letsencrypt.log

2025-03-24 12:24:27,647:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 106, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 206, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2025-03-24 12:24:27,648:DEBUG:certbot._internal.error_handler:Calling registered functions
2025-03-24 12:24:27,648:INFO:certbot._internal.auth_handler:Cleaning up challenges
2025-03-24 12:24:27,654:DEBUG:certbot._internal.plugins.standalone:Stopping server at 0.0.0.0:80...
2025-03-24 12:24:28,070:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 33, in <module>
    sys.exit(load_entry_point('certbot==2.1.0', 'console_scripts', 'certbot')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line 1736, in main
    return config.func(config, plugins)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line 1590, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line 138, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/client.py", line 516, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/client.py", line 428, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/client.py", line 496, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 106, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 206, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.
2025-03-24 12:24:28,080:ERROR:certbot._internal.log:Some challenges have failed.

No, that would not be wise :slight_smile:

The error is for a connection problem to those domains (your public IP) using HTTP and port 80. I see HTTPS and port 443 are working fine but I get these same connection failures right now.

You mention pi so I assume this is a residential ISP. Check your router to ensure port 80 is still allowed and routing to the correct server (local IP). Check any other firewalls you have to ensure port 80 is allowed.

2 Likes

There has been no change to router-config. The router should pass port 80 and 443 directly to the pi (192.168.178.42):

When nginx is running the webserver is accessible so I thing the router passes ports properly.

The pi does not have any firewall enabled:

root@pi# iptables -L

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy DROP)
target     prot opt source               destination         
DOCKER-USER  all  --  anywhere             anywhere            
DOCKER-ISOLATION-STAGE-1  all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain DOCKER (4 references)
target     prot opt source               destination         
ACCEPT     tcp  --  anywhere             172.17.0.2           tcp dpt:5000
ACCEPT     tcp  --  anywhere             172.17.0.3           tcp dpt:5232

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target     prot opt source               destination         
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere            
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere            
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere            
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            

Chain DOCKER-ISOLATION-STAGE-2 (4 references)
target     prot opt source               destination         
DROP       all  --  anywhere             anywhere            
DROP       all  --  anywhere             anywhere            
DROP       all  --  anywhere             anywhere            
DROP       all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            

Chain DOCKER-USER (1 references)
target     prot opt source               destination         
RETURN     all  --  anywhere             anywhere

root@pi# ufw status 

bash: ufw: command not found

So certbot should be able to use these ports in the back on the pi.

As I read the log certbot relies on an nginx. Is it possible that the installed nginx is used and that this existing installation with some wired (twisted) config could cause these failures?

Is it accessible via HTTP (not HTTPS) from outside of your LAN?

Really, stopping nginx to run certbot in standalone mode is a poor configuration; a much better arrangement would be to just get the cert using the webroot authenticator. But if your ISP is blocking port 80, that won't make a difference.

3 Likes

Not with HTTP (port 80). Only port 443. See: Let's Debug

I get the same failure to your "home" page from my own test server in the USA:

curl -i http://dav.knowscore.de
curl: (7) Failed to connect to dav.knowscore.de port 80 after 264 ms: 
Connection refused

curl -Ik https://dav.knowscore.de
HTTP/1.1 302 Found
Server: nginx/1.22.1

That is in the response from the Let's Encrypt API.

2 Likes

Strange, very strange! Even from a closer point, I get

root@pi# curl -i http://dav.knowscore.de
curl: (7) Failed to connect to dav.knowscore.de port 80 after 6 ms: Couldn't connect to server

But using a browser (https://dav.knowscore.de, port 443) the normal nginx on the pi does answer (with an expired certificate though).

It looks like the router that usually does pass incoming requests from ports 80 / 443 to backend-pi 192.168.178.42 except for certbot's challenges.

Confusing because what certainly did change since last successful renewal is some software on the pi (Python!) while the router was not touched at all.

Port 443 is not port 80. I'm likewise getting "connection refused" on port 80. Let's Encrypt must be able to connect on port 80, which neither I, Mike, nor Let's Encrypt (nor you, for that matter) are able to do.

3 Likes

OK, desperation-time. Let's see if a reconnect of this DSL-router (with a new IP) changes the situation ...

1 Like

Agreed.

From around the world no Port 80 connections are working Permanent link to this check report.

$ nmap -Pn -p80,443 dav.knowscore.de
Starting Nmap 7.94SVN ( https://nmap.org ) at 2025-03-24 16:51 UTC
Nmap scan report for dav.knowscore.de (91.10.88.7)
Host is up (0.17s latency).
rDNS record for 91.10.88.7: p5b0a5807.dip0.t-ipconnect.de

PORT    STATE  SERVICE
80/tcp  closed http
443/tcp open   https

Nmap done: 1 IP address (1 host up) scanned in 2.29 seconds

1 Like

Until some voodoo makes that port 80 accessible I can not do anything but pray?

I'm not even seeing your HTTPS nginx on port 443.

Currently only port 5060 seems to be open?

PORT     STATE  SERVICE
80/tcp   closed http
113/tcp  closed ident
443/tcp  closed https
1080/tcp closed socks
5060/tcp open   sip

But I don't even see that port in your netstat output.

Are you sure the portmaps are to the correct host in your LAN? Or is that just the Fritz!Box listening on port 5060?

1 Like

Hi all,

it worked for me in two steps. First I had to remove the port-forwarding on the router (FRITZ!box). Second I had to set it up again - and after that it all worked like cream again.

Thank you, let's encrypt!

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.