Http-01 challenge fails using Certbot with step-ca

Help
1

My domain is: netbox.intern.sfgroup.de

I ran this command: certbot renew --debug-challenges -vv

It produced this output:

Summary 
Root logging level set at 10
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Notifying user: Processing /etc/letsencrypt/renewal/netbox.intern.sfgroup.de.conf

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/netbox.intern.sfgroup.de.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Requested authenticator None and installer None
Requested authenticator None and installer None
Should renew, less than 30 days before certificate expiry 2025-09-19 13:37:15 UTC.
Certificate is due for renewal, auto-renewing...
Requested authenticator nginx and installer nginx
Single candidate plugin: * nginx
Description: Nginx Web Server plugin
Interfaces: Authenticator, Installer, Plugin
Entry point: EntryPoint(name='nginx', value='certbot_nginx._internal.configurator:NginxConfigurator', group='certbot.plugins')
Initialized: <certbot_nginx._internal.configurator.NginxConfigurator object at 0x73df03e61d60>
Prep: True
Single candidate plugin: * nginx
Description: Nginx Web Server plugin
Interfaces: Authenticator, Installer, Plugin
Entry point: EntryPoint(name='nginx', value='certbot_nginx._internal.configurator:NginxConfigurator', group='certbot.plugins')
Initialized: <certbot_nginx._internal.configurator.NginxConfigurator object at 0x73df03e61d60>
Prep: True
Selected authenticator <certbot_nginx._internal.configurator.NginxConfigurator object at 0x73df03e61d60> and installer <certbot_nginx._internal.configurator.NginxConfigurator object at 0x73df03e61d60>
Plugins selected: Authenticator nginx, Installer nginx
Picked account: <Account(RegistrationResource(body=Registration(key=None, contact=(), agreement=None, status=None, terms_of_service_agreed=None, only_return_existing=None, external_account_binding=None), uri='https://step-ca.sfg.local:8443/acme/acme/account/UAdkT1me25orgQyucmdEHL1dU8rHnLuJ', new_authzr_uri=None, terms_of_service=None), b6a6f36b0067644817008dba178c0738, Meta(creation_dt=datetime.datetime(2025, 7, 21, 13, 37, 15, tzinfo=<UTC>), creation_host='netbox', register_to_eff=None))>
Sending GET request to https://step-ca.sfg.local:8443/acme/acme/directory.
Starting new HTTPS connection (1): step-ca.sfg.local:8443
https://step-ca.sfg.local:8443 "GET /acme/acme/directory HTTP/1.1" 200 347
Received response:
HTTP 200
Content-Type: application/json
Date: Wed, 24 Sep 2025 09:38:40 GMT
Content-Length: 347

{"newNonce":"https://step-ca.sfg.local:8443/acme/acme/new-nonce","newAccount":"https://step-ca.sfg.local:8443/acme/acme/new-account","newOrder":"https://step-ca.sfg.local:8443/acme/acme/new-order","revokeCert":"https://step-ca.sfg.local:8443/acme/acme/revoke-cert","keyChange":"https://step-ca.sfg.local:8443/acme/acme/key-change"}

Notifying user: Renewing an existing certificate for netbox.intern.sfgroup.de
Renewing an existing certificate for netbox.intern.sfgroup.de
Requesting fresh nonce
Sending HEAD request to https://step-ca.sfg.local:8443/acme/acme/new-nonce.
https://step-ca.sfg.local:8443 "HEAD /acme/acme/new-nonce HTTP/1.1" 200 0
Received response:
HTTP 200
Cache-Control: no-store
Link: <https://step-ca.sfg.local:8443/acme/acme/directory>;rel="index"
Replay-Nonce: U3lhaE5HcFhvSWNtOU9id0FmNmJtZ2JGcEI0MVEyVlo
Date: Wed, 24 Sep 2025 09:38:40 GMT


Storing nonce: U3lhaE5HcFhvSWNtOU9id0FmNmJtZ2JGcEI0MVEyVlo
JWS payload:
b'{\n  "identifiers": [\n    {\n      "type": "dns",\n      "value": "netbox.intern.sfgroup.de"\n    }\n  ]\n}'
Sending POST request to https://step-ca.sfg.local:8443/acme/acme/new-order:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vbC1zZmwtZGMxLnMuZmwubG9jYWw6ODQ0My9hY21lL2FjbWUvYWNjb3VudC9VQWRrVDFtZTI1b3JnUXl1Y21kRUhMMWRVOHJIbkx1SiIsICJub25jZSI6ICJVM2xoYUU1SGNGaHZTV050T1U5aWQwRm1ObUp0WjJKR2NFSTBNVkV5VmxvIiwgInVybCI6ICJodHRwczovL2wtc2ZsLWRjMS5zLmZsLmxvY2FsOjg0NDMvYWNtZS9hY21lL25ldy1vcmRlciJ9",
  "signature": "NQ37hgRipmMiF3pk5k_b1nHXF01C7lFjDWs-YRrrHyhix9QhWmKKi4KqLZJZMTUhuwePaYX2yaKKpdui9ixrGHULac4cPUhdmDyRMkAlfojBNyA0gtZx0V8GIDO5mhHW7hai4yAYL6EglQiLzF_JTzngF2Iev-MvIuc7arg152dRclzPs4QRDfCJh5ZnM8J-6YiYGKXH2BgV-Tl2zgMAj8gg7tNdZSssmUe4b0Poq33h80XeMIWiGNT749WYRckpcRygN0ZmO3lZCcYG1taiUgP1F7sZYxMT5nLx5r4QVsfF7Cd68yLGnEjqH6wR1ga9yaO85lpbgenqj3fDLteVNw",
  "payload": "ewogICJpZGVudGlmaWVycyI6IFsKICAgIHsKICAgICAgInR5cGUiOiAiZG5zIiwKICAgICAgInZhbHVlIjogIm5ldGJveC5pbnRlcm4uc2Zncm91cC5kZSIKICAgIH0KICBdCn0"
}
https://step-ca.sfg.local:8443 "POST /acme/acme/new-order HTTP/1.1" 201 438
Received response:
HTTP 201
Cache-Control: no-store
Content-Type: application/json
Link: <https://step-ca.sfg.local:8443/acme/acme/directory>;rel="index"
Location: https://step-ca.sfg.local:8443/acme/acme/order/rE4pE0p6cnNBJtG7Fqdiqq6TGxA8yKaw
Replay-Nonce: WThhYTdHZVNtTlFkUWhOZFRqSXVwWlVISDhmWmQ4RjE
Date: Wed, 24 Sep 2025 09:38:40 GMT
Content-Length: 438

{"id":"rE4pE0p6cnNBJtG7Fqdiqq6TGxA8yKaw","status":"pending","expires":"2025-09-25T09:38:40Z","identifiers":[{"type":"dns","value":"netbox.intern.sfgroup.de"}],"notBefore":"2025-09-24T09:37:40Z","notAfter":"2025-11-23T09:38:40Z","authorizations":["https://step-ca.sfg.local:8443/acme/acme/authz/C1HtjUnmIhE9gkKminskyJYhg9tU0N5W"],"finalize":"https://step-ca.sfg.local:8443/acme/acme/order/rE4pE0p6cnNBJtG7Fqdiqq6TGxA8yKaw/finalize"}

Storing nonce: WThhYTdHZVNtTlFkUWhOZFRqSXVwWlVISDhmWmQ4RjE
JWS payload:
b''
Sending POST request to https://step-ca.sfg.local:8443/acme/acme/authz/C1HtjUnmIhE9gkKminskyJYhg9tU0N5W:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vbC1zZmwtZGMxLnMuZmwubG9jYWw6ODQ0My9hY21lL2FjbWUvYWNjb3VudC9VQWRrVDFtZTI1b3JnUXl1Y21kRUhMMWRVOHJIbkx1SiIsICJub25jZSI6ICJXVGhoWVRkSFpWTnRUbEZrVVdoT1pGUnFTWFZ3V2xWSVNEaG1XbVE0UmpFIiwgInVybCI6ICJodHRwczovL2wtc2ZsLWRjMS5zLmZsLmxvY2FsOjg0NDMvYWNtZS9hY21lL2F1dGh6L0MxSHRqVW5tSWhFOWdrS21pbnNreUpZaGc5dFUwTjVXIn0",
  "signature": "Pqahf1EMD3TjtVLhklpU65-J8QjApLZ2PKosiJpb9YuWhvmCoD1VYk3p-NMfT2kODqZbcNOTdjufp8nov5CPkvas9aPrpvTv4Ve-duoBixkRQaI_dNlI6IuTb6ePfM8yu4x6PeDZr-lP0QiPdf3ORNxBlKUMVynVgQzDhtBZpPX1NIrtUMqLLBWKlxzgPv9v3fRSxrQ6WYbyqq7i67RbAJUX3USDhJz6d3PemFM9ioyU38yRyHoNIoGtIpyhFywx-fxhE6pNGoOa-d46x940mlxhKqII50reeM5oneFPCfZa1pVCZOVIY4yKRaT8pZOAT1zzl68sOlZeRs-WhsKJbg",
  "payload": ""
}
https://step-ca.sfg.local:8443 "POST /acme/acme/authz/C1HtjUnmIhE9gkKminskyJYhg9tU0N5W HTTP/1.1" 200 779
Received response:
HTTP 200
Cache-Control: no-store
Content-Type: application/json
Link: <https://step-ca.sfg.local:8443/acme/acme/directory>;rel="index"
Location: https://step-ca.sfg.local:8443/acme/acme/authz/C1HtjUnmIhE9gkKminskyJYhg9tU0N5W
Replay-Nonce: b2h5c0pxcmpVR1hjQVExSGVvbzdONm1Cb0prUjZ2QjM
Date: Wed, 24 Sep 2025 09:38:40 GMT
Content-Length: 779

{"identifier":{"type":"dns","value":"netbox.intern.sfgroup.de"},"status":"pending","challenges":[{"type":"dns-01","status":"pending","token":"6BU6f9eDTvhMmQB6Q5JWCBnRUeYTPeEn","url":"https://step-ca.sfg.local:8443/acme/acme/challenge/C1HtjUnmIhE9gkKminskyJYhg9tU0N5W/mpVAaHV9ZnRXYjBGT1rNxf3x46SNR52t"},{"type":"http-01","status":"pending","token":"6BU6f9eDTvhMmQB6Q5JWCBnRUeYTPeEn","url":"https://step-ca.sfg.local:8443/acme/acme/challenge/C1HtjUnmIhE9gkKminskyJYhg9tU0N5W/SigTnXfyJ0HCrpCKCldZU28XVW9fpXS2"},{"type":"tls-alpn-01","status":"pending","token":"6BU6f9eDTvhMmQB6Q5JWCBnRUeYTPeEn","url":"https://step-ca.sfg.local:8443/acme/acme/challenge/C1HtjUnmIhE9gkKminskyJYhg9tU0N5W/sk9TDDFQ8habX1Bzh62MtYrWO7MWJ2KG"}],"wildcard":false,"expires":"2025-09-25T09:38:40Z"}

Storing nonce: b2h5c0pxcmpVR1hjQVExSGVvbzdONm1Cb0prUjZ2QjM
Performing the following challenges:
http-01 challenge for netbox.intern.sfgroup.de
Generated server block:
[]
Creating backup of /etc/letsencrypt/options-ssl-nginx.conf
Creating backup of /etc/nginx/mime.types
Creating backup of /etc/nginx/nginx.conf
Creating backup of /etc/nginx/sites-enabled/netbox
Writing nginx conf tree to /etc/nginx/nginx.conf:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
error_log /var/log/nginx/error.log;
include /etc/nginx/modules-enabled/*.conf;

events {
        worker_connections 768;
        # multi_accept on;
}

http {
server_names_hash_bucket_size 128;
include /etc/letsencrypt/le_http_01_cert_challenge.conf;

        ##
        # Basic Settings
        ##

        sendfile on;
        tcp_nopush on;
        types_hash_max_size 2048;
        # server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/access.log;

        ##
        # Gzip Settings
        ##

        gzip on;

        # gzip_vary on;
        # gzip_proxied any;
        # gzip_comp_level 6;
        # gzip_buffers 16 8k;
        # gzip_http_version 1.1;
        # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}


#mail {
#       # See sample authentication script at:
#       # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
#
#       # auth_http localhost/auth.php;
#       # pop3_capabilities "TOP" "USER";
#       # imap_capabilities "IMAP4rev1" "UIDPLUS";
#
#       server {
#               listen     localhost:110;
#               protocol   pop3;
#               proxy      on;
#       }
#
#       server {
#               listen     localhost:143;
#               protocol   imap;
#               proxy      on;
#       }
#}

Writing nginx conf tree to /etc/nginx/sites-enabled/netbox:
server {rewrite ^(/.well-known/acme-challenge/.*) $1 break; # managed by Certbot


    if ($host = netbox.intern.sfgroup.de) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


        listen 80 ;
        listen [::]:80 ;
    server_name netbox.intern.sfgroup.de;
    return 404; # managed by Certbot
location = /.well-known/acme-challenge/6BU6f9eDTvhMmQB6Q5JWCBnRUeYTPeEn{default_type text/plain;return 200 6BU6f9eDTvhMmQB6Q5JWCBnRUeYTPeEn.H49FA-f3uyvsrzU87SCr2AbanO_b5WIjvkkCQDOQIn0;} # managed by Certbot

}


server {rewrite ^(/.well-known/acme-challenge/.*) $1 break; # managed by Certbot


    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot

    server_name netbox.intern.sfgroup.de;

    ssl_certificate /etc/letsencrypt/live/netbox.intern.sfgroup.de/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/netbox.intern.sfgroup.de/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

    client_max_body_size 25m;

    location /static/ {
        alias /opt/netbox/netbox/static/;
    }

    location /.test/ {default_type text/plain;return 200 uC2Rb2Y1oHXAeiuKT8AWYwHitmgaPo5R.H49FA-f3uyvsrzU87SCr2AbanO_b5WIjvkkCQDOQIn0;} #>

    location / {
        # Remove these lines if using uWSGI instead of Gunicorn
        proxy_pass http://127.0.0.1:8001;
        proxy_set_header X-Forwarded-Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Uncomment these lines if using uWSGI instead of Gunicorn
        # include uwsgi_params;
        # uwsgi_pass  127.0.0.1:8001;
        # uwsgi_param Host $host;
        # uwsgi_param X-Real-IP $remote_addr;
        # uwsgi_param X-Forwarded-For $proxy_add_x_forwarded_for;
        # uwsgi_param X-Forwarded-Proto $http_x_forwarded_proto;

    }
location = /.well-known/acme-challenge/6BU6f9eDTvhMmQB6Q5JWCBnRUeYTPeEn{default_type text/plain;return 200 6BU6f9eDTvhMmQB6Q5JWCBnRUeYTPeEn.H49FA-f3uyvsrzU87SCr2AbanO_b5WIjvkkCQDOQIn0;} # managed by Certbot

}


Notifying user: Challenges loaded. Press continue to submit to CA.

The following URLs should be accessible from the internet and return the value
mentioned:

URL:
http://netbox.intern.sfgroup.de/.well-known/acme-challenge/6BU6f9eDTvhMmQB6Q5JWCBnRUeYTPeEn
Expected value:
6BU6f9eDTvhMmQB6Q5JWCBnRUeYTPeEn.H49FA-f3uyvsrzU87SCr2AbanO_b5WIjvkkCQDOQIn0

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Challenges loaded. Press continue to submit to CA.

The following URLs should be accessible from the internet and return the value
mentioned:

URL:
http://netbox.intern.sfgroup.de/.well-known/acme-challenge/6BU6f9eDTvhMmQB6Q5JWCBnRUeYTPeEn
Expected value:
6BU6f9eDTvhMmQB6Q5JWCBnRUeYTPeEn.H49FA-f3uyvsrzU87SCr2AbanO_b5WIjvkkCQDOQIn0
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
JWS payload:
b'{}'
Sending POST request to https://step-ca.sfg.local:8443/acme/acme/challenge/C1HtjUnmIhE9gkKminskyJYhg9tU0N5W/SigTnXfyJ0HCrpCKCldZU28XVW9fpXS2:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vbC1zZmwtZGMxLnMuZmwubG9jYWw6ODQ0My9hY21lL2FjbWUvYWNjb3VudC9VQWRrVDFtZTI1b3JnUXl1Y21kRUhMMWRVOHJIbkx1SiIsICJub25jZSI6ICJiMmg1YzBweGNtcFZSMWhqUVZFeFNHVnZiemRPTm0xQ2IwcHJValoyUWpNIiwgInVybCI6ICJodHRwczovL2wtc2ZsLWRjMS5zLmZsLmxvY2FsOjg0NDMvYWNtZS9hY21lL2NoYWxsZW5nZS9DMUh0alVubUloRTlna0ttaW5za3lKWWhnOXRVME41Vy9TaWdUblhmeUowSENycENLQ2xkWlUyOFhWVzlmcFhTMiJ9",
  "signature": "Qp_2LK8YzrfTi-RirrxkApkVv_Y7mH5H3YdYsif2MKFZI-tSXSmNy_BlU_dMpLJ_WvEIfUQVMosZ6RIxgxBd-qwNqNB236TJMAa2VPTn4Ponq3VZ9EeCxdMBjyjgiwQEL6ZDnvM90sCcm52P9HK2RHbfDoKSZ91H3-1JmgC_TuaNO1uOfcnMS9QT9iFmwR-7KMDUuMCLBIOXWQAyG0i76NYUiED2RG74quJd3Y2C96Wq7rYjBj1on5i8AlubI3rhOvcqx6mhO8gp7P5oLs-nB5u7b14IGvbw_A7roK-_nmvsxlTuSD9VBhLpUbzMfosj8MKoQXd7XiRl0Vq9qcBQ8A",
  "payload": "e30"
}
https://step-ca.sfg.local:8443 "POST /acme/acme/challenge/C1HtjUnmIhE9gkKminskyJYhg9tU0N5W/SigTnXfyJ0HCrpCKCldZU28XVW9fpXS2 HTTP/1.1" 200 327
Received response:
HTTP 200
Cache-Control: no-store
Content-Type: application/json
Link: <https://step-ca.sfg.local:8443/acme/acme/directory>;rel="index", <https://step-ca.sfg.local:8443/acme/acme/authz/C1HtjUnmIhE9gkKminskyJYhg9tU0N5W>;rel="up"
Location: https://step-ca.sfg.local:8443/acme/acme/challenge/C1HtjUnmIhE9gkKminskyJYhg9tU0N5W/SigTnXfyJ0HCrpCKCldZU28XVW9fpXS2
Replay-Nonce: MWEwNkh6WGZYUDY2bnZLdjVCR29lcm1QVnBPdnFCMmg
Date: Wed, 24 Sep 2025 09:38:41 GMT
Content-Length: 327

{"type":"http-01","status":"pending","token":"6BU6f9eDTvhMmQB6Q5JWCBnRUeYTPeEn","url":"https://step-ca.sfg.local:8443/acme/acme/challenge/C1HtjUnmIhE9gkKminskyJYhg9tU0N5W/SigTnXfyJ0HCrpCKCldZU28XVW9fpXS2","error":{"type":"urn:ietf:params:acme:error:connection","detail":"The server could not connect to validation target"}}

Storing nonce: MWEwNkh6WGZYUDY2bnZLdjVCR29lcm1QVnBPdnFCMmg
Waiting for verification...

###########################
# many more tries cut out #
###########################

Encountered exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 216, in _poll_authorizations
    raise errors.AuthorizationError('All authorizations were not finalized by the CA.')
certbot.errors.AuthorizationError: All authorizations were not finalized by the CA.

Calling registered functions
Cleaning up challenges
Failed to renew certificate netbox.intern.sfgroup.de with error: All authorizations were not finalized by the CA.
Traceback was:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/_internal/renewal.py", line 540, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line 1550, in renew_cert
    renewed_lineage = _get_and_save_cert(le_client, config, lineage=lineage)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line 131, in _get_and_save_cert
    renewal.renew_cert(config, domains, le_client, lineage)
  File "/usr/lib/python3/dist-packages/certbot/_internal/renewal.py", line 399, in renew_cert
    new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/client.py", line 428, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/client.py", line 496, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 216, in _poll_authorizations
    raise errors.AuthorizationError('All authorizations were not finalized by the CA.')
certbot.errors.AuthorizationError: All authorizations were not finalized by the CA.

Notifying user:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All renewals failed. The following certificates could not be renewed:
  /etc/letsencrypt/live/netbox.intern.sfgroup.de/fullchain.pem (failure)
Notifying user: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 33, in <module>
    sys.exit(load_entry_point('certbot==2.9.0', 'console_scripts', 'certbot')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line 1894, in main
    return config.func(config, plugins)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line 1642, in renew
    renewed_domains, failed_domains = renewal.handle_renewal_request(config)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/renewal.py", line 568, in handle_renewal_request
    raise errors.Error(
certbot.errors.Error: 1 renew failure(s), 0 parse failure(s)
1 renew failure(s), 0 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is: nginx/1.24.0 (Ubuntu)

The operating system my web server runs on is: Ubuntu 24.04.3 LTS

My hosting provider, if applicable, is: ---

I can login to a root shell on my machine: yes

I'm using a control panel to manage my site: no

The version of my client is: certbot 2.9.0

Additional notes:

  • This resource is not publically available, and I'm using step-ca as an internal custom ca.
  • I have checked that the website in question is reachable by the ca server.
  • When I visit the listed challenge URL in a browser (with certficate warnings ignored) while the challenge is active, I reach a 404 error page of the netbox webserver.
  • I think that I initially created the certificate with the wrong nginx default vhost, since the auto-created #managed by certbot comments were all in this vhost file. I have tried to transfer all these lines to the netbox vhost as best as I could, but I must have gone wrong somewhere.
2

I'll start by saying we don't often work on Certbot problems with step-ca. We focus on helping people get and use Let's Encrypt certs. If you don't get many responses that is why. You might try the Certbot github or even a step-ca forum for other help.

I have never used step-ca but am very familiar with ACME and nginx.

Hmm. The error in the log says:

"error":{"type":"urn:ietf:params:acme:error:connection","detail":"The server could not connect to validation target"}}

If google is believed that means that step-ca could not even reach your nginx.

This is puzzling. step-ca says it can't reach it but from a different machine you can reach it but don't get the expected response from the nginx server block that Cerbot updated.

The first thing I would do is simplify the setup. Instead of using --nginx option use certonly --webroot -w (path) instead. You'll need to modify your port 80 server block a bit.

Change JUST the port 80 server block (in /etc/nginx/sites-enabled/netbox) to:

server {
  listen 80;
  listen [::]:80;
  server_name netbox.intern.sfgroup.de;

  location /.well-known/acme-challenge/ {
    root (path);   # as you prefer, must match Certbot -w (path)
  }
  location / {
    return 301 https://$host$request_uri;
  }
}

You can then place any file in (path) and test it can be reached from step-ca (or other internal device). Using certonly --webroot removes the --nginx plugin from the mix and allows easier debug.

If you place a file named TestFile in (path) this should see it from other devices

curl -i http://netbox.intern.sfgroup.de/.well-known/acme-challenge/TestFile
2 Likes
3

One quick thought -- by default step-ca will use the host's default resolver, but you can pass a --resolver parameter on the command line to make it use a different one. Could it actually be talking to a different resolver to the default -- one that isn't able to resolve netbox.intern.sfgroup.de?

Note that --resolver takes an IP address and a port -- not just an IP address.

4

That is a better question for experts at the step-ca forum.

Maybe this one: smallstep/certificates · Discussions · GitHub

3 Likes
5

Thanks a bunch for the help!
Sadly, I must report that the error resolved itself - and I don't exactly know why or how :frowning:

@MikeMcQ : I followed your suggestion and simplified the vhost-config for port 80. After some fiddling (mostly me failing to grasp how webroots and resulting paths work) I had to pause my efforts since other stuff came up. Today - some days later - I resumed troubleshooting, found out what I did wrong (forgot to create the subfolders under the webroot), and it worked as you predicted:

curl -i http://netbox.intern.sfgroup.de/.well-known/acme-challenge/TestFile 
HTTP/1.1 200 OK
Server: nginx/1.24.0 (Ubuntu)
Date: Tue, 30 Sep 2025 09:10:37 GMT
Content-Type: application/octet-stream
Content-Length: 16
Last-Modified: Sun, 28 Sep 2025 09:01:51 GMT
Connection: keep-alive
ETag: "68d8f97f-10"
Accept-Ranges: bytes

This is a test.

I subsequently did the suggested dry run:

certbot certonly --webroot --server https://step-ca.sfg.local:8443/acme/acme/directory --dry-run -m administration@sfgroup.de --agree-tos --non-interactive -w /var/www/html --domain netbox.intern.sfgroup.de 
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Simulating renewal of an existing certificate for netbox.intern.sfgroup.de
The dry run was successful.

Since that worked so well, I just gave this a go:

certbot renew 
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/netbox.intern.sfgroup.de.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Certificate not yet due for renewal

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
The following certificates are not due for renewal yet:
  /etc/letsencrypt/live/netbox.intern.sfgroup.de/fullchain.pem expires on 2025-11-27 (skipped)
No renewals were attempted.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

...so the renewal was suddenly already successfully done, presumably by the cron job. I just checked to be sure:

certbot renew --force-renewal 
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/netbox.intern.sfgroup.de.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Renewing an existing certificate for netbox.intern.sfgroup.de
Reloading nginx server after certificate renewal

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all renewals succeeded:
  /etc/letsencrypt/live/netbox.intern.sfgroup.de/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

FTR, here the current vhost config:

vhost: netbox 
server {
  listen 80;
  listen [::]:80;
  server_name netbox.intern.sfgroup.de;

  location / {
    return 301 https://$host$request_uri;
  }
}

server {
    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot

    server_name netbox.intern.sfgroup.de;

    ssl_certificate /etc/letsencrypt/live/netbox.intern.sfgroup.de/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/netbox.intern.sfgroup.de/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

    client_max_body_size 25m;

    location /static/ {
        alias /opt/netbox/netbox/static/;
    }

    location / {
        proxy_pass http://127.0.0.1:8001;
        proxy_set_header X-Forwarded-Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

@met24 : that's a good point, but I don't think that this was the issue here - we don't have any custom resolver config for step-ca, the same workflow of creating DNS records and step-ca using them worked without problems previously, and we didn't change anything on step-ca's or DNS' side.

All in all: :man_shrugging:
Maybe the vhost's port 80 config got better through the fiddling, maybe something else broke and fixed itself - I can't tell for sure. And for what it's worth, I'm pretty certain I reloaded the vhost config after any change properly.

Still, thanks for taking the time to help out!

2 Likes
6

Glad you got it working. A couple things though ...

The main concern is the server block as you have it now. The port 80 server block redirects all traffic to port 443 which in turn uses proxy for most everything. That is a complex flow for handling an HTTP Challenge using --webroot. You'll note I had a location block in my example to handle the challenge in port 80 server block.

That said, if your Certbot renewal profile is using --nginx the config you have is fine. Certbot will add temp settings to your port 80 server block (and port 443) to handle the challenge and remove those when done.

If you post your renewal config we can confirm what is happening. Redact your account number if you wish.
/etc/letsencrypt/renewal/netbox.intern.sfgroup.de.conf
If it is using --webroot you'll want to adjust your port 80 server to avoid the extra traffic flows. This is easier to debug and will be the most reliable. Is also a better template for future setups.

I don't recall suggesting --dry-run and care must be taken when used with --server. The --dry-run option itself sets the Let's Encrypt Staging server and does not save the resulting cert. I believe if you specify --server after --dry-run it overrides the LE staging and still does not save the cert. In your case you listed --server before so I would double-check from the Certbot logs that it actually did the test against step-ca.

Same goes for this. The best way to test with renew is with --dry-run. But, be sure to add --server after the --dry-run to use your CA.

Doing --force is generally discouraged although with your own CA that may be fine. LE has rate limits and --force often leads to people getting blocked by those.

3 Likes
7

That said, if your Certbot renewal profile is using --nginx the config you have is fine. Certbot will add temp settings to your port 80 server block (and port 443) to handle the challenge and remove those when done.

Yep, the renewal profile uses --nginx:

/etc/letsencrypt/renewal/netbox.intern.sfgroup.de.conf 
version = 2.9.0
archive_dir = /etc/letsencrypt/archive/netbox.intern.sfgroup.de
cert = /etc/letsencrypt/live/netbox.intern.sfgroup.de/cert.pem
privkey = /etc/letsencrypt/live/netbox.intern.sfgroup.de/privkey.pem
chain = /etc/letsencrypt/live/netbox.intern.sfgroup.de/chain.pem
fullchain = /etc/letsencrypt/live/netbox.intern.sfgroup.de/fullchain.pem

[renewalparams]
account = $REDACTED
server = https://step-ca.sfg.local:8443/acme/acme/directory
authenticator = nginx
installer = nginx
key_type = ecdsa

It was very weird, since the certbot logs (and manual checking of the netbox vhost config with --debug-challenges active) also showed that the temporary location blocks were being added and cleaned up correctly.

My only other hunch was that certbot may have confused the vhost configs in some way, since the initial certificate was probably generated with the default vhost config (commented lines removed):

vhost 'default' 
server {
        listen 80 default_server;
        listen [::]:80 default_server;

        root /var/www/html;

        index index.html index.htm index.nginx-debian.html;

        server_name _;

        location / {
                try_files $uri $uri/ =404;
        }
}

server {
        root /var/www/html;

        index index.html index.htm index.nginx-debian.html;
    server_name netbox.intern.sfgroup.de; # managed by Certbot


        location / {
                try_files $uri $uri/ =404;
        }

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/netbox.intern.sfgroup.de/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/netbox.intern.sfgroup.de/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    if ($host = netbox.intern.sfgroup.de) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


        listen 80 ;
        listen [::]:80 ;
    server_name netbox.intern.sfgroup.de;
    return 404; # managed by Certbot
}

(On second look, it's also weird how there are two blocks for port 80 in there. I haven't manually touched this one - just disabled the vhost as soon as I noticed that it was still active.)

But how that would be the case: no idea. As I said, the renewal tries actually used the correct vhost config. It's just the only thing that comes to mind that happened in error and seemed suspicious to me.

I don't recall suggesting --dry-run and care must be taken when used with --server. The --dry-run option itself sets the Let's Encrypt Staging server and does not save the resulting cert. I believe if you specify --server after --dry-run it overrides the LE staging and still does not save the cert. In your case you listed --server before so I would double-check from the Certbot logs that it actually did the test against step-ca.

Sorry for the misquote :grimacing: and thanks for the advice. But even though the --server flag was placed before --dry-run, it worked for me and correctly used my custom server. Since these are internal servers whose DNS records are not public, the challenges wouldn't have worked anyway if it fell back to LE's public servers.

1 Like
8

The two port 80 blocks are the default with a fake server_name and the other the actual one with the domain as server_name. That doesn't seem especially odd.

Or, do you mean this default vhost duplicates the one you showed in your post prior? Because duplicates should be reported by nginx. It's usually a WARN level error in the log. Or, also check with sudo nginx -t or sudo systemctl status nginx

As for the odd startup issue a few things come to mind. The first is that a problem can result if nginx was not running when you issued the Certbot command using --nginx . In that case Certbot starts nginx but not using systemd which can cause poor behavior and inability to control nginx normally.

The other issues relate to especially large nginx configs. Do you have a large number of server blocks? Or extensions loaded that might cause a very slow nginx reload? If so I can explain more about those concerns.

3 Likes
9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.