/var/www/certbot/.well-known/acme-challenge/" failed (2: No such file or directory)

Help
1

Hi there,

I am currently using a docker-compose deployed nginx container, which uses mtls, and letsencrypt for the server certificate. Let's Encrypt worked for the initial certificate, but it seems the renewal does not work. I was following this guide SSL with Docker Swarm, Let's Encrypt and Nginx - Finnian Anderson

My initial build command which worked and acquired the certificate:

docker run --rm \
  -p 443:443 -p 80:80 --name letsencrypt \
  -v "/media/volumes/nginx-certificates:/etc/letsencrypt" \
  -v "/media/volumes/nginx-certificates:/var/lib/letsencrypt" \
  certbot/certbot certonly -n \
  -m "<redacted-email>" \
  -d <redacted-webserver-domain> \
  --standalone --agree-tos

and my renewal command:

  docker run --rm --name letsencrypt \
    -v "/media/volumes/nginx-certificates:/etc/letsencrypt" \
    -v "/media/volumes/nginx-certificates:/var/lib/letsencrypt" \
    -v "/media/volumes/nginx-certificates:/var/www/certbot" \
    certbot/certbot:latest \
    certonly --webroot -w /var/www/certbot \
    renew --quiet

which results in the following error:

[error] 37#37: *69565465 open() "/var/www/certbot/.well-known/acme-challenge/sp590OucdzdWmuOvDH0Szd6QVUdA4G0s0kfZB658EyE" failed (2: No such file or directory), client: <redacted>, server: <redacted>, request: "GET /.well-known/acme-challenge/sp590OucdzdWmuOvDH0Szd6QVUdA4G0s0kfZB658EyE HTTP/1.1", host: "<redacted>"

My nginx configuration file is as follows:

server {
    listen 80;
    server_name <webserver.domain>;
    server_tokens off;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl http2;
	server_name <webserver.domain>;
    server_tokens off;

	proxy_intercept_errors on;
	error_page 401 495 496 @json401error;

    ssl_certificate /etc/letsencrypt/live/<webserver.domain>/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/<webserver.domain>/privkey.pem;
	ssl_protocols TLSv1.2;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    # mTLS handler
	# This will return a 401 to all clients without a proper certificate
    if ($ssl_client_verify != SUCCESS) { return 401; }
	
	# This tells Nginx what CA to verify against
    ssl_client_certificate /certificates/cacert.pem;
    ssl_verify_depth 2;
	
	# This tells Nginx to verify clients
    ssl_verify_client optional;
	
    location @json401error {
        default_type application/json;
		add_header 'Access-Control-Allow-Origin' '*' always;
		return 401 '{"errors":["401 Unauthorized"]}';
	}

    location / {
		proxy_pass http://<redacted>;
		#proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header SSL_Client $ssl_client_s_dn;
        proxy_set_header SSL_Client_Verify $ssl_client_verify;
		proxy_ssl_verify off;
		#proxy_set_header X-Forwarded-Proto $scheme;
    }
}

EDIT: Yaml Below:

version: '3.8'
services:
  nginx:
    image: nginx:1.23.1
    restart: unless-stopped
    logging:
      driver: "json-file"
      options:
        max-size: "20m"
        max-file: "10"
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints:
          - node.labels.Z3==1
        preferences:
          - spread: node.labels.Z3
    ports:
    - '80:80'
    - '443:443'
    volumes:
      #- /opt/wott/certs:/opt/wott/certs:ro
      - nginx-templates:/etc/nginx/templates
      - nginx-clientca:/certificates
      - nginx-certificates:/etc/letsencrypt
      - nginx-certbot:/var/www/certbot
    networks:
    # Use the public network created to be shared between Traefik and
    # any other service that needs to be publicly available with HTTPS
    - api-entry


volumes:
  # Create a volume to store the certificates, there is a constraint to make sure
  # nginx is always deployed to the same Docker node with the same volume containing
  # the HTTPS certificates
  nginx-certificates:
    name: nginx-certificates
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /media/volumes/nginx-certificates
  nginx-clientca:
    name: nginx-clientca
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /media/volumes/nginx-clientca
  nginx-certbot:
    name: nginx-certbot
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /media/volumes/nginx-certbot
  nginx-templates:
    name: nginx-templates
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /media/volumes/nginx-templates
networks:
  api-entry:
    external: true

I've confirmed that the folders have appropriate permissions from the nginx container perspective, and that the file path does exist, but I have a feeling it's pointing somewhere else, and can't quite determine how to resolve it or, there is something not quite right with the acme location in nginx.

Any assistance is greatly appreciated.

Thanks,
bm0

1 Like
2

Hi @bm0, and welcome to the LE community forum :slight_smile:

I think that might need to be:
/var/log/letsencrypt

Do you have any certbot logs?

5 Likes
3

I'm not that familiar with Docker, but are you mounting the same volume at three different locations? I'm pretty sure that's not how it's supposed to be, right?

6 Likes
4

Yes that's right, I've cleaned this up into separate directories now so the command is like this:

  docker run --rm --name letsencrypt \
    -v "/media/volumes/nginx-certificates:/etc/letsencrypt" \
    -v "/media/volumes/nginx-certbot:/var/log/letsencrypt" \
    certbot/certbot:latest \
    renew --quiet
5

Thanks @rg305, much appreciated.

Okay thanks, I've added the logs in, run the renew command below:

  docker run --rm --name letsencrypt \
    -v "/media/volumes/nginx-certificates:/etc/letsencrypt" \
    -v "/media/volumes/nginx-certbot:/var/log/letsencrypt" \
    certbot/certbot:latest \
    renew --quiet

and received the attached log.

2022-12-07 10:40:09,285:DEBUG:acme.client:Storing nonce: 371C0PURQf-qF7AJckhmjjf6e7RxChQVZiZHtmJ8QexMd2Y
2022-12-07 10:40:09,285:INFO:certbot._internal.auth_handler:Performing the following challenges:
2022-12-07 10:40:09,285:INFO:certbot._internal.auth_handler:http-01 challenge for <redacted.webserver.domain>
2022-12-07 10:40:09,305:DEBUG:acme.standalone:Successfully bound to :80 using IPv6
2022-12-07 10:40:09,305:DEBUG:acme.standalone:Certbot wasn't able to bind to :80 using IPv4, this is often expected due to the dual stack nature of IPv6 socket implementations.
2022-12-07 10:40:09,307:DEBUG:acme.client:JWS payload:
b'{}'
2022-12-07 10:40:09,308:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/chall-v3/184971178057/fDCKPA:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvNzQwMjg0MDE3IiwgIm5vbmNlIjogIjM3MUMwUFVSUWYtcUY3QUpja2htampmNmU3UnhDaFFWWmlaSHRtSjhRZXhNZDJZIiwgInVybCI6ICJodHRwczovL2FjbWUtdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9jaGFsbC12My8xODQ5NzExNzgwNTcvZkRDS1BBIn0",
  "signature": "bqt4fh2MjPzDga47dmkVrBlix20nZftju4pslFeQgbUU8IjL5G-YJpsl8NgmW15vJfG382Xi_EEP5_ShhdhM9-NqZuA8IBQHpKtRIxHHdPhYCiAlPqEKgOJDu-VQiweAEo0vhMXoQVQpsGbRoeF2AoJXOrHXcQsiDQWU2ZfzErmMLnVCJmIaKi0a18QUOrLyx1AVwO2ifID_HGFxTpcHvxveO3ym9ng7ba6GF3GrVk3hMuDBbkNzzh8Tc8aai7cGqN9gcasd-zsvH2qskXvPQt6z9lM0IR97tu3iL9zOp7770UJ_brP_zrX9eG5CGGBaUOtgBhtX_J-K2htYQjQHGA",
  "payload": "e30"
}
2022-12-07 10:40:09,489:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/chall-v3/184971178057/fDCKPA HTTP/1.1" 200 187
2022-12-07 10:40:09,490:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Wed, 07 Dec 2022 10:40:09 GMT
Content-Type: application/json
Content-Length: 187
Connection: keep-alive
Boulder-Requester: 740284017
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index", <https://acme-v02.api.letsencrypt.org/acme/authz-v3/184971178057>;rel="up"
Location: https://acme-v02.api.letsencrypt.org/acme/chall-v3/184971178057/fDCKPA
Replay-Nonce: C878HtWWDuLRNyGTLOz4e8xPtHy6CovVEoxIWE5L3Tv5W2k
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "type": "http-01",
  "status": "pending",
  "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/184971178057/fDCKPA",
  "token": "HcdnnXfPZE34HmzL22X8RMLowQIlJJBYhbV4GNm5SY4"
}
2022-12-07 10:40:09,490:DEBUG:acme.client:Storing nonce: C878HtWWDuLRNyGTLOz4e8xPtHy6CovVEoxIWE5L3Tv5W2k
2022-12-07 10:40:09,491:INFO:certbot._internal.auth_handler:Waiting for verification...
2022-12-07 10:40:10,492:DEBUG:acme.client:JWS payload:
b''
2022-12-07 10:40:10,494:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/authz-v3/184971178057:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvNzQwMjg0MDE3IiwgIm5vbmNlIjogIkM4NzhIdFdXRHVMUk55R1RMT3o0ZTh4UHRIeTZDb3ZWRW94SVdFNUwzVHY1VzJrIiwgInVybCI6ICJodHRwczovL2FjbWUtdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hdXRoei12My8xODQ5NzExNzgwNTcifQ",
  "signature": "hxX_0GvuCmOY5qBPnOIzhh9EOLCFa_dFqjgwZx27KE-TQvDkf4HgK2y0IjDqA79q0zMcCpBhgudAPeJvQzpcePj-dVuLMRCZLMxc8g28njCsHgiO61bWPKeUJ9BGoy2GaWZTus2WEkQWUbGvmUdZQZJBEXLJ3ArO_W8wYA3EXUCfgxYA6k_faek_gtQdDa0vWYIDddcDr8-_v2HOKJYM_TfIM4VYFRBCdtGdNwG-G3D_L9F9e59DDEJXlStBQLrcXCVmJXLneizi4eLR8vPvHgxMTR4-Uo_-F0vfQbwV6kth8L698mrzYUVX8lBLwlAvM0Yem3er76Ku-YgRjVCAKw",
  "payload": ""
}
2022-12-07 10:40:10,672:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/184971178057 HTTP/1.1" 200 1093
2022-12-07 10:40:10,673:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Wed, 07 Dec 2022 10:40:10 GMT
Content-Type: application/json
Content-Length: 1093
Connection: keep-alive
Boulder-Requester: 740284017
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 371CrXnLzEdHKfPBJ8KnGVUiFuWKqPrAo_GwCf-TpZgYXWU
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "<redacted.webserver.domain>"
  },
  "status": "invalid",
  "expires": "2022-12-14T10:40:08Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:unauthorized",
        "detail": "<redacted.webserver.ipaddress>: Invalid response from http://<redacted.webserver.domain>/.well-known/acme-challenge/HcdnnXfPZE34HmzL22X8RMLowQIlJJBYhbV4GNm5SY4: 404",
        "status": 403
      },
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/184971178057/fDCKPA",
      "token": "HcdnnXfPZE34HmzL22X8RMLowQIlJJBYhbV4GNm5SY4",
      "validationRecord": [
        {
          "url": "http://<redacted.webserver.domain>/.well-known/acme-challenge/HcdnnXfPZE34HmzL22X8RMLowQIlJJBYhbV4GNm5SY4",
          "hostname": "<redacted.webserver.domain>",
          "port": "80",
          "addressesResolved": [
            "<redacted.webserver.ipaddress>"
          ],
          "addressUsed": "<redacted.webserver.ipaddress>"
        }
      ],
      "validated": "2022-12-07T10:40:09Z"
    }
  ]
}
2022-12-07 10:40:10,673:DEBUG:acme.client:Storing nonce: 371CrXnLzEdHKfPBJ8KnGVUiFuWKqPrAo_GwCf-TpZgYXWU
2022-12-07 10:40:10,673:INFO:certbot._internal.auth_handler:Challenge failed for domain <redacted.webserver.domain>
2022-12-07 10:40:10,673:INFO:certbot._internal.auth_handler:http-01 challenge for <redacted.webserver.domain>
2022-12-07 10:40:10,673:DEBUG:certbot._internal.display.obj:Notifying user: 
Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems:
  Domain: <redacted.webserver.domain>
  Type:   unauthorized
  Detail: <redacted.webserver.ipaddress>: Invalid response from http://<redacted.webserver.domain>/.well-known/acme-challenge/HcdnnXfPZE34HmzL22X8RMLowQIlJJBYhbV4GNm5SY4: 404

Hint: The Certificate Authority failed to download the challenge files from the temporary standalone webserver started by Certbot on port 80. Ensure that the listed domains point to this machine and that it can accept inbound connections from the internet.

2022-12-07 10:40:10,674:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/opt/certbot/src/certbot/certbot/_internal/auth_handler.py", line 106, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/opt/certbot/src/certbot/certbot/_internal/auth_handler.py", line 206, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2022-12-07 10:40:10,674:DEBUG:certbot._internal.error_handler:Calling registered functions
2022-12-07 10:40:10,674:INFO:certbot._internal.auth_handler:Cleaning up challenges
2022-12-07 10:40:10,674:DEBUG:certbot._internal.plugins.standalone:Stopping server at :::80...
2022-12-07 10:40:10,811:ERROR:certbot._internal.renewal:Failed to renew certificate <redacted.webserver.domain> with error: Some challenges have failed.

It's domain definitely is pointing to the correct machine, so I'm a bit unsure what is causing this.

1 Like
6

Can you also show the nginx docker file?

5 Likes
7

no worries @Osiris, I've updated the original post with the nginx yaml

8

Sorry, didn't notice it before, but your updated Certbot command doesn't include /var/www/certbot. Where did it go?

5 Likes
9

Something is not lined up right.
We should have a look at the nginx config, with:
nginx -T

6 Likes
10

Okay @rg305 I've made some very minor process. So when I follow that link to the location, it returns a 404 not found:

image
image1147×185 9.82 KB

I've checked the acme folder location and it appears that no acme-challenge was created.

Similar to your recommendation @rg305 in this arctile here: .well-known/acme-challenge path access 404 - Help - Let's Encrypt Community Support (letsencrypt.org)

I created a test acme-challenge file in that location, and it worked. So it's not actually and access issue to the url, but it's the fact that a acme-challenge file is not being created for it to reference...

also please see nginx -T below - apologies I did have to slightly sanitise the FQDN in these.

server {
    listen 80;
    server_name testing.web.private.company.net;
    server_tokens off;
        autoindex_localtime on;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl http2;
        server_name testing.web.private.company.net;
    server_tokens off;

        proxy_intercept_errors on;
        error_page 401 495 496 @json401error;

    ssl_certificate /etc/letsencrypt/live/testing.web.private.company.net/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/testing.web.private.company.net/privkey.pem;
    ssl_protocols TLSv1.2;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    # mTLS handler
    # This will return a 401 to all clients without a proper certificate
    if ($ssl_client_verify != SUCCESS) { return 401; }

    # This tells Nginx what CA to verify against
    ssl_client_certificate /certificates/cacert.pem;
    ssl_verify_depth 2;

    # This tells Nginx to verify clients
    ssl_verify_client optional;

    location @json401error {
        default_type application/json;
                add_header 'Access-Control-Allow-Origin' '*' always;
                return 401 '{"errors":["401 Unauthorized"]}';
        }

    location / {
        proxy_pass http://tea-leaves-webapi;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header SSL_Client $ssl_client_s_dn;
        proxy_set_header SSL_Client_Verify $ssl_client_verify;
        proxy_ssl_verify off;

    }
}


# configuration file /etc/letsencrypt/options-ssl-nginx.conf:
# This file contains important security parameters. If you modify this file
# manually, Certbot will be unable to automatically provide future security
# updates. Instead, Certbot will print and log an error message with a path to
# the up-to-date file that you will need to refer to when manually updating
# this file. Contents are based on https://ssl-config.mozilla.org

ssl_session_cache shared:le_nginx_SSL:10m;
ssl_session_timeout 1440m;
ssl_session_tickets off;

ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers off;

ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";
11

my apologies @Osiris I somehow sent through the wrong copy paste (working a bit late on this one):

  docker run --rm --name letsencrypt \
    -v "/media/volumes/nginx-certificates:/etc/letsencrypt" \
    -v "/media/volumes/nginx-certbot:/var/log/letsencrypt" \
    -v "/media/volumes/nginx-certbot:/var/www/certbot" \
    certbot/certbot:latest \
    renew --quiet

Please note, I am indeed using two of the same there, but one is just for logs.

1 Like
12

Hi @rg305 and @Osiris I was wondering if anyone might have had some thoughts on the above? it seems to be the acme file is not being created.

13

The logs are not conclusive.
You may need to rerun them with the added:
-v
OR
-vv
OR
-vvv

[One of those should show us what is going on]

6 Likes
15

And as an unrelated side note:
Every time I read "bm0" I see and think of "BMO".
Which makes me think of "Finn and Jake".
So, intentional or not, thanks for putting that smile on my face :slight_smile:

4 Likes
16

You can use the --debug-challenges option to figure out where or if the challenge file has been generated on your file system. It obviously should make its way into /media/volumes/nginx-certbot, /media/volumes/nginx-certbot/.well-known/acme-challenge/ to be precise.

As you're using certbot renew it might also be a good idea to check the contents of the renewal configuration file to see what Certbot actually is planning to do. And also show the contents of the Certbot log file.

5 Likes
17

Thanks all, was recently on road so couldn't try these until now.

@rg305 please see the below output of the -vvv and haha yes, it is a reference to BMO!

2022-12-12 02:05:00,005:DEBUG:certbot._internal.main:certbot version: 1.30.0
2022-12-12 02:05:00,005:DEBUG:certbot._internal.main:Location of certbot entry point: /usr/local/bin/certbot
2022-12-12 02:05:00,005:DEBUG:certbot._internal.main:Arguments: ['-vvv', '--debug-challenges']
2022-12-12 02:05:00,005:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2022-12-12 02:05:00,157:DEBUG:certbot._internal.log:Root logging level set at 0
2022-12-12 02:05:00,177:DEBUG:certbot._internal.display.obj:Notifying user: Processing /etc/letsencrypt/renewal/testing.web.private.company.net.conf
2022-12-12 02:05:00,238:DEBUG:certbot._internal.plugins.selection:Requested authenticator <certbot._internal.cli.cli_utils._Default object at 0x7fe1f36c8e80> and installer <certbot._internal.cli.cli_utils._Default object at 0x7fe1f36c8e80>
2022-12-12 02:05:00,307:DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): r3.o.lencr.org:80
2022-12-12 02:05:00,315:DEBUG:urllib3.connectionpool:http://r3.o.lencr.org:80 "POST / HTTP/1.1" 200 503
2022-12-12 02:05:00,316:DEBUG:certbot.ocsp:OCSP response for certificate /etc/letsencrypt/archive/testing.web.private.company.net/cert1.pem is signed by the certificate's issuer.
2022-12-12 02:05:00,318:DEBUG:certbot.ocsp:OCSP certificate status for /etc/letsencrypt/archive/testing.web.private.company.net/cert1.pem is: OCSPCertStatus.GOOD
2022-12-12 02:05:00,347:DEBUG:certbot._internal.storage:Should renew, less than 30 days before certificate expiry 2022-12-19 13:03:53 UTC.
2022-12-12 02:05:00,348:INFO:certbot._internal.renewal:Certificate is due for renewal, auto-renewing...
2022-12-12 02:05:00,348:INFO:certbot._internal.renewal:Non-interactive renewal: random delay of 269.12494338699236 seconds
2022-12-12 02:09:29,574:DEBUG:certbot._internal.plugins.selection:Requested authenticator standalone and installer None
2022-12-12 02:09:29,582:DEBUG:certbot._internal.plugins.selection:Single candidate plugin: * standalone
Description: Spin up a temporary webserver
Interfaces: Authenticator, Plugin
Entry point: standalone = certbot._internal.plugins.standalone:Authenticator
Initialized: <certbot._internal.plugins.standalone.Authenticator object at 0x7fe1f36c8d00>
Prep: True
2022-12-12 02:09:29,583:DEBUG:certbot._internal.plugins.selection:Selected authenticator <certbot._internal.plugins.standalone.Authenticator object at 0x7fe1f36c8d00> and installer None
2022-12-12 02:09:29,583:INFO:certbot._internal.plugins.selection:Plugins selected: Authenticator standalone, Installer None
2022-12-12 02:09:29,655:DEBUG:certbot._internal.main:Picked account: <Account(RegistrationResource(body=Registration(key=None, contact=(), agreement=None, status=None, terms_of_service_agreed=None, only_return_existing=None, external_account_binding=None), uri='https://acme-v02.api.letsencrypt.org/acme/acct/740284017', new_authzr_uri=None, terms_of_service=None), c7fe34dfe347f698d0d3e6ac30caa58a, Meta(creation_dt=datetime.datetime(2022, 9, 20, 13, 35, 12, tzinfo=<UTC>), creation_host='aaf652da6a7c', register_to_eff=None))>
2022-12-12 02:09:29,656:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2022-12-12 02:09:29,658:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org:443
2022-12-12 02:09:30,209:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 659
2022-12-12 02:09:30,210:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Mon, 12 Dec 2022 02:09:30 GMT
Content-Type: application/json
Content-Length: 659
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "Aq8pUKxm6mc": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.3-September-21-2022.pdf",
    "website": "https://letsencrypt.org"
  },
  "newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",
  "revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"
}
2022-12-12 02:09:30,234:DEBUG:certbot._internal.display.obj:Notifying user: Renewing an existing certificate for testing.web.private.company.net
2022-12-12 02:09:30,400:DEBUG:certbot.crypto_util:Generating RSA key (2048 bits): /etc/letsencrypt/keys/0026_key-certbot.pem
2022-12-12 02:09:30,486:DEBUG:certbot.crypto_util:Creating CSR: /etc/letsencrypt/csr/0026_csr-certbot.pem
2022-12-12 02:09:30,488:DEBUG:acme.client:Requesting fresh nonce
2022-12-12 02:09:30,488:DEBUG:acme.client:Sending HEAD request to https://acme-v02.api.letsencrypt.org/acme/new-nonce.
2022-12-12 02:09:30,665:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "HEAD /acme/new-nonce HTTP/1.1" 200 0
2022-12-12 02:09:30,666:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Mon, 12 Dec 2022 02:09:30 GMT
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: C878PxK0gtZ8rB5aF9NRekLZ7lfBD2Dn0DSTnAS7IyyTi6s
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800


2022-12-12 02:09:30,666:DEBUG:acme.client:Storing nonce: C878PxK0gtZ8rB5aF9NRekLZ7lfBD2Dn0DSTnAS7IyyTi6s
2022-12-12 02:09:30,667:DEBUG:acme.client:JWS payload:
b'{\n  "identifiers": [\n    {\n      "type": "dns",\n      "value": "testing.web.private.company.net"\n    }\n  ]\n}'
2022-12-12 02:09:30,670:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/new-order:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvNzQwMjg0MDE3IiwgIm5vbmNlIjogIkM4NzhQeEswZ3RaOHJCNWFGOU5SZWtMWjdsZkJEMkRuMERTVG5BUzdJeXlUaTZzIiwgInVybCI6ICJodHRwczovL2FjbWUtdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9uZXctb3JkZXIifQ",
  "signature": "NgH16bNqGidFwhx6WxKrtLZ7VA-IVmYPganHh7U2-UfzVyL9xX3uKlSuyYXAyqx-_5zsb9onQoz8utLspAK3duCuVMPEXElyNGw5TStEA6EWJAseoCDZN-Kw_59OfWae9-nP3_PFlz32ul4c9dJiZWGC0vPAa_qHM8gys0MtwVsoUAsxLEsT4QRmyjEsDGTegi6AAzehDZKX-FQfi43Zvg3ZFklk8IexWGYIn6E1zLF0oHWkZ4chfBsME_4cAD4j3-Dd1sp2XG6BYzyDWGMuFFaBeWOWuewzihawzK1jAWZbi44mksHs7IO8z3hM2a64HH8fsj6kklHotinbPwRwsQ",
  "payload": "ewogICJpZGVudGlmaWVycyI6IFsKICAgIHsKICAgICAgInR5cGUiOiAiZG5zIiwKICAgICAgInZhbHVlIjogInRlcy5lbW1zLnByaXZhdGUuY3JvbWFydHkubmV0LmF1IgogICAgfQogIF0KfQ"
}
2022-12-12 02:09:31,080:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/new-order HTTP/1.1" 201 357
2022-12-12 02:09:31,081:DEBUG:acme.client:Received response:
HTTP 201
Server: nginx
Date: Mon, 12 Dec 2022 02:09:30 GMT
Content-Type: application/json
Content-Length: 357
Connection: keep-alive
Boulder-Requester: 740284017
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Location: https://acme-v02.api.letsencrypt.org/acme/order/740284017/152204702347
Replay-Nonce: 1DFAk5BWURbG6GtMU3dB2EB1psa6AiaxWb0RKpHE4dNNehE
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "status": "pending",
  "expires": "2022-12-19T02:09:30Z",
  "identifiers": [
    {
      "type": "dns",
      "value": "testing.web.private.company.net"
    }
  ],
  "authorizations": [
    "https://acme-v02.api.letsencrypt.org/acme/authz-v3/186305587547"
  ],
  "finalize": "https://acme-v02.api.letsencrypt.org/acme/finalize/740284017/152204702347"
}
2022-12-12 02:09:31,081:DEBUG:acme.client:Storing nonce: 1DFAk5BWURbG6GtMU3dB2EB1psa6AiaxWb0RKpHE4dNNehE
2022-12-12 02:09:31,081:DEBUG:acme.client:JWS payload:
b''
2022-12-12 02:09:31,083:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/authz-v3/186305587547:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvNzQwMjg0MDE3IiwgIm5vbmNlIjogIjFERkFrNUJXVVJiRzZHdE1VM2RCMkVCMXBzYTZBaWF4V2IwUktwSEU0ZE5OZWhFIiwgInVybCI6ICJodHRwczovL2FjbWUtdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hdXRoei12My8xODYzMDU1ODc1NDcifQ",
  "signature": "Y7pIP2cYF_QnqNrT9QobhUXtqFLJJTwoToEYeHj2dIPzxlS5nORFKE3QkTsoFLVM5LfEd18QfMobL2yYFOpszZoXpsgTwfY651kiasgkYs-7ySISUwtCAaqamFvWGaaIgV-jFN7oNCp1iXiWBMOtrPqsCXK7nBkrrRDC5BBfGzFycmhRpxzd1jopbxM6A8T33jlvGiqDasSP6m3b5xrBZjhkCrZcQ-TwImamw7LzsF38wDBXvNSc0JcfGZWPAs2GWAOqXVeTOwWYNnzZ9HFU6zbLIptBqGbmj9iCVSmhRH3t1Y9RleaUBk_nM67hNWVt43xsLKeDBhFxDyv9pBV4SA",
  "payload": ""
}
2022-12-12 02:09:31,262:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/186305587547 HTTP/1.1" 200 816
2022-12-12 02:09:31,263:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Mon, 12 Dec 2022 02:09:31 GMT
Content-Type: application/json
Content-Length: 816
Connection: keep-alive
Boulder-Requester: 740284017
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 5CA2WS9uR5tdqWNDAsuVC-ov9hznt0435uNJhnBT2DFF-SM
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "testing.web.private.company.net"
  },
  "status": "pending",
  "expires": "2022-12-19T02:09:30Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/186305587547/FCkiUQ",
      "token": "p73HJUYVwMOpfi4_p3ss5eVS0jModlclIyer5SMdq-s"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/186305587547/Aem1AA",
      "token": "p73HJUYVwMOpfi4_p3ss5eVS0jModlclIyer5SMdq-s"
    },
    {
      "type": "tls-alpn-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/186305587547/pPEAsQ",
      "token": "p73HJUYVwMOpfi4_p3ss5eVS0jModlclIyer5SMdq-s"
    }
  ]
}
2022-12-12 02:09:31,263:DEBUG:acme.client:Storing nonce: 5CA2WS9uR5tdqWNDAsuVC-ov9hznt0435uNJhnBT2DFF-SM
2022-12-12 02:09:31,264:INFO:certbot._internal.auth_handler:Performing the following challenges:
2022-12-12 02:09:31,264:INFO:certbot._internal.auth_handler:http-01 challenge for testing.web.private.company.net
2022-12-12 02:09:31,284:DEBUG:acme.standalone:Successfully bound to :80 using IPv6
2022-12-12 02:09:31,285:DEBUG:acme.standalone:Certbot wasn't able to bind to :80 using IPv4, this is often expected due to the dual stack nature of IPv6 socket implementations.
2022-12-12 02:09:31,287:DEBUG:certbot._internal.display.obj:Notifying user: Challenges loaded. Press continue to submit to CA.

The following URLs should be accessible from the internet and return the value
mentioned:

URL:
http://testing.web.private.company.net/.well-known/acme-challenge/p73HJUYVwMOpfi4_p3ss5eVS0jModlclIyer5SMdq-s
Expected value:
p73HJUYVwMOpfi4_p3ss5eVS0jModlclIyer5SMdq-s.ofpxAIJnHdH-RZ5aeBCPjpHdU70rTXd377oE7XjTo20
2022-12-12 02:09:31,287:DEBUG:acme.client:JWS payload:
b'{}'
2022-12-12 02:09:31,289:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/chall-v3/186305587547/FCkiUQ:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvNzQwMjg0MDE3IiwgIm5vbmNlIjogIjVDQTJXUzl1UjV0ZHFXTkRBc3VWQy1vdjloem50MDQzNXVOSmhuQlQyREZGLVNNIiwgInVybCI6ICJodHRwczovL2FjbWUtdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9jaGFsbC12My8xODYzMDU1ODc1NDcvRkNraVVRIn0",
  "signature": "GR87TFlyWT1cohS5dIHvAF8KwvnaRf2yT9YoieavbioVXCroY9BNulJmj0NZM6adkq5H8eH8_ARL-68QJPnFOrZ8ACfP4cntam5RM8CkzypzmRM2LgN6NwgrGmN5Snnb5v6Re_CvmjNz0BRymYzSfm5JBFWpixNSJiOA0XvXYW0HXjpdxhw4Pe7oPTlgbbzjo1KdkzI8G00DThH0VvPGpW22AT5-4wspFjhRHMWU-o-BhWdU377zR13u3MIm8r0F7waGM6kkW4w7Lx61p1oQBfIbiat7oRyMaOAW5Mt4AxYD3n-5V0q1zHivQLRDrT7KfAbZIYR4OZqyYOPdhwTUog",
  "payload": "e30"
}
2022-12-12 02:09:31,471:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/chall-v3/186305587547/FCkiUQ HTTP/1.1" 200 187
2022-12-12 02:09:31,472:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Mon, 12 Dec 2022 02:09:31 GMT
Content-Type: application/json
Content-Length: 187
Connection: keep-alive
Boulder-Requester: 740284017
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index", <https://acme-v02.api.letsencrypt.org/acme/authz-v3/186305587547>;rel="up"
Location: https://acme-v02.api.letsencrypt.org/acme/chall-v3/186305587547/FCkiUQ
Replay-Nonce: 5CA2ZZBVsT8obq8ESV9XsOM_ITwYP8OdgBT3Ouwf-qkUKQ4
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "type": "http-01",
  "status": "pending",
  "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/186305587547/FCkiUQ",
  "token": "p73HJUYVwMOpfi4_p3ss5eVS0jModlclIyer5SMdq-s"
}
2022-12-12 02:09:31,472:DEBUG:acme.client:Storing nonce: 5CA2ZZBVsT8obq8ESV9XsOM_ITwYP8OdgBT3Ouwf-qkUKQ4
2022-12-12 02:09:31,473:INFO:certbot._internal.auth_handler:Waiting for verification...
2022-12-12 02:09:32,474:DEBUG:acme.client:JWS payload:
b''
2022-12-12 02:09:32,476:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/authz-v3/186305587547:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvNzQwMjg0MDE3IiwgIm5vbmNlIjogIjVDQTJaWkJWc1Q4b2JxOEVTVjlYc09NX0lUd1lQOE9kZ0JUM091d2YtcWtVS1E0IiwgInVybCI6ICJodHRwczovL2FjbWUtdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hdXRoei12My8xODYzMDU1ODc1NDcifQ",
  "signature": "M8_GATmKVWInD4DWvSFj_zg5_IpvoxyeQ9hqzbS_7ipJQtAjrsxPgR2HH7HmopVkaQWF6aRGgyoGJwLK1-70YYfqnDMn1egySmc8fe_tCqCvYf0ZebX0n5eGMCGS7UeGkTVw5uSYPCARKb5PnbkHn0g2ZiBDbbSau54ptK9ZXT-ipQS0xZcWU4GI8VjYf0NCw2dJ4wxh4XSsPHk3PKyhs8aye4a_gnH8RJD2t7PuWKZJJjqcQhAi1hjtROo6N54Nr-Ua-5Cs3Kz3gB2Mvhdfg37-2DWf_pOunVd1_jiVo1qak_OA9ceJkHBXbbMVTJzhXt6WDK8fEcPz6l6YUpApxg",
  "payload": ""
}
2022-12-12 02:09:32,657:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/186305587547 HTTP/1.1" 200 1093
2022-12-12 02:09:32,658:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Mon, 12 Dec 2022 02:09:32 GMT
Content-Type: application/json
Content-Length: 1093
Connection: keep-alive
Boulder-Requester: 740284017
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: C878rdp4lRW70HKptQWYUelxkDbrmDXdBb7AxsW1hmqgfL8
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "testing.web.private.company.net"
  },
  "status": "invalid",
  "expires": "2022-12-19T02:09:30Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:unauthorized",
        "detail": "<redacted.ipaddress>: Invalid response from http://testing.web.private.company.net/.well-known/acme-challenge/p73HJUYVwMOpfi4_p3ss5eVS0jModlclIyer5SMdq-s: 404",
        "status": 403
      },
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/186305587547/FCkiUQ",
      "token": "p73HJUYVwMOpfi4_p3ss5eVS0jModlclIyer5SMdq-s",
      "validationRecord": [
        {
          "url": "http://testing.web.private.company.net/.well-known/acme-challenge/p73HJUYVwMOpfi4_p3ss5eVS0jModlclIyer5SMdq-s",
          "hostname": "testing.web.private.company.net",
          "port": "80",
          "addressesResolved": [
            "<redacted.ipaddress>"
          ],
          "addressUsed": "<redacted.ipaddress>"
        }
      ],
      "validated": "2022-12-12T02:09:31Z"
    }
  ]
}
2022-12-12 02:09:32,659:DEBUG:acme.client:Storing nonce: C878rdp4lRW70HKptQWYUelxkDbrmDXdBb7AxsW1hmqgfL8
2022-12-12 02:09:32,659:INFO:certbot._internal.auth_handler:Challenge failed for domain testing.web.private.company.net
2022-12-12 02:09:32,660:INFO:certbot._internal.auth_handler:http-01 challenge for testing.web.private.company.net
2022-12-12 02:09:32,660:DEBUG:certbot._internal.display.obj:Notifying user: 
Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems:
  Domain: testing.web.private.company.net
  Type:   unauthorized
  Detail: <redacted.ipaddress>: Invalid response from http://testing.web.private.company.net/.well-known/acme-challenge/p73HJUYVwMOpfi4_p3ss5eVS0jModlclIyer5SMdq-s: 404

Hint: The Certificate Authority failed to download the challenge files from the temporary standalone webserver started by Certbot on port 80. Ensure that the listed domains point to this machine and that it can accept inbound connections from the internet.

2022-12-12 02:09:32,661:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/opt/certbot/src/certbot/certbot/_internal/auth_handler.py", line 106, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/opt/certbot/src/certbot/certbot/_internal/auth_handler.py", line 206, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2022-12-12 02:09:32,661:DEBUG:certbot._internal.error_handler:Calling registered functions
2022-12-12 02:09:32,661:INFO:certbot._internal.auth_handler:Cleaning up challenges
2022-12-12 02:09:32,662:DEBUG:certbot._internal.plugins.standalone:Stopping server at :::80...
2022-12-12 02:09:32,790:ERROR:certbot._internal.renewal:Failed to renew certificate testing.web.private.company.net with error: Some challenges have failed.
2022-12-12 02:09:32,792:DEBUG:certbot._internal.renewal:Traceback was:
Traceback (most recent call last):
  File "/opt/certbot/src/certbot/certbot/_internal/renewal.py", line 484, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/opt/certbot/src/certbot/certbot/_internal/main.py", line 1541, in renew_cert
    renewed_lineage = _get_and_save_cert(le_client, config, lineage=lineage)
  File "/opt/certbot/src/certbot/certbot/_internal/main.py", line 129, in _get_and_save_cert
    renewal.renew_cert(config, domains, le_client, lineage)
  File "/opt/certbot/src/certbot/certbot/_internal/renewal.py", line 344, in renew_cert
    new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
  File "/opt/certbot/src/certbot/certbot/_internal/client.py", line 442, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/opt/certbot/src/certbot/certbot/_internal/client.py", line 510, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/opt/certbot/src/certbot/certbot/_internal/auth_handler.py", line 106, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/opt/certbot/src/certbot/certbot/_internal/auth_handler.py", line 206, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2022-12-12 02:09:32,794:DEBUG:certbot._internal.display.obj:Notifying user: 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2022-12-12 02:09:32,794:ERROR:certbot._internal.renewal:All renewals failed. The following certificates could not be renewed:
2022-12-12 02:09:32,794:ERROR:certbot._internal.renewal:  /etc/letsencrypt/live/testing.web.private.company.net/fullchain.pem (failure)
2022-12-12 02:09:32,795:DEBUG:certbot._internal.display.obj:Notifying user: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2022-12-12 02:09:32,795:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/local/bin/certbot", line 33, in <module>
    sys.exit(load_entry_point('certbot', 'console_scripts', 'certbot')())
  File "/opt/certbot/src/certbot/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
  File "/opt/certbot/src/certbot/certbot/_internal/main.py", line 1744, in main
    return config.func(config, plugins)
  File "/opt/certbot/src/certbot/certbot/_internal/main.py", line 1630, in renew
    renewal.handle_renewal_request(config)
  File "/opt/certbot/src/certbot/certbot/_internal/renewal.py", line 510, in handle_renewal_request
    raise errors.Error(
certbot.errors.Error: 1 renew failure(s), 0 parse failure(s)
2022-12-12 02:09:32,796:ERROR:certbot._internal.log:1 renew failure(s), 0 parse failure(s)

@Osiris thank you, the --debug-challenges was added to the above output command as well. Please also see the below the renewal config below:

# renew_before_expiry = 30 days
version = 1.30.0
archive_dir = /etc/letsencrypt/archive/testing.web.private.company.net
cert = /etc/letsencrypt/live/testing.web.private.company.net/cert.pem
privkey = /etc/letsencrypt/live/testing.web.private.company.net/privkey.pem
chain = /etc/letsencrypt/live/testing.web.private.company.net/chain.pem
fullchain = /etc/letsencrypt/live/testing.web.private.company.net/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = c7fe34dfe347f698d0d3e6ac30caa58a
authenticator = standalone
server = https://acme-v02.api.letsencrypt.org/directory
key_type = rsa

Side Question: I'm just wondering, the original build command was like the below:

docker run --rm \
  -p 443:443 -p 80:80 --name letsencrypt \
  -v "/media/volumes/nginx-certificates:/etc/letsencrypt" \
  -v "/media/volumes/nginx-certificates:/var/lib/letsencrypt" \
  certbot/certbot certonly -n \
  -m "<redacted.email>" \
  -d tes.emms.private.cromarty.net.au \
  --standalone --agree-tos

and the renew is now like the following:

  docker run --rm --name letsencrypt \
    -v "/media/volumes/nginx-certificates:/etc/letsencrypt" \
    -v "/media/volumes/nginx-certbot:/var/log/letsencrypt" \
    -v "/media/volumes/nginx-certbot:/var/www/certbot" \
    certbot/certbot:latest \
    renew --quiet

I'm not hugely familiar with the certbot command, would the slight change in directories above affect it?

Thanks

1 Like
19

What happened to?:

The error is:

 "type": "http-01",
 "status": "invalid",
 "error": {
 "type": "urn:ietf:params:acme:error:unauthorized",
 "detail": "<redacted.ipaddress>: Invalid response from http://testing.web.private.company.net/.well-known/acme-challenge/p73HJUYVwMOpfi4_p3ss5eVS0jModlclIyer5SMdq-s: 404",
 "status": 403

But, while using --standalone, it will be difficult to troubleshoot.

5 Likes
20

I see nginx mentioned within your command...
Are you against using nginx to handle the HTTP challenge requests?

5 Likes
21

Hm, standalone.. I'd figure the webroot authenticator was used. Standalone could be used if requests for /.well-known/acme-challenge/ were to reverse proxied to the Certbot container from nginx. But webroot makes much more sense.

4 Likes
22

@rg305 the nginx is configured with port 80/443 and handles the http challenge request with the below configuration:

server {
    listen 80;
    server_name <webserver.domain>;
    server_tokens off;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl http2;
	server_name <webserver.domain>;
    server_tokens off;

	proxy_intercept_errors on;
	error_page 401 495 496 @json401error;

    ssl_certificate /etc/letsencrypt/live/<webserver.domain>/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/<webserver.domain>/privkey.pem;
	ssl_protocols TLSv1.2;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    # mTLS handler
	# This will return a 401 to all clients without a proper certificate
    if ($ssl_client_verify != SUCCESS) { return 401; }
	
	# This tells Nginx what CA to verify against
    ssl_client_certificate /certificates/cacert.pem;
    ssl_verify_depth 2;
	
	# This tells Nginx to verify clients
    ssl_verify_client optional;
	
    location @json401error {
        default_type application/json;
		add_header 'Access-Control-Allow-Origin' '*' always;
		return 401 '{"errors":["401 Unauthorized"]}';
	}

    location / {
		proxy_pass http://<redacted>;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header SSL_Client $ssl_client_s_dn;
        proxy_set_header SSL_Client_Verify $ssl_client_verify;
		proxy_ssl_verify off;
		
    }
}

Hmm unless is there anything I might be missing here?