Docker + Alpine 3.15 + Apache 2.4 -> Some challenges have failed

Can't seem to get the challenge to go through successfully on Docker + Alpine 3.15 + Apache 2.4.

Not sure what I'm doing wrong, it would be super helpful if I could suspend the process part way through and inspect /etc/apache2/conf/vhost.conf and /var/lib/letsencrypt/http_challenges to see that the access rules and challenge are where they are supposed to be... is there a way to do that or tell certbot to leave the files afterwards?

My domain is: ah-test.duckdns.org

I ran this command:

Dockerfile

FROM httpd:2.4-alpine3.15
RUN apk add --no-cache apache2-proxy apache2-ssl certbot-apache
EXPOSE 443
### Build on docker host
docker build --pull --no-cache . --tag ah-proxy:latest
docker run --rm -it -p 8888:80/tcp --entrypoint /bin/sh ah-proxy:latest

## below is running inside the container

# Configure host
echo '
<VirtualHost _default_:80>
    DocumentRoot "/var/www/html"
    ServerName ah-test.duckdns.org

#    RewriteEngine on
#    RewriteRule ^/\.well-known/acme-challenge/([A-Za-z0-9-_=]+)$ /var/lib/letsencrypt/http_challenges/$1
#    <Directory /var/lib/letsencrypt/http_challenges>
#      Require all granted
#    </Directory>
#    <Location /.well-known/acme-challenge>
#      Require all granted
#    </Location>
</VirtualHost>

# <VirtualHost *:80>
#    DocumentRoot "/var/www/idm"
#    ServerName idm.ah-test.duckdns.org
#
#    # Other directives here
# </VirtualHost>
' > /etc/apache2/conf.d/vhost.conf

# help certbot which is looking for bins in the wrong places
ln -s /usr/local/apache2/bin/apachectl /usr/local/apache2/bin/apache2ctl
ln -s /usr/lib/apache2 /etc/apache2/modules

# start Apache
apachectl -t
# Syntax OK
apachectl start

certbot -n -m deftdawg@gmail.com --agree-tos -v --apache --test-cert -d ah-test.duckdns.org

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Requesting a certificate for ah-test.duckdns.org
Performing the following challenges:
http-01 challenge for ah-test.duckdns.org
Waiting for verification...
Challenge failed for domain ah-test.duckdns.org
http-01 challenge for ah-test.duckdns.org

Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
  Domain: ah-test.duckdns.org
  Type:   unauthorized
  Detail: Invalid response from http://ah-test.duckdns.org/.well-known/acme-challenge/jSimK4Au0d_MjeJ3l266pBH6ec0v1A1sniUCnJQ7jNM [142.116.103.226]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.

Cleaning up challenges
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

cat /var/log/letsencrypt/letsencrypt.log

2022-01-05 23:45:43,585:DEBUG:certbot._internal.main:certbot version: 1.21.0
2022-01-05 23:45:43,586:DEBUG:certbot._internal.main:Location of certbot entry point: /usr/bin/certbot
2022-01-05 23:45:43,586:DEBUG:certbot._internal.main:Arguments: ['-n', '-m', 'deftdawg@gmail.com', '--agree-tos', '-v', '--apache', '--test-cert', '-d', 'ah-test.duckdns.org']
2022-01-05 23:45:43,586:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2022-01-05 23:45:43,600:DEBUG:certbot._internal.log:Root logging level set at 20
2022-01-05 23:45:43,601:DEBUG:certbot._internal.plugins.selection:Requested authenticator apache and installer apache
2022-01-05 23:45:43,662:DEBUG:certbot_apache._internal.configurator:Apache version is 2.4.52
2022-01-05 23:45:43,835:DEBUG:certbot._internal.plugins.selection:Single candidate plugin: * apache
Description: Apache Web Server plugin
Interfaces: Installer, Authenticator, Plugin
Entry point: apache = certbot_apache._internal.entrypoint:ENTRYPOINT
Initialized: <certbot_apache._internal.configurator.ApacheConfigurator object at 0x7f6cc004d880>
Prep: True
2022-01-05 23:45:43,836:DEBUG:certbot._internal.plugins.selection:Selected authenticator <certbot_apache._internal.configurator.ApacheConfigurator object at 0x7f6cc004d880> and installer <certbot_apache._internal.configurator.ApacheConfigurator object at 0x7f6cc004d880>
2022-01-05 23:45:43,836:INFO:certbot._internal.plugins.selection:Plugins selected: Authenticator apache, Installer apache
2022-01-05 23:45:43,843:DEBUG:certbot._internal.main:Picked account: <Account(RegistrationResource(body=Registration(key=None, contact=(), agreement=None, status=None, terms_of_service_agreed=None, only_return_existing=None, external_account_binding=None), uri='https://acme-staging-v02.api.letsencrypt.org/acme/acct/39496698', new_authzr_uri=None, terms_of_service=None), cc1dc424a980947e887f0cd95fb4db52, Meta(creation_dt=datetime.datetime(2022, 1, 5, 23, 39, 27, tzinfo=<UTC>), creation_host='e084f40237a5', register_to_eff=None))>
2022-01-05 23:45:43,844:DEBUG:acme.client:Sending GET request to https://acme-staging-v02.api.letsencrypt.org/directory.
2022-01-05 23:45:43,847:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org:443
2022-01-05 23:45:44,011:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 822
2022-01-05 23:45:44,012:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Wed, 05 Jan 2022 23:45:44 GMT
Content-Type: application/json
Content-Length: 822
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "Ivpi16rSxuU": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-staging-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
    "website": "https://letsencrypt.org/docs/staging-environment/"
  },
  "newAccount": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order",
  "renewalInfo": "https://acme-staging-v02.api.letsencrypt.org/get/draft-aaron-ari/renewalInfo/",
  "revokeCert": "https://acme-staging-v02.api.letsencrypt.org/acme/revoke-cert"
}
2022-01-05 23:45:44,013:DEBUG:certbot._internal.display.obj:Notifying user: Requesting a certificate for ah-test.duckdns.org
2022-01-05 23:45:44,203:DEBUG:certbot.crypto_util:Generating RSA key (2048 bits): /etc/letsencrypt/keys/0003_key-certbot.pem
2022-01-05 23:45:44,205:DEBUG:certbot.crypto_util:Creating CSR: /etc/letsencrypt/csr/0003_csr-certbot.pem
2022-01-05 23:45:44,206:DEBUG:acme.client:Requesting fresh nonce
2022-01-05 23:45:44,206:DEBUG:acme.client:Sending HEAD request to https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce.
2022-01-05 23:45:44,257:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "HEAD /acme/new-nonce HTTP/1.1" 200 0
2022-01-05 23:45:44,258:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Wed, 05 Jan 2022 23:45:44 GMT
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 0001YUEL8aBHKDri_ZSnFf1KHV_We_90aJvvAZWVBqUfB6Q
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800


2022-01-05 23:45:44,258:DEBUG:acme.client:Storing nonce: 0001YUEL8aBHKDri_ZSnFf1KHV_We_90aJvvAZWVBqUfB6Q
2022-01-05 23:45:44,258:DEBUG:acme.client:JWS payload:
b'{\n  "identifiers": [\n    {\n      "type": "dns",\n      "value": "ah-test.duckdns.org"\n    }\n  ]\n}'
2022-01-05 23:45:44,260:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/new-order:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8zOTQ5NjY5OCIsICJub25jZSI6ICIwMDAxWVVFTDhhQkhLRHJpX1pTbkZmMUtIVl9XZV85MGFKdnZBWldWQnFVZkI2USIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9uZXctb3JkZXIifQ",
  "signature": "ErD0COE9HljzHkj9e6LOuokf6AIeRDqOWN7nFQn14Os39MiAHNLNDVnuqqVY6rEUivqPhS0kNm0BaPp9_y0cetq4VACp_BmMQ8M760QUoQ4j90C2sz-4dQlQrbIQmYcOUFy25T5kz8po_-wJKUDUH1gJLWarKJB3ZpTaUY3WYNkjCin_FdkEbf-PR_h_dfsQ7yo4Mnp-EOy8COz3iaHG1NyPd_j_BcfSg-Qy5CtxnzLq7HgiPaXoBCjsrm9B25ekp3FsYt1aLWpdfLmt8D0gqGn3b1i5ppMq5udcZmrIW79e8YFf4_hIjzJE6Rin4vLnvWtny0ZrFUwsXheyoIHSYg",
  "payload": "ewogICJpZGVudGlmaWVycyI6IFsKICAgIHsKICAgICAgInR5cGUiOiAiZG5zIiwKICAgICAgInZhbHVlIjogImFoLXRlc3QuZHVja2Rucy5vcmciCiAgICB9CiAgXQp9"
}
2022-01-05 23:45:44,333:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/new-order HTTP/1.1" 201 355
2022-01-05 23:45:44,334:DEBUG:acme.client:Received response:
HTTP 201
Server: nginx
Date: Wed, 05 Jan 2022 23:45:44 GMT
Content-Type: application/json
Content-Length: 355
Connection: keep-alive
Boulder-Requester: 39496698
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Location: https://acme-staging-v02.api.letsencrypt.org/acme/order/39496698/1452655838
Replay-Nonce: 000264Io2gejngjtJ1Igas-izZCjYl1RktjOqgjT1q3Z2LY
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "status": "pending",
  "expires": "2022-01-12T23:45:44Z",
  "identifiers": [
    {
      "type": "dns",
      "value": "ah-test.duckdns.org"
    }
  ],
  "authorizations": [
    "https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/1353108898"
  ],
  "finalize": "https://acme-staging-v02.api.letsencrypt.org/acme/finalize/39496698/1452655838"
}
2022-01-05 23:45:44,334:DEBUG:acme.client:Storing nonce: 000264Io2gejngjtJ1Igas-izZCjYl1RktjOqgjT1q3Z2LY
2022-01-05 23:45:44,334:DEBUG:acme.client:JWS payload:
b''
2022-01-05 23:45:44,335:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/1353108898:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8zOTQ5NjY5OCIsICJub25jZSI6ICIwMDAyNjRJbzJnZWpuZ2p0SjFJZ2FzLWl6WkNqWWwxUmt0ak9xZ2pUMXEzWjJMWSIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hdXRoei12My8xMzUzMTA4ODk4In0",
  "signature": "NXMGxmV94eF45q7bFezPBar1v1BhWHoJ-YRGDuU4FJn_0B2Zv29yudYv__Jt67-WupgUigJlu4zr1CHTa7s5_l4_xbo3-17dtbAVddMspofq77Q1h0DUUHGHqphaXllcg-dZN-MGJlks191DQRHtMkod-N4B9ypRM7u7W7W3sk0yC5eVZjkqFUSatoEMHYT1CwkvRp6_ypHNrZWBLvFUwG7o9CiBzvINlj0E0xvm2FBGkRj5nYHNIoxh3bERiU_sFI9wwfooBnpBE74nytB05byo0SvgGFb1zzWXjDnD6cpFYYfCTX85xFAq7awB_OaenNZ-QBRuz6gTBRXX84HDOA",
  "payload": ""
}
2022-01-05 23:45:44,390:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/1353108898 HTTP/1.1" 200 821
2022-01-05 23:45:44,391:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Wed, 05 Jan 2022 23:45:44 GMT
Content-Type: application/json
Content-Length: 821
Connection: keep-alive
Boulder-Requester: 39496698
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 0002vq8E137xEcPewO-0VzieBVXpUg3qZ0K-RNsLEUQbjBw
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "ah-test.duckdns.org"
  },
  "status": "pending",
  "expires": "2022-01-12T23:45:44Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/1353108898/N-8RjQ",
      "token": "jSimK4Au0d_MjeJ3l266pBH6ec0v1A1sniUCnJQ7jNM"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/1353108898/rqZJ8g",
      "token": "jSimK4Au0d_MjeJ3l266pBH6ec0v1A1sniUCnJQ7jNM"
    },
    {
      "type": "tls-alpn-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/1353108898/reMtnA",
      "token": "jSimK4Au0d_MjeJ3l266pBH6ec0v1A1sniUCnJQ7jNM"
    }
  ]
}
2022-01-05 23:45:44,391:DEBUG:acme.client:Storing nonce: 0002vq8E137xEcPewO-0VzieBVXpUg3qZ0K-RNsLEUQbjBw
2022-01-05 23:45:44,391:INFO:certbot._internal.auth_handler:Performing the following challenges:
2022-01-05 23:45:44,392:INFO:certbot._internal.auth_handler:http-01 challenge for ah-test.duckdns.org
2022-01-05 23:45:44,395:DEBUG:certbot_apache._internal.http_01:Adding a temporary challenge validation Include for name: ah-test.duckdns.org in: /etc/apache2/conf.d/vhost.conf
2022-01-05 23:45:44,396:DEBUG:certbot_apache._internal.http_01:writing a pre config file with text:
         RewriteEngine on
        RewriteRule ^/\.well-known/acme-challenge/([A-Za-z0-9-_=]+)$ /var/lib/letsencrypt/http_challenges/$1 [END]
    
2022-01-05 23:45:44,396:DEBUG:certbot_apache._internal.http_01:writing a post config file with text:
         <Directory /var/lib/letsencrypt/http_challenges>
            Require all granted
        </Directory>
        <Location /.well-known/acme-challenge>
            Require all granted
        </Location>
    
2022-01-05 23:45:44,413:DEBUG:certbot.reverter:Creating backup of /etc/apache2/conf.d/vhost.conf
2022-01-05 23:45:47,496:DEBUG:acme.client:JWS payload:
b'{}'
2022-01-05 23:45:47,499:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/1353108898/N-8RjQ:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8zOTQ5NjY5OCIsICJub25jZSI6ICIwMDAydnE4RTEzN3hFY1Bld08tMFZ6aWVCVlhwVWczcVowSy1STnNMRVVRYmpCdyIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9jaGFsbC12My8xMzUzMTA4ODk4L04tOFJqUSJ9",
  "signature": "HvMXLfs2D8vr2k5C-leNnSb9Kn0lamnSjmDuIyVCwtLgg1VUZR883MVeuHUQ55iIy3IJoR3Gh5JW1PDvVPXpmifT0ZgJ57rTlfhSdY4e3TH99AaIMLZSaFoTq94-XEJmZthQrlwLcPMBlzGpb0n152EKUvbkd9n0hiOpFUROBeV_7MrraG389AZ6uNXre6ulpyS_pRy5uAy91HYGD3kZNjOQI6rVwW5Mx1Say0IAYigsSsM3w0mSlcV23vScUOAupNxyPKQ4-LfZgyQDh-gRciC7NyykLskUujeZAd7XAKuis3-1GA8hTBrYPFgBCJ0ddqcoy0pOP5DIOspFcrMNHw",
  "payload": "e30"
}
2022-01-05 23:45:47,563:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/chall-v3/1353108898/N-8RjQ HTTP/1.1" 200 193
2022-01-05 23:45:47,564:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Wed, 05 Jan 2022 23:45:47 GMT
Content-Type: application/json
Content-Length: 193
Connection: keep-alive
Boulder-Requester: 39496698
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index", <https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/1353108898>;rel="up"
Location: https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/1353108898/N-8RjQ
Replay-Nonce: 0002jBriTXuC1B3sglo7Bf0To_4Mm7U2a9-h6QfZvWpRBLQ
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "type": "http-01",
  "status": "pending",
  "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/1353108898/N-8RjQ",
  "token": "jSimK4Au0d_MjeJ3l266pBH6ec0v1A1sniUCnJQ7jNM"
}
2022-01-05 23:45:47,565:DEBUG:acme.client:Storing nonce: 0002jBriTXuC1B3sglo7Bf0To_4Mm7U2a9-h6QfZvWpRBLQ
2022-01-05 23:45:47,565:INFO:certbot._internal.auth_handler:Waiting for verification...
2022-01-05 23:45:48,569:DEBUG:acme.client:JWS payload:
b''
2022-01-05 23:45:48,572:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/1353108898:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8zOTQ5NjY5OCIsICJub25jZSI6ICIwMDAyakJyaVRYdUMxQjNzZ2xvN0JmMFRvXzRNbTdVMmE5LWg2UWZadldwUkJMUSIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hdXRoei12My8xMzUzMTA4ODk4In0",
  "signature": "pUnDYdKMa8UsQ-Yi8ZAcenNDoArs5G8CIaEHrmpX23MXpLc2EczTTwYYG6E7lcoKmJ55MYHNq-5_x-TAS1RhLvCtRo95SnGzLUlFL4pTNqpy4bzEVHkWP0HMuAyf30pnCFi7y7VtnsvQ4bobL0PsgT-YI3JKECtswK8crSjoHk9XDaK_yLwuSiwmo0tGnY2wl8LNt6UPyzdNORlbgO-8CAgnnX86iZNJUi5Tc2yV7Ciy49RcH6bJ8yr3I8JVcjH0Bn7wXpfBuUnhuwkRoNd1uTUyXPs47PDMFSI7NSII_dADbIfLMGRjyAfPez4Gkb1AcMMpryoorgXzUm6ud2HcIA",
  "payload": ""
}
2022-01-05 23:45:48,633:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/1353108898 HTTP/1.1" 200 1297
2022-01-05 23:45:48,634:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Wed, 05 Jan 2022 23:45:48 GMT
Content-Type: application/json
Content-Length: 1297
Connection: keep-alive
Boulder-Requester: 39496698
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 00021Ulp3NG_Qk1e4aStU2AS-TSncY8QGsxPRHav8KvAsVQ
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "ah-test.duckdns.org"
  },
  "status": "invalid",
  "expires": "2022-01-12T23:45:44Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:unauthorized",
        "detail": "Invalid response from http://ah-test.duckdns.org/.well-known/acme-challenge/jSimK4Au0d_MjeJ3l266pBH6ec0v1A1sniUCnJQ7jNM [142.116.103.226]: \"\u003c!DOCTYPE HTML PUBLIC \\\"-//IETF//DTD HTML 2.0//EN\\\"\u003e\\n\u003chtml\u003e\u003chead\u003e\\n\u003ctitle\u003e404 Not Found\u003c/title\u003e\\n\u003c/head\u003e\u003cbody\u003e\\n\u003ch1\u003eNot Found\u003c/h1\u003e\\n\u003cp\"",
        "status": 403
      },
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/1353108898/N-8RjQ",
      "token": "jSimK4Au0d_MjeJ3l266pBH6ec0v1A1sniUCnJQ7jNM",
      "validationRecord": [
        {
          "url": "http://ah-test.duckdns.org/.well-known/acme-challenge/jSimK4Au0d_MjeJ3l266pBH6ec0v1A1sniUCnJQ7jNM",
          "hostname": "ah-test.duckdns.org",
          "port": "80",
          "addressesResolved": [
            "142.116.103.226"
          ],
          "addressUsed": "142.116.103.226"
        }
      ],
      "validated": "2022-01-05T23:45:47Z"
    }
  ]
}
2022-01-05 23:45:48,635:DEBUG:acme.client:Storing nonce: 00021Ulp3NG_Qk1e4aStU2AS-TSncY8QGsxPRHav8KvAsVQ
2022-01-05 23:45:48,635:INFO:certbot._internal.auth_handler:Challenge failed for domain ah-test.duckdns.org
2022-01-05 23:45:48,636:INFO:certbot._internal.auth_handler:http-01 challenge for ah-test.duckdns.org
2022-01-05 23:45:48,636:DEBUG:certbot._internal.display.obj:Notifying user: 
Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
  Domain: ah-test.duckdns.org
  Type:   unauthorized
  Detail: Invalid response from http://ah-test.duckdns.org/.well-known/acme-challenge/jSimK4Au0d_MjeJ3l266pBH6ec0v1A1sniUCnJQ7jNM [142.116.103.226]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.

2022-01-05 23:45:48,637:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/certbot/_internal/auth_handler.py", line 90, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/auth_handler.py", line 178, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2022-01-05 23:45:48,638:DEBUG:certbot._internal.error_handler:Calling registered functions
2022-01-05 23:45:48,638:INFO:certbot._internal.auth_handler:Cleaning up challenges
2022-01-05 23:45:48,735:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 33, in <module>
    sys.exit(load_entry_point('certbot==1.21.0', 'console_scripts', 'certbot')())
  File "/usr/lib/python3.9/site-packages/certbot/main.py", line 15, in main
    return internal_main.main(cli_args)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/main.py", line 1574, in main
    return config.func(config, plugins)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/main.py", line 1287, in run
    new_lineage = _get_and_save_cert(le_client, config, domains,
  File "/usr/lib/python3.9/site-packages/certbot/_internal/main.py", line 133, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/client.py", line 459, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/client.py", line 389, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/client.py", line 439, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/auth_handler.py", line 90, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/auth_handler.py", line 178, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.
2022-01-05 23:45:48,736:ERROR:certbot._internal.log:Some challenges have failed.

My web server is (include version): apache2-2.4.52

apachectl -S

VirtualHost configuration:
ServerRoot: "/usr/local/apache2"
Main DocumentRoot: "/usr/local/apache2/htdocs"
Main ErrorLog: "/proc/self/fd/2"
Mutex default: dir="/usr/local/apache2/logs/" mechanism=default 
PidFile: "/usr/local/apache2/logs/httpd.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
User: name="www-data" id=82
Group: name="www-data" id=82

The operating system my web server runs on is (include version): Alpine 3.15 (amd64)

My hosting provider, if applicable, is: (docker self-hosted)

I can login to a root shell on my machine (yes or no, or I don't know): yes

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.21.0

I'm NO docker expert... But that seems backwards to me.
8888:80
shouldn't that be:
80:8888 (or 80:80)

1 Like

This is the only thing that catches my eye:

The string default , which is used only with IP virtual hosting to catch unmatched IP addresses.

Try replace it with

<VirtualHost *:80>
1 Like

It's right, I should've mentioned my firewall is mapping public_ip:80 -> dockerhost:8888 and dockerhost:8888 is ingressing to the container's port 80 where Apache is listening.

Ah shoot... I just noticed Apache -S

ServerRoot: "/usr/local/apache2"

This container is looking for conf and hotdogs under /usr/local/apache2 not under /etc/apache2!

1 Like

Yes, just noticed the same thing while trying it out myself:

[pid  1298] openat(AT_FDCWD, "/usr/local/apache2/conf/httpd.conf", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3

Oops! Looks like some confusion with the container as to where the authoritative Apache installation is, due to mixing apk with the apache.org version of httpd.

Moving the vhost to the other Apache installation worked OK for me (making sure to include it + enable mod_rewrite) in combination with this Certbot flag:

 --apache-server-root /usr/local/apache2/

Or I guess you could just try normal apache2 from alpine:3.15, rather than the first-party Apache.org Alpine image.

3 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.