Weird issue certbotting one of my domains

My domain is:
geronimostade.de
www.geronimostade.de

I ran this command:
sudo certbot --apache

It produced this output:

Detail: 85.214.142.73: Invalid response from https://www.geronimostade.de/.well-known/acme-challenge/-rdUEV5djDFU7jHKm1QcTTEDa4ShmCSwHSVI63Wl6_8: 404

To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address

My web server is (include version):
apache2 Ubuntu 18.04

The operating system my web server runs on is (include version):
Ubuntu 18.04

My hosting provider, if applicable, is:
strato.de

I can login to a root shell on my machine (yes or no, or I don't know):
Sure

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
No - command line ninja here :ninja:

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

certbot 0.27.0 (ancient)

Hi together,

let me explain a weird issue.

I have my domains configured at providerside to just link to an IP. In this case its: 85.214.142.73

  1. mentalfall.de
  2. www.mentalfall.de
  3. mentalfall.com
  4. www.mentalfall.com
  5. geronimostade.de
  6. www.geronimostade.de

All are individual vhosts in my apache config. And all accept for no 6 went through the certbot without any significant issue. Only the www.geronimostade.de refuses to do the acme challenge.
I have to mention that 5. and 6. point to the exact same apache folder, both are configured the same accept for the vhost name of course.

Stripped down here for good measure.

<VirtualHost www.geronimostade.de:80>

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html/web/geronimostade.de/

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

Any ideas where to look for? For now i have myself locked out for "who knows how long :smiley: " because of attemting to often to fix it.
I cant see the error.... Please help me :smile:

Heres some output of the letsencrypt log:

2024-05-30 22:03:04,681:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/357547420592 HTTP/1.1" 200 1376
2024-05-30 22:03:04,681:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Thu, 30 May 2024 20:03:04 GMT
Content-Type: application/json
Content-Length: 1376
Connection: keep-alive
Boulder-Requester: 1755133552
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: cp09ZmYxBXqE-uTgSR3DsZd4uI3zPW7IQyXuozo3pFgmie9tV18
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
{
  "identifier": {
    "type": "dns",
    "value": "www.geronimostade.de"
  },
  "status": "invalid",
  "expires": "2024-06-06T20:02:57Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:unauthorized",
        "detail": "85.214.142.73: Invalid response from https://www.geronimostade.de/.well-known/acme-challenge/-rdUEV5djDFU7jHKm1QcTTEDa4ShmCSwHSVI63Wl6_8: 404",
        "status": 403
      },
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/357547420592/YFZiUA",
      "token": "-rdUEV5djDFU7jHKm1QcTTEDa4ShmCSwHSVI63Wl6_8",
      "validationRecord": [
        {
          "url": "http://www.geronimostade.de/.well-known/acme-challenge/-rdUEV5djDFU7jHKm1QcTTEDa4ShmCSwHSVI63Wl6_8",
          "hostname": "www.geronimostade.de",
          "port": "80",
          "addressesResolved": [
            "85.214.142.73"
          ],
          "addressUsed": "85.214.142.73"
        },
        {
          "url": "https://www.geronimostade.de/.well-known/acme-challenge/-rdUEV5djDFU7jHKm1QcTTEDa4ShmCSwHSVI63Wl6_8",
          "hostname": "www.geronimostade.de",
          "port": "443",
          "addressesResolved": [
            "85.214.142.73"
          ],
          "addressUsed": "85.214.142.73"
        }
      ],
      "validated": "2024-05-30T20:03:01Z"
    }
  ]
}
2024-05-30 22:03:04,681:DEBUG:acme.client:Storing nonce: cp09ZmYxBXqE-uTgSR3DsZd4uI3zPW7IQyXuozo3pFgmie9tV18
2024-05-30 22:03:04,682:DEBUG:certbot.reporter:Reporting to user: The following errors were reported by the server:

Domain: www.geronimostade.de
Type:   unauthorized
Detail: 85.214.142.73: Invalid response from https://www.geronimostade.de/.well-known/acme-challenge/-rdUEV5djDFU7jHKm1QcTTEDa4ShmCSwHSVI63Wl6_8: 404

To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address.
2024-05-30 22:03:04,683:DEBUG:certbot.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 82, in handle_authorizations
    self._respond(aauthzrs, resp, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 155, in _respond
    self._poll_challenges(aauthzrs, chall_update, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 226, in _poll_challenges
    raise errors.FailedChallenges(all_failed_achalls)
certbot.errors.FailedChallenges: Failed authorization procedure. www.geronimostade.de (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: 85.214.142.73: Invalid response from https://www.geronimostade.de/.well-known/acme-challenge/-rdUEV5djDFU7jHKm1QcTTEDa4ShmCSwHSVI63Wl6_8: 404

I am missing some fundamental knowledge here but is it possible that the http-01 challenge should get answer from http instead of https ?

 "detail": "85.214.142.73: Invalid response from https://www.geronimostade.de ... ???

From my side there is no redirect yet on that vhost.

Edit:
I solved the problem above.

I have just noticed my mistake.
The vhost config for geronimostade.de (non-www version) had wrong ServerName configured with www. :sneezing_face:

Just by isolating the logs for this post and by reviewing it step for step i was able to solve the issue.

However I have now run into another problem:

Failed redirect for www.geronimostade.de
Unable to set enhancement redirect for www.geronimostade.de

Which I assume I can fix somehow... Thanks anyway. And have an awesome weekend!

2 Likes

Hello @guitronimo, welcome to the Let's Encrypt community. :slightly_smiling_face:

The HTTP-01 challenge states
"Our implementation of the HTTP-01 challenge follows redirects, up to 10 redirects deep. It only accepts redirects to “http:” or “https:”, and only to ports 80 or 443. It does not accept redirects to IP addresses. When redirected to an HTTPS URL, it does not validate certificates (since this challenge is intended to bootstrap valid certificates, it may encounter self-signed or expired certificates along the way)."

Here is showing the redirection from HTTP to HTTPS, which is valid from the above.

$ curl -Ii http://www.geronimostade.de/.well-known/acme-challenge/sometestfile
HTTP/1.1 301 Moved Permanently
Date: Sat, 01 Jun 2024 01:40:52 GMT
Server: Apache/2.4.29 (Ubuntu)
Location: https://www.geronimostade.de/.well-known/acme-challenge/sometestfile
Content-Type: text/html; charset=iso-8859-1
3 Likes

Do you actually use ServerName inside the VirtualHost? Or are you just using the VirtualHost statement itself to lock into an IP?

Apache has both IP-based and Name-Based Virtual hosts and they are very different. Name-Based VHosts use SNI and are more typical
This is IP-based

<VirtualHost www.geronimostade.de:80>

Is not the same as this (name-based)

<VirtualHost *:80>
  ServerName www.geronimostade.de

I am not certain but Certbot may not treat an IP-based VirtualHost the same as far as adding a redirect clause.

Especially a version this ancient :slight_smile:

3 Likes

Thanks for your replies @Bruce5051 & @MikeMcQ

I must mention that i only have limited knowlege about all this stuff but i been doing it for a while.

My vhosts all contain ServerName and ServerAlias. I may have had stripped the config down for testing because of failing over and over again. All the vhosts are basically just copies from one of another with modified parameters. The issue in the end was on my side of course.
The non www. vhost geronimostade.de had wrong ServerName www.geronimostade.de configured... And until yesterday i would have believed that this wont be an issue at all. But i am glad i have at least learned something from 2 days of frustration.

Its working perfectly fine now accept that the www.geronimostade.de didnt want to be autoconfigured for redirect - yes probably because of then missing ServerName / ServerAlias due to messing around with the config.

Thanks for your services guys!

PS: I am aware that i am using an ancient version here. Should probably update this immediately.

4 Likes

Ubuntu 18.04 should easily support the recommended snap install. Just follow the below instructions carefully

4 Likes

also be upgraded unless you are paying Canonical for extended support. It went out of Long Tern Support in April 2023 and no longer receives any updates, including critical security updates, without a support contract with Canonical.

6 Likes
SSL certificate details for geronimostade.de:
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            04:34:50:d5:62:35:fb:c9:af:a8:96:90:e0:2d:b0:95:7b:62
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = Let's Encrypt, CN = R3
        Validity
            Not Before: May 31 19:33:11 2024 GMT
            Not After : Aug 29 19:33:10 2024 GMT
        Subject: CN = www.geronimostade.de

I'm sure I am missing something here...
in the vhost is there a:
<VirtualHost *:80>
ServerName ... APEX DOMAIN
ServerAlias www. ... SUB DOMAIN

The cert only shows "Subject: CN = www.geronimostade.de"

5 Likes

Configure them in the same vhost as suggested above.

5 Likes

Thanks for your replies,

yes i have to dig into a merged config. I`m aware of that. At the moment there is no wildcard for the domain at all. Its treated as two separate domains if you will, not ideal of course.
When stuff was non ssl i had the geronimostade.de running at the domain provider (safe space - including mail) and all the subdomains including www. went to another host. I sorta wanted to slowly grow the website (which then never happened) and keep the non-www as backup for downloads & private stuff, in case i screw my vps.

If i have understood correctly then merging both to one vhost allows to run only one certificate for domain.tld and www.domain.tld right?
Thats clearly not how it is now.

i suspect that it must be *geronimostade.de:80 then? Sorry if this is a dummy question. The host runs multiple websites. Wouldnt <VirtualHost *:80> match all requests?

2 Likes

You might want to read about IP-based and Name-Based VirtualHosts in the Apache docs (see link below)

Most systems work well relying on just name-based and SNI to match the ServerName and ServerAlias to match the VHost with the incoming URL. This is by far the more commonly used method.

So, yes <VirtualHost *:80> matches all IP addresses and selects the VHost based purely on the domain in the URL matching the ServerName or ServerAlias

When you do it like you did it looks like you are using a name in the VirtualHost statement but you are not. Apache translates that name at startup to an IP address.

https://httpd.apache.org/docs/current/vhosts/name-based.html

If you insist on using IP-based then be sure to read about these caveats
https://httpd.apache.org/docs/current/dns-caveats.html

4 Likes

Thanks for the Input Mike,

I am a bit confused now about ip based and name based vhosts - this webstuff is clearly not by daily business :smiley:

If i understood the links correct then i am not running an IP Based config. I just have parted the www. and non www. requests into two files - which can be merged to one actually.

geronimostade.de.conf:

<VirtualHost geronimostade.de:80>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html/web/geronimostade.de/

        ServerName geronimostade.de
        ServerAlias geronimostade.de
</VirtualHost>

www.geronimostade.de.conf:

<VirtualHost www.geronimostade.de:80>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html/web/geronimostade.de/

        ServerName www.geronimostade.de
        ServerAlias www.geronimostade.de
</VirtualHost>

I am just not sure how to merge them into one file in order to get only one certificate which contains both www. and non www but still staying name based. Would it look like this?

<VirtualHost *geronimostade.de:80>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html/web/geronimostade.de/

        ServerName geronimostade.de
        ServerAlias geronimostade.de
</VirtualHost>

<VirtualHost *geronimostade.de:80>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html/web/geronimostade.de/

        ServerName www.geronimostade.de
        ServerAlias www.geronimostade.de
</VirtualHost>

Sorry if thats probably not really a letsencrypt question but more apache cofig thing.

Have a great weekend!

1 Like

This doesn't make much sense. Why have identical ServerName and ServerAlias directives?

Wouldn't the following suffice?

<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html/web/geronimostade.de/

        ServerName geronimostade.de
        ServerAlias www.geronimostade.de
</VirtualHost>
2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.