Using wrong domain for http request to .well-known

My domain is: artfulrobot.uk

I ran this command:

certbot --authenticator nginx --installer nginx \
  --email redacted@artfulrobot.uk --agree-tos  \
  -d artfulrobot.uk -d www.artfulrobot.uk

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log

Please choose an account
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: ar3.artfulrobot.uk@2016-09-23T15:39:02Z (d472)
2: ar7.artfulrobot.uk@2025-06-20T15:34:37Z (230f)
3: ar5.artfulrobot.uk@2016-02-22T14:59:32Z (62e1)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-3] then [enter] (press 'c' to cancel): 2
Requesting a certificate for artfulrobot.uk and www.artfulrobot.uk

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
  Domain: artfulrobot.uk
  Type:   unauthorized
  Detail: 2a01:4f8:c17:b03b::1: Invalid response from https://obs2.artfulrobot.uk/.well-known/acme-challenge/8vSAF1-ywwDtPyP3-Zfj1Bg0D54KUic3_tbQUGdngG0: 404

  Domain: www.artfulrobot.uk
  Type:   unauthorized
  Detail: 2a01:4f8:c17:b03b::1: Invalid response from https://obs2.artfulrobot.uk/.well-known/acme-challenge/teZWsfamwSFIoirf_5oTgd55xwCLDbJF5f4EqiJOcxE: 404

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

My web server is (include version): nginx 1.22

The operating system my web server runs on is (include version): Debian 12

My hosting provider, if applicable, is: Hetzner

I have root shell access, I do not use a UI.

The version of my client is: certbot 2.1.0

Note it is trying to fetch from a subdomain at obs2. !

Background: I have moved stuff around between servers a bit, most recently moving /etc/letsencrypt from a host called ar3.artfulrobot.uk to one called ar7.artfulrobot.uk.

DNS A records both 4 and 6 are correct for the artfulrobot.uk domain.

This usually implies a redirect is setup that goes to that domain.

2 Likes

Welcome @nettie

It is unusual to see a 404 (Not Found) error using the --nginx authenticator. We also should not be seeing an HTTPS: URL in the message as --nginx plugin causes a reply to the original HTTP request to your server. Seeing HTTPS means the HTTP request got redirected which doesn't happen when --nginx worked correctly.

Would you show the server block for port 80 for these two domains?

6 Likes

@MikeMcQ @webprofusion thank you both for your time.

I'm embarrassed to say I eventually found the problem ipv6!

Specifically, on this host nginx was only listening on ipv6 to a couple of domains, and not the ones I was trying to renew.

This caused a situation where if letsencrypt used ipv4, it worked, but if it used ipv6, nginx hunted for a server block listening on ipv6 with a matching server_name and came up blank, so fell back to the last/first one it found for ipv6 (I'm not sure that's a sensible default but there you go).

Now I have nginx listening on listen [::]:80 (and the ssl one) then the renewals are working.

Drove me into a bit of a panic for couple of hours though!

Hopefully future readers can either enjoy laughing at me, or perhaps my confession might just be helpful to someone else too. :slight_smile:

3 Likes

Yes, you got that exactly right.

For future reference the Certbot log in /var/log/letsencrypt shows the exact changes it makes to your nginx config for the HTTP challenge. It often becomes clear what went wrong if you know enough about nginx (and I think you do).

Yes, that is the normal nginx behavior. It first filters on listening criteria and within those candidates it matches on domain name. Listening criteria can also include specific IP for the rare cases that need it. And, yes, while versatile can easily lead to results that are not intuitive :slight_smile:

When no domain name matches it should use the first server block satisfying the listen criteria unless one of the qualifying listen statements has default_server set. Also see: How nginx processes a request

Anyway, good work finding that on your own. Just wanted to add some color. Cheers

4 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.