Webroot /.well-known/acme-challenge/ configuration

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

[I have vyas.pro and vayu.vyas.pro working OK in https mode with their individual webroot created certificates from Lets encrypt it.
However, I cannot set for renewal as I get results given below.
My webroot is /srv/www/vyas/public and /srv/www/vayu.vyas/public,
It has /srv/www/vyas/public/.well-known/acme-challenge and /srv/www/vayu.vyas/public/.well-known/acme-challenge and is writable and readable
What is solution?

I shall give example of only vyas.pro, though there are similar issue with vayu.vyas.pro as well.]

I ran this command:
[sudo certbot --dry-run --webroot --webroot-path /srv/www/vyas/public -d vyas.pro]

It produced this output:
[--dry-run currently only works with the 'certonly' or 'renew' subcommands ('run')
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/certbot-log-67wmig1_/log or re-run Certbot with -v for more details.
spd@ml330:~$ sudo certbot certificates
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Found the following certs:
Certificate Name: vayu.vyas.pro
Serial Number: 40c415fe62b5d8461d7e524f6dc69b3e5fc
Key Type: ECDSA
Domains: vayu.vyas.pro
Expiry Date: 2024-06-22 18:20:35+00:00 (VALID: 87 days)
Certificate Path: /etc/letsencrypt/live/vayu.vyas.pro/fullchain.pem
Private Key Path: /etc/letsencrypt/live/vayu.vyas.pro/privkey.pem
Certificate Name: vyas.pro
Serial Number: 344a47d6e512e03037cf27bb4da40b5e6df
Key Type: ECDSA
Domains: vyas.pro
Expiry Date: 2024-06-22 18:19:54+00:00 (VALID: 87 days)
Certificate Path: /etc/letsencrypt/live/vyas.pro/fullchain.pem
Private Key Path: /etc/letsencrypt/live/vyas.pro/privkey.pem

spd@ml330:~$ sudo certbot certonly --dry-run --webroot --webroot-path /srv/www/vyas/public -d vyas.pro
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Simulating renewal of an existing certificate for vyas.pro

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: vyas.pro
Type: unauthorized
Detail: 79.77.23.86: Invalid response from http://vyas.pro/.well-known/acme-challenge/ZkW9Muxuux83qg1Ahmgs23BbbYsUaqPnxIg5DydNEl4: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with v for more details.]

Giving output of the first command-
As above.

My web server is (include version):
[Apache/2.4.55 (Ubuntu)]

The operating system my web server runs on is (include version):
[Debian Linux Bookworm]

My hosting provider, if applicable, is:
[I run my own web servers]

I can login to a root shell on my machine (yes or no, or I don't know):
[Yes, I can login as su and sudo ]

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
[No cPanel]

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
[certbot 2.9.0] and [sudo: certboot-auto: command not found]

Thanks

2

Hello @spd here is a list of issued certificates crt.sh | vyas.pro
Testing and debugging are best done using the Staging Environment as the Rate Limits are much higher.
All of these are very recent

image
image847×513 5.4 KB

1 Like
3

Also HTTPS and HTTP have different Apache versions.

HTTPS - Server: Apache/2.4.55 (Ubuntu)

$ curl -k -Ii https://vyas.pro/.well-known/acme-challenge/sometestfile
HTTP/1.1 404 Not Found
Date: Tue, 26 Mar 2024 22:18:06 GMT
Server: Apache/2.4.55 (Ubuntu)
Content-Type: text/html; charset=iso-8859-1

HTTP - Server: Apache/2.4.57 (Debian)

$ curl -Ii http://vyas.pro/.well-known/acme-challenge/sometestfile
HTTP/1.1 404 Not Found
Date: Tue, 26 Mar 2024 21:15:23 GMT
Server: Apache/2.4.57 (Debian)
Content-Type: text/html; charset=iso-8859-1
1 Like
4

Xubuntu installed its own version to begin with. While trying to figure out source of errors, I had to completely purge apache2 and some other apps. And then installed using sudo apt (rather than snap).

This is the current output.
Server version: Apache/2.4.55 (Ubuntu)
Server built: 2023-10-26T13:37:01

Any remedy for the problem?

Will it cause

5

Kindly wait to see if there are more knowledgeable Let's Encrypt community volunteers willing to assist.

1 Like
6

Just some added efforts.

I can write in to /.well-known/acme-challenge/ using following commands-
echo "test" >> /srv/www/vyas/public/.well-known/acme-challenge/test.txt

And cannot access using http as follows-
curl -Ii http://vyas.pro/.well-known/acme-challenge/test.txt
HTTP/1.1 404 Not Found
Date: Tue, 26 Mar 2024 22:12:03 GMT
Server: Apache/2.4.57 (Debian)
Content-Type: text/html; charset=iso-8859-1

But can acces it using https as follows-
curl -Ii https://vyas.pro/.well-known/acme-challenge/test.txt
HTTP/1.1 200 OK
Date: Tue, 26 Mar 2024 23:15:02 GMT
Server: Apache/2.4.55 (Ubuntu)
Upgrade: h2
Connection: Upgrade
Last-Modified: Tue, 26 Mar 2024 23:14:41 GMT
ETag: "f-61498747443d5"
Accept-Ranges: bytes
Content-Length: 15
Content-Type: text/plain

So does certbot uses http or https to read the written file?

7

So it would appear you are running two distinct copies of Apache, probably each running its own configuration.

1 Like
8

The HTTP-01 challenge, of the Challenge Types - Let's Encrypt, states "The HTTP-01 challenge can only be done on port 80."
Best Practice - Keep Port 80 Open

1 Like
9

I'd begin at the beginning...

What shows?:
sudo apachectl -t -D DUMP_VHOSTS

3 Likes
10

Here is the output.

VirtualHost configuration:

*:443	is a NameVirtualHost
         default server vyas.pro (/etc/apache2/sites-enabled/vyas-ssl.conf:11)
         port 443 namevhost vyas.pro (/etc/apache2/sites-enabled/vyas-ssl.conf:11)
         port 443 namevhost vayu.vyas.pro (/etc/apache2/sites-enabled/vyas-ssl.conf:64)
         
*:80	is a NameVirtualHost
         default server vyas.pro (/etc/apache2/sites-available/vyas.conf:1)
         port 80 namevhost vyas.pro (/etc/apache2/sites-available/vyas.conf:1)
         port 80 namevhost vayu.vyas.pro (/etc/apache2/sites-available/vyas.conf:23)
         port 80 namevhost vyas.pro (/etc/apache2/sites-enabled/vyas-ssl.conf:56)
         port 80 namevhost vayu.vyas.pro (/etc/apache2/sites-enabled/vyas-ssl.conf:109)
         port 80 namevhost vyas.pro (/etc/apache2/sites-enabled/vyas.conf:1)
         port 80 namevhost vayu.vyas.pro (/etc/apache2/sites-enabled/vyas.conf:23)
#----------------------

Can you please help in interpreting it?
Thanks

1 Like
11

I still see 2 different Apache instances running, one on HTTPS and the other on HTTP

HTTPS - Server: Apache/2.4.55 (Ubuntu)

$ curl -k -Ii https://vyas.pro/.well-known/acme-challenge/sometestfile

HTTP/1.1 404 Not Found
Date: Wed, 27 Mar 2024 18:11:32 GMT
Server: Apache/2.4.55 (Ubuntu)
Content-Type: text/html; charset=iso-8859-1

HTTP - Server: Apache/2.4.57 (Debian)

$ curl -Ii http://vyas.pro/.well-known/acme-challenge/sometestfile
HTTP/1.1 404 Not Found
Date: Wed, 27 Mar 2024 17:08:48 GMT
Server: Apache/2.4.57 (Debian)
Content-Type: text/html; charset=iso-8859-1
2 Likes
12

From @Bruce5051 info are you forwarding port 80 to one server but port 443 to a different one?

Also, you have the same domain names listed more than once. It looks like you might have an include for both sites-available and sites-enabled in your main apache conf file. Normally only the sites-enabled is included.

See these names

You have even further duplication in the below two files. It is unusual to have a port 80 VirtualHost defined in a file with ssl in its name. It is not illegal but maybe helps point to what went wrong.

Keep repeating the sudo apachectl -t -D DUMP_VHOSTS command until you have each name showing only once for each port.

3 Likes
13

I can see mention of Ubuntu version in Synaptics, but I can't Apache2 2.4.57 version anywhere. I have n idea how to remove or purge Apache2 2.4.57. The router allows port 80 and 443 to the same local ip running this (these) servers virtual hosting. I can a2dissite vyas.conf (hhtp) or vyas-ssl.conf (https). but it did not make any difference. I have tried using ChatGPT with no success. ?

14 
#----------------------------
sudo apachectl -t -D DUMP_VHOSTS

VirtualHost configuration:

*:443                  is a NameVirtualHost
         default server vyas.pro (/etc/apache2/sites-enabled/vyas-ssl.conf:11)
         port 443 namevhost vyas.pro (/etc/apache2/sites-enabled/vyas-ssl.conf:11)
         port 443 namevhost vayu.vyas.pro (/etc/apache2/sites-enabled/vyas-ssl.conf:64)

*:80                   is a NameVirtualHost
         default server vyas.pro (/etc/apache2/sites-enabled/vyas-ssl.conf:56)
         port 80 namevhost vyas.pro (/etc/apache2/sites-enabled/vyas-ssl.conf:56)
         port 80 namevhost vayu.vyas.pro (/etc/apache2/sites-enabled/vyas-ssl.conf:109)
#----------------------------

Now vyas.pro and vayu.vyas.pro each has one port 443 and one port 80 (to redirect to https) assigned.
Still dry-run "sudo certbot certonly --dry-run --webroot --webroot-path /srv/www/vyas/public -d vyas.pro" failed.

15

Can you show the error message?

Also, the Apache replying to HTTP requests is a Debian system. Is that where you are running Certbot from?

curl -i http://vyas.pro/.well-known/acme-challenge/Test404
HTTP/1.1 404 Not Found
Server: Apache/2.4.57 (Debian)
3 Likes
16 
#-----------------------
curl -I -L -k -X GET http://vyas.pro/.well-known/acme-challenge/test.txt
HTTP/1.1 404 Not Found
Date: Wed, 27 Mar 2024 19:07:35 GMT
Server: Apache/2.4.57 (Debian)
Content-Length: 270
Content-Type: text/html; charset=iso-8859-1

#-----------------------
curl -I -L -k -X GET https://vyas.pro/.well-known/acme-challenge/test.txt
HTTP/1.1 200 OK
Date: Wed, 27 Mar 2024 20:10:53 GMT
Server: Apache/2.4.55 (Ubuntu)
Upgrade: h2
Connection: Upgrade
Last-Modified: Wed, 27 Mar 2024 00:24:44 GMT
ETag: "5-614996ef966cd"
Accept-Ranges: bytes
Content-Length: 5
Content-Type: text/plain
#-----------------------

You can see here, it can read and write on https (Debian), but not on http (UBUNTU). And I can't find / see Apache2 (debian) to purge it. Any suggestion?

#-----------------------
Here is the error message from dry-run

sudo certbot certonly --dry-run --webroot --webroot-path /srv/www/vyas/public -d vyas.pro

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Simulating renewal of an existing certificate for vyas.pro

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: vyas.pro
  Type:   unauthorized
  Detail: 79.77.23.86: Invalid response from http://vyas.pro/.well-known/acme-challenge/vR0KVcYemxcmGfVXpjtakWezLlX3tRSU1THHpJsfJfc: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
#-----------------------

Thanks

17

I don't understand that. You did not show using httpS with the Debian response only http.

When you say "write" do you mean you can place the file and find it on the Ubuntu system but don't know how to update the Debian one?

Do you have a Debian system somewhere? Maybe in a container or VM of some kind?

If you don't have a Debian system maybe it is on some equipment provided by your ISP. It could be intercepting the HTTP request coming from the Let's Encrypt server. Is there some kind of router?

Or do you have some other kind of proxy software somewhere?

An HTTP request for your "home" page just gets the default Apache Debian page. It must be coming from somewhere

curl -i http://vyas.pro
HTTP/1.1 200 OK
Server: Apache/2.4.57 (Debian)
Last-Modified: Thu, 14 Sep 2023 14:48:59 GMT

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>Apache2 Debian Default Page: It works</title>
2 Likes
18

[quote="MikeMcQ, post:17, topic:215456"]

Dear @MikeMcQ.

I run the these server only part time when I am working on them, usually in the evening, Debian or xubuntu with apache2.

text.txt has been written in http://vyas.pro/.well-known/acme-challenge/test.txt
usimg command echo "test" >> /srv/www/vyas/public/.well-known/acme-challenge/test.txt
and it can be read using following command

curl -I -L -k -X GET https://vyas.pro/.well-known/acme-challenge/test.txt
HTTP/1.1 200 OK
Date: Thu, 28 Mar 2024 17:51:14 GMT
Server: Apache/2.4.55 (Ubuntu)
Upgrade: h2
Connection: Upgrade
Last-Modified: Thu, 28 Mar 2024 01:37:36 GMT
ETag: "a-614ae9160d74e"
Accept-Ranges: bytes
Content-Length: 10
Content-Type: text/plain

#but cant be read if use http option as below
curl -I -L -k -X GET http://vyas.pro/.well-known/acme-challenge/test.txt
HTTP/1.1 404 Not Found
Date: Thu, 28 Mar 2024 16:48:13 GMT
Server: Apache/2.4.57 (Debian)
Content-Length: 270
Content-Type: text/html; charset=iso-8859-1

I hope it is clear, only difference of https (Ubuntu) and http (Debian), one succeeds and other does not reading the text.

May I suggest you give me a complete of example of multi-domain virtualhosting (port:80, Debian Bookwork) in /serv/www/example.com and /serv/www/sub.example.com with their respective webroot at /serv/www/example.com/./well-known/acme-challenge/ and /serv/www/sub.example.com/./well-known/acme-challenge/

And I shall purge apaches, certbot etc as on letsencrypt - How to completely remove "certbot" from ubuntu? - Ask Ubuntu or site you provide, and I shall try to get new certificates and see if that work.

Thanks for your help in advance.

1 Like
19

Do you mean both need to have a cert but that only one of those systems runs at one time? That is do you literally mean "OR"

Because sometimes people want two systems running at the same time using a reverse proxy or other methods. It would help to understand what you are trying to achieve.

I am an unpaid volunteer offering my time and experience for free to you. Most of the other helpers here are the same.

There are many ways to organize servers. I don't have the time or desire to explain all options to you - or anyone really :slight_smile:

I am happy to offer help as my time permits to work through specific problems with you. But asking for a general outline of many options is too much for me.

2 Likes
20

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.