Docker Run Certbot urn:acme:error:unauthorized

jasperf · January 6, 2019, 6:49am

Trying to get an LE SSL Certificate running a Laradock Certbot setup.

My domain is:

larastud.io

Command

I run this command
docker-compose -f prod-docker-compose.yml up certbot from the Ubuntu host to run the Certbot script on the Docker image using a basic Ubuntu image. This so I can use an SSL certificate for the domain larastud.io which is run on a Laradock Docker containers setup.

Output

It produced this output:

docker-compose -f prod-docker-compose.yml up certbot
Starting laradock_certbot_1 ... done
Attaching to laradock_certbot_1
certbot_1     | Failed authorization procedure. larastud.io (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://larastud.io/.well-known/acme-challenge/P-aR8--JpAPJEHP-jhkFT3KWFwteX0kHAX1B5U3_5m0: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body>\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>nginx</center>\r\n"
certbot_1     | IMPORTANT NOTES:
certbot_1     |  - The following errors were reported by the server:
certbot_1     |
certbot_1     |    Domain: larastud.io
certbot_1     |    Type:   unauthorized
certbot_1     |    Detail: Invalid response from http://larastud.io/.well-known/acme-
certbot_1     |    challenge/P-aR8--JpAPJEHP-jhkFT3KWFwteX0kHAX1B5U3_5m0:
certbot_1     |    "<html>\r\n<head><title>404 Not
certbot_1     |    Found</title></head>\r\n<body>\r\n<center><h1>404 Not
certbot_1     |    Found</h1></center>\r\n<hr><center>nginx</center>\r\n"
certbot_1     |
certbot_1     |    To fix these errors, please make sure that your domain name was
certbot_1     |    entered correctly and the DNS A record(s) for that domain
certbot_1     |    contain(s) the right IP address.
certbot_1     | cp: cannot stat '/etc/letsencrypt/archive/larastud.io/cert1.pem': No such file or directory
certbot_1     | cp: cannot stat '/etc/letsencrypt/archive/larastud.io/privkey1.pem': No such file or directory

My web server is Digital Ocean
The operating system my web server runs on is (include version): Ubuntu 18.0.4 Droplet with Docker, Docker Compose and UFW
My hosting provider, if applicable, is:
I can login to a root shell on my machine (yes or no, or I don’t know): Yes
I’m using a control panel to manage my site (no, or provide the name and version of the control panel):No

Any idea what is going on? Is it true Certbot should create the letsencrypt folder in my application root as well as the .well-known? Script is run using:

#!/bin/bash

letsencrypt certonly --webroot -w /var/www/letsencrypt -d "$CN" --agree-tos --email "$EMAIL" --non-interactive --text

cp /etc/letsencrypt/archive/"$CN"/cert1.pem /var/certs/cert1.pem
cp /etc/letsencrypt/archive/"$CN"/privkey1.pem /var/certs/privkey1.pem

and as you see as root. It is run based on this Dockerfile:

FROM phusion/baseimage:latest

LABEL maintainer="Mahmoud Zalt <mahmoud@zalt.me>"

COPY run-certbot.sh /root/certbot/run-certbot.sh

RUN apt-get update
RUN apt-get install -y letsencrypt

ENTRYPOINT bash -c "bash /root/certbot/run-certbot.sh && sleep infinity"

So why would there be an authorization issue?

JuergenAuer · January 6, 2019, 11:11am

Hi @jasperf

I don't know enough about such a docker-setup. But is there a rewrite rule or something else, so /.well-known/acme-challenge/1234 works with this directory?

I see, you have tested your domain. The http status 404 with

http://larastud.io/.well-known/acme-challenge/1234

is ok, if the file doesn't exist. Perhaps create a file in /var/www/letsencrypt and try to load this file via browser.

Or use your "standard webroot" without such a rewrite.

jasperf · January 7, 2019, 6:23am

@JuergenAuer I changed the webroot to /var/www/public/letsencrypt and so added public to it as Laravel loads stuff from there. I can load http://larastud.io/letsencrypt/test.txt inside /var/www/public/letsencrypt which I made myself (directory and file created by me).

Then I still got an error though. So I changed default.conf to represent this change as well:

server {

    listen 80 default_server;
    listen [::]:80 default_server ipv6only=on;

    # For https
    listen 443 ssl default_server;
    listen [::]:443 ssl default_server ipv6only=on;
    ssl_certificate /etc/nginx/ssl/default.crt;
    ssl_certificate_key /etc/nginx/ssl/default.key;

    server_name localhost;
    root /var/www/public;
    index index.php index.html index.htm;

    location / {
         try_files $uri $uri/ /index.php$is_args$args;
    }

    location ~ \.php$ {
        try_files $uri /index.php =404;
        fastcgi_pass php-upstream;
        fastcgi_index index.php;
        fastcgi_buffers 16 16k;
        fastcgi_buffer_size 32k;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        #fixes timeouts
        fastcgi_read_timeout 600;
        include fastcgi_params;
    }

    location ~ /\.ht {
        deny all;
    }

    location /.well-known/acme-challenge/ {
        root /var/www/public/letsencrypt/;
        log_not_found off;
    }
}

still no joy. Perhaps its an issue with certbot in one container dealing with http access from another container running nginx… Do not think so though as they are on the same network.

Question: Does certbot normally create the letsencrypt and or .well-known directories?

schoen · January 7, 2019, 6:33am

Yes, it creates .well-known in your webroot (that you specified with -w) and it creates /etc/letsencrypt under various circumstances. There is no such thing as letsencrypt in your webroot as part of the validation process (this uses the path /.well-known/acme-challenge).

Can you try manually creating a file /.well-known/acme-challenge/test2.txt inside /var/www/public (i.e., /var/www/public/.well-known/acme-challenge/test2.txt) and see if you can load that over the web? (the /var/www/public should be from the point of view of the same Docker environment where you are running the Certbot application)

jasperf · January 7, 2019, 6:38am

Hmm. Do have /var/www/public/.well-known/acme-challenge now and added test2.txt but Nginx still gives me a 404 on https://larastud.io/.well-known/acme-challenge/test2.txt . So perhaps it is an issue with a directory with a . and my current nginx conf I pasted here earlier...

schoen · January 7, 2019, 6:43am

Oh, I didn't notice the bottom part!

    location /.well-known/acme-challenge/ {
    root /var/www/public/letsencrypt/;

This could cause trouble because Certbot expects to create .well-known/acme-challenge itself within the directory you specify. Maybe that's what you were asking about before.

You might want to look at

for the distinction between root and alias in nginx (although that particular person is using a different client application which may have a slightly different concept of webroot). You might need alias rather than root in your configuration here.

jasperf · January 7, 2019, 7:08am

Well I am still learning the Docker as well as LE SSL ropes. The url http://larastud.io/.well-known/acme-challenge/test2.txt does now load with me using

location ~ /.well-known {
        allow all;
        root /var/www/public;
    }

However the same error occurs with that:

ttaching to laradock_certbot_1
certbot_1     | Failed authorization procedure. larastud.io (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://larastud.io/.well-known/acme-challenge/AdUDLAsUKSrA_B2EnMvE2Igmmg6_RrsbdfpkMThbdRg: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body>\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>nginx</center>\r\n"
certbot_1     | IMPORTANT NOTES:
certbot_1     |  - The following errors were reported by the server:
certbot_1     |
certbot_1     |    Domain: larastud.io
certbot_1     |    Type:   unauthorized
certbot_1     |    Detail: Invalid response from http://larastud.io/.well-known/acme-
certbot_1     |    challenge/AdUDLAsUKSrA_B2EnMvE2Igmmg6_RrsbdfpkMThbdRg:
certbot_1     |    "<html>\r\n<head><title>404 Not
certbot_1     |    Found</title></head>\r\n<body>\r\n<center><h1>404 Not
certbot_1     |    Found</h1></center>\r\n<hr><center>nginx</center>\r\n"
certbot_1     |
certbot_1     |    To fix these errors, please make sure that your domain name was
certbot_1     |    entered correctly and the DNS A record(s) for that domain
certbot_1     |    contain(s) the right IP address.
certbot_1     | cp: cannot stat '/etc/letsencrypt/archive/larastud.io/cert1.pem': No such file or directory
certbot_1     | cp: cannot stat '/etc/letsencrypt/archive/larastud.io/privkey1.pem': No such file or directory

Even with letsencrypt certonly --webroot -w /var/www/public:

web@laradock-setup:~/lsdock$ docker-compose -f prod-docker-compose.yml up certbot
Starting laradock_certbot_1 ... done
Attaching to laradock_certbot_1
certbot_1     | The webroot plugin is not working; there may be problems with your existing configuration.
certbot_1     | The error was: PluginError('/var/www/public does not exist or is not a directory',)
certbot_1     | cp: cannot stat '/etc/letsencrypt/archive/larastud.io/cert1.pem': No such file or directory
certbot_1     | cp: cannot stat '/etc/letsencrypt/archive/larastud.io/privkey1.pem': No such file or directory
laradock_certbot_1 exited with code 1

So the webroot setup may be different here then. Perhaps because we have a Docker webroot and host Ubuntu webroot… Not sure yet…

JuergenAuer · January 7, 2019, 9:42am

But this file

http://larastud.io/letsencrypt/test.txt

is irrelevant. You must be able to load a file via

http://larastud.io/.well-known/acme-challenge/1234

The webroot is very simple. But it's complicated if there are redirects or internal proxies.

jasperf · January 8, 2019, 4:34am

Really hard stuff dealing with a host, two containers that depend on each other and so on. In the end I followed https://www.digitalocean.com/community/tutorials/how-to-secure-a-containerized-node-js-application-with-nginx-let-s-encrypt-and-docker-compose and used several volumes including a named volume that loads data from the host using

web-root:
    driver: local
    driver_opts:
      type: none
      device: /home/web/lslaravel/public/
      o: bind

and

### Certbot #########################################
    certbot:
      image: certbot/certbot
      container_name: certbot
      volumes:
        - certbot-etc:/etc/letsencrypt
        - certbot-var:/var/lib/letsencrypt
        - web-root:/var/www/html
      depends_on:
        - nginx
      command: certonly --webroot --webroot-path=/var/www/html --email nowayjosé@gmail.com --agree-tos --no-eff-email --staging -d larastud.io

for the new Certbot container.

Also had to adjust Nginx container to load the volumes needed and have both containers depend on each other which was not the case before. All is in progress at https://github.com/Larastudio/lsdock .

Now I need to work out the proper certs loading, http and https block. But I was really happy to see this using staging mode:

certbot       | Saving debug log to /var/log/letsencrypt/letsencrypt.log
certbot       | Plugins selected: Authenticator webroot, Installer None
certbot       | Obtaining a new certificate
certbot       | Performing the following challenges:
certbot       | http-01 challenge for larastud.io
certbot       | Using the webroot path /var/www/html for all unmatched domains.
certbot       | Waiting for verification...
nginx_1       | 192.168.112.1 - - [08/Jan/2019:04:30:04 +0000] "GET /.well-known/acme-challenge/RQQvKc8LI4-UV8pUjTRP83EV5x0lxw9vkA3BmxJD2LA HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
nginx_1       | 192.168.112.1 - - [08/Jan/2019:04:30:05 +0000] "GET /.well-known/acme-challenge/RQQvKc8LI4-UV8pUjTRP83EV5x0lxw9vkA3BmxJD2LA HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
nginx_1       | 192.168.112.1 - - [08/Jan/2019:04:30:05 +0000] "GET /.well-known/acme-challenge/RQQvKc8LI4-UV8pUjTRP83EV5x0lxw9vkA3BmxJD2LA HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
certbot       | Cleaning up challenges
certbot       | IMPORTANT NOTES:
certbot       |  - Congratulations! Your certificate and chain have been saved at:
certbot       |    /etc/letsencrypt/live/larastud.io/fullchain.pem
certbot       |    Your key file has been saved at:
certbot       |    /etc/letsencrypt/live/larastud.io/privkey.pem
certbot       |    Your cert will expire on 2019-04-08. To obtain a new or tweaked
certbot       |    version of this certificate in the future, simply run certbot
certbot       |    again. To non-interactively renew *all* of your certificates, run
certbot       |    "certbot renew"
certbot       |  - Your account credentials have been saved in your Certbot
certbot       |    configuration directory at /etc/letsencrypt. You should make a
certbot       |    secure backup of this folder now. This configuration directory will
certbot       |    also contain certificates and private keys obtained by Certbot so
certbot       |    making regular backups of this folder is ideal.

and certs have been added:

docker-compose exec nginx ls -la /etc/letsencrypt/live
total 16
drwx------    3 root     root          4096 Jan  8 04:30 .
drwxr-xr-x    9 root     root          4096 Jan  8 04:40 ..
-rw-r--r--    1 root     root           740 Jan  8 04:30 README
drwxr-xr-x    2 root     root          4096 Jan  8 04:30 larastud.io

will update this thread once I made progress loading the certs properly .

system · February 7, 2019, 4:34am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Certbot failing acme-challenge (unauthorized) Help	7	11444	July 16, 2021
Docker Certbot Connection Refused 2 ways Help	23	1728	March 29, 2023
LetsEncrypt Cannot Obtain Certificate Help	40	2940	March 13, 2019
Certbot failed to authenticate Help	5	695	November 5, 2021
Certbot failed to authenticate some domains Help	34	1662	February 28, 2023

Docker Run Certbot urn:acme:error:unauthorized

Command

Output

Related topics