Renew failure with certbot or letsencrypt-auto

Help
1

My domain is: dev.stephane-huc.net

I ran this command: certbot renew --cert-name dev.stephane-huc.net --dry-run

It produced this output:

Processing /etc/letsencrypt/renewal/dev.stephane-huc.net.conf

Cert is due for renewal, auto-renewing...
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for dev.stephane-huc.net
Waiting for verification...
Cleaning up challenges
Unable to clean up challenge directory /srv/www/dev.stephane-huc.net/www/.well-known/acme-challenge
Attempting to renew cert from /etc/letsencrypt/renewal/dev.stephane-huc.net.conf produced an unexpected error: Failed authorization procedure. dev.stephane-huc.net (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://dev.stephane-huc.net/.well-known/acme-challenge/4u3UDymEci7--oXKHkHnNMi3dFUxIdmielM_IW-zR3o: Timeout. Skipping.
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates below have not been saved.)

All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/dev.stephane-huc.net/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates above have not been saved.)
1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

  • The following errors were reported by the server:

Domain: dev.stephane-huc.net
Type: connection
Detail: Fetching
http://dev.stephane-huc.net/.well-known/acme-challenge/4u3UDymEci7--oXKHkHnNMi3dFUxIdmielM_IW-zR3o:
Timeout

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A record(s) for that domain
contain(s) the right IP address. Additionally, please check that
your computer has a publicly routable IP address and that no
firewalls are preventing the server from communicating with the
client. If you're using the webroot plugin, you should also verify
that you are serving files from the webroot path you provided.

My web server is (include version): Nginx (1.12.0-1)

The operating system my web server runs on is (include version): Ubuntu Xenial

My hosting provider, if applicable, is: @home

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

I resolved error 404. (https://dev.stephane-huc.net/.well-known/acme-challenge/test <= success)
If i watch the directory 'acme-challenge' by 'watch -n1 ls', i seen the file created and deleted.

If i attempt with letsencrypt tool (updated by git), i've another error.

Command: /opt/letsencrypt/letsencrypt-auto certonly --renew-by-default --config "${file_cfg_le}"

File config content:

# Prefer to use a 4096 bit RSA key instead of 2048
rsa-key-size = 4096

email = webmaster@stephane-huc.net
domains = dev.stephane-huc.net

authenticator = webroot

# This is the webroot directory of your domain in which
# letsencrypt will write a hash in /.well-known/acme-challenge directory.

webroot-path = /srv/www/dev.stephane-huc.net/www/

The output:

Renewing an existing certificate
Performing the following challenges:
http-01 challenge for dev.stephane-huc.net
Using the webroot path /srv/www/dev.stephane-huc.net/www for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Unable to clean up challenge directory /srv/www/dev.stephane-huc.net/www/.well-known/acme-challenge
Failed authorization procedure. dev.stephane-huc.net (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Could not connect to dev.stephane-huc.net

IMPORTANT NOTES:

  • The following errors were reported by the server:

Domain: dev.stephane-huc.net
Type: connection
Detail: Could not connect to dev.stephane-huc.net

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A record(s) for that domain
contain(s) the right IP address. Additionally, please check that
your computer has a publicly routable IP address and that no
firewalls are preventing the server from communicating with the
client. If you're using the webroot plugin, you should also verify
that you are serving files from the webroot path you provided.

But both commands launched with rights root!

2

Hi @hucste,

You offer the IPv6 address 2001:470:cc33:147::3 for your site, but don’t accept connections there. You only accept connections on the IPv4 address 78.237.104.116. The certificate authority is validating over IPv6 because of your AAAA record.

3

But, i configured nginx with:

listen 443 ssl http2;
listen [::]:443 ssl http2;

And my ip6tables:

-A INPUT -i em0 -p tcp -m multiport --dports 80,443 -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -m recent --set --name HTTP --mask ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff --rsource
-A INPUT -i em0 -p tcp -m multiport --dports 80,443 -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -m recent --update --seconds 60 --hitcount 10 --name HTTP --mask ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff --rsource -j LOG --log-prefix "IPv6 BAD HTTP: "
-A INPUT -i em0 -p tcp -m multiport --dports 80,443 -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -m recent --update --seconds 60 --hitcount 10 --name HTTP --mask ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff --rsource -j logbad6
-A INPUT -i em0 -p tcp -m multiport --dports 80,443 -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -m limit --limit 909/sec -m comment --comment "HTTP,HTTPS" -j ACCEPT

It’s egual if i stop ip6tables!

4

The validation method you’re using, called HTTP-01, requires an initial connection by HTTP on port 80 in order to prove your control over the domain name.

5

i think it’s the case:

Nginx config:

server {
    listen 80;
    listen [::]:80;

    server_name dev.stephane-huc.net;

    include /etc/nginx/dir_cfg/well_known.cfg;

    return 301 https://$host$request_uri;
}

server {

    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    ###########
    ### SSL cfguration
    ###########
    include /etc/nginx/dir_ssl/ssl.cfg;

    server_name dev.stephane-huc.net;

    root /dir_www/dev.stephane-huc.net/;

...
}

It’s not sufficient?

$ sudo netstat -nlp | grep nginx
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      37144/nginx -g daem
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      37144/nginx -g daem
tcp6       0      0 :::80                   :::*                    LISTEN      37144/nginx -g daem
tcp6       0      0 :::443                  :::*                    LISTEN      37144/nginx -g daem

Really, i dont known my problem!?

6

Although I’m not really familiar with ip6tables, I suspect the problem is somewhere outside of your own machine, like in a router or a firewall that’s preventing IPv6 connections to your host.

7

My firewall is OPNSense (managing both IPv4&6), and i use a Tunnel HE to get IPv6 functionnal! :wink:

Do you think it’s better to change my records dns to shut down ipv6?

8

TRY:
server {
listen 80;
listen [::]:80;
server_name dev.stephane-huc.net;
location /.well-known {
root /well-known/folder/path;
allow all
}
include /etc/nginx/dir_cfg/well_known.cfg;
return 301 https://$host$request_uri;
}

You should be able to place a test.txt file at that folder and then access it from the Internet:
http://dev.stephane-huc.net/.well-known/acme-challenge/test.txt

9

into the include /etc/nginx/dir_cfg/well_known.cfg;:

location '/.well-known' {
        allow all;
        default_type "text/plain";
        root /dir_www/dev.stephane-huc.net;
        #try_files $uri /$1;
}
location '/.well-known/acme-challenge' {
        allow all;
        default_type "text/plain";
        root /dir_www/dev.stephane-huc.net;
        #try_files $uri /$1;
}

http://dev.stephane-huc.net/.well-known/acme-challenge/test
http://dev.stephane-huc.net/.well-known/acme-challenge/test.txt
both redirected on https :wink:

10

HTTPS redirection is NOT desired and explicitly intentionally averted with use of the LOCATION section.
But as you can see below the server is ignoring the exclusion:
[EDIT - removed all left carets for better readability]

telnet dev.stephane-huc.net 80
GET /.well-known/acme-challenge/test.txt HTTP/1.1
host: dev.stephane-huc.net

HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Wed, 14 Jun 2017 00:48:29 GMT
Content-Type: text/html
Content-Length: 178
Connection: keep-alive
Location: https://dev.stephane-huc.net/.well-known/acme-challenge/test.txt

html>
head>title>301 Moved Permanently/title>/head>
body bgcolor=“white”>
center>h1>301 Moved Permanently/h1>/center>
hr>center>nginx/center>
/body>
/html>

11

Maybe try removing the single quotes...???

12

It's due to directive:

I comment it, and restart nginx, and attempt with curl:

$ curl -Iv http://dev.stephane-huc.net/.well-known/acme-challenge/test.txt 
*   Trying 78.237.104.116...
* TCP_NODELAY set
* Connected to dev.stephane-huc.net (78.237.104.116) port 80 (#0)
> HEAD /.well-known/acme-challenge/test.txt HTTP/1.1
> Host: dev.stephane-huc.net
> User-Agent: curl/7.53.1
> Accept: */*
> 
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Server: nginx
Server: nginx
< Date: Wed, 14 Jun 2017 00:54:26 GMT
Date: Wed, 14 Jun 2017 00:54:26 GMT
< Content-Type: text/plain
Content-Type: text/plain
< Content-Length: 19
Content-Length: 19
< Last-Modified: Wed, 14 Jun 2017 00:43:48 GMT
Last-Modified: Wed, 14 Jun 2017 00:43:48 GMT
< Connection: keep-alive
Connection: keep-alive
< ETag: "594086c4-13"
ETag: "594086c4-13"
< Accept-Ranges: bytes
Accept-Ranges: bytes

< 
* Connection #0 to host dev.stephane-huc.net left intact
13

But the redirection is desired.
And also the LOCATION exclusion.
They should both be enabled at the same time.
That is, only requests to /.well-known/* would be allowed to remain as HTTP
All other requests would redirect to HTTPS.

14

Done, and same! :frowning:

I understand, but it's not, despite the config.

15

Connexions with curl in IPv6 seem succeeded!

$ curl -Iv6 dev.stephane-huc.net     
* Rebuilt URL to: dev.stephane-huc.net/
*   Trying 2001:470:cc33:147::3...
* TCP_NODELAY set
* Connected to dev.stephane-huc.net (2001:470:cc33:147::3) port 80 (#0)
> HEAD / HTTP/1.1
> Host: dev.stephane-huc.net
> User-Agent: curl/7.53.1
> Accept: */*
> 
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Server: nginx
Server: nginx
< Date: Wed, 14 Jun 2017 01:20:46 GMT
Date: Wed, 14 Jun 2017 01:20:46 GMT
< Content-Type: text/html
Content-Type: text/html
< Content-Length: 612
Content-Length: 612
< Last-Modified: Wed, 12 Apr 2017 14:46:01 GMT
Last-Modified: Wed, 12 Apr 2017 14:46:01 GMT
< Connection: keep-alive
Connection: keep-alive
< ETag: "58ee3da9-264"
ETag: "58ee3da9-264"
< Accept-Ranges: bytes
Accept-Ranges: bytes

< 
* Connection #0 to host dev.stephane-huc.net left intact
$ curl -Iv6 https://dev.stephane-huc.net
* Rebuilt URL to: https://dev.stephane-huc.net/
*   Trying 2001:470:cc33:147::3...
* TCP_NODELAY set
* Connected to dev.stephane-huc.net (2001:470:cc33:147::3) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=dev.stephane-huc.net
*  start date: Mar 16 10:16:00 2017 GMT
*  expire date: Jun 14 10:16:00 2017 GMT
*  subjectAltName: host "dev.stephane-huc.net" matched cert's "dev.stephane-huc.net"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0xb98e7ef4000)
> HEAD / HTTP/2
> Host: dev.stephane-huc.net
> User-Agent: curl/7.53.1
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 404 
HTTP/2 404 
< server: nginx
server: nginx
< date: Wed, 14 Jun 2017 01:21:03 GMT
date: Wed, 14 Jun 2017 01:21:03 GMT
< content-type: text/html
content-type: text/html
< content-length: 162
content-length: 162
< access-control-allow-origin: dev.stephane-huc.net
access-control-allow-origin: dev.stephane-huc.net
< strict-transport-security: max-age=31536000; includeSubdomains; preload
strict-transport-security: max-age=31536000; includeSubdomains; preload
< x-content-type-options: nosniff
x-content-type-options: nosniff
< frame-options: DENY
frame-options: DENY
< x-frame-options: DENY
x-frame-options: DENY
< x-powered-by: HucSte: Power by Me, Myself, and moults softs :D
x-powered-by: HucSte: Power by Me, Myself, and moults softs :D
< x-xss-protection: 1; mode=block
x-xss-protection: 1; mode=block

< 
* Connection #0 to host dev.stephane-huc.net left intact
16

Is that on the same machine, or from a different machine elsewhere on the Internet?

17

with my openbsd station on my private lan. server is onto dmz (with his own different segment network).
between i’ve the opnsense firewall.

18

So I’m still thinking it’s a firewalling or routing problem because I tried from another machine that has IPv6 connectivity, elsewhere on the Internet.

$ curl -Iv6 ipv6.google.com
* Rebuilt URL to: ipv6.google.com/
*   Trying 2607:f8b0:4003:c14::65...
* Connected to ipv6.google.com (2607:f8b0:4003:c14::65) port 80 (#0)
> HEAD / HTTP/1.1
> Host: ipv6.google.com
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Date: Wed, 14 Jun 2017 01:36:50 GMT
Date: Wed, 14 Jun 2017 01:36:50 GMT
< Expires: -1
Expires: -1 [...]

but:

`$ curl -Iv6 dev.stephane-huc.net

and it just hangs indefinitely like that!

19

Ok, i change my records DNS into Gandi, to shutdown IPv6.
After few hours (min. 3), i retry the renew. Now, i’m going to my bed! :stuck_out_tongue:

See you later!
Good night, all.

20

That is definitely an issue but even with IPv4, all requests are being redirected to HTTPS.

For now, since the https://.../acme-challenge/test.txt file is working, I would suggest modifying the auth to HTTPS:
certbot renew --cert-name dev.stephane-huc.net --dry-run --preferred-challenges tls-sni