Certbot renew fails authorization process (possibly because I'm an idiot?)

Help
1

It looks like I may have to break this up into several posts because “new users can only include 20 links in a post”? I guess it’s seeing the domain names in the error output, because I haven’t included any links at all. Hopefully breaking it up will work. I apologize for the inconvenience.

My domain is: ethanfulbright.com, cethanfulbright.com and related subdomains, specifically lament.cethanfulbright.com (It may be important that cethanfulbright.com redirects to ethanfulbright.com?)

I ran this command: sudo certbot renew --dry-run (I’ve also tried it with preferred-challenge http, which produces a slightly different error, noted below)

It produced this output:

certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/cethanfulbright.com.conf

Cert is due for renewal, auto-renewing…
Plugins selected: Authenticator nginx, Installer nginx
Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
tls-sni-01 challenge for cethanfulbright.com
tls-sni-01 challenge for ethanfulbright.com
tls-sni-01 challenge for lament.ethanfulbright.com
tls-sni-01 challenge for lament.cethanfulbright.com
Waiting for verification…
Cleaning up challenges
Attempting to renew cert (cethanfulbright.com) from /etc/letsencrypt/renewal/cethanfulbright.com.conf produced an unexpected error: Failed authorization procedure. lament.cethanfulbright.com (tls-sni-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Connection refused. Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/cethanfulbright.com/fullchain.pem (failure)

** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates below have not been saved.)

All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/cethanfulbright.com/fullchain.pem (failure)
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates above have not been saved.)

1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: lament.cethanfulbright.com
    Type: connection
    Detail: Connection refused

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A/AAAA record(s) for that domain
    contain(s) the right IP address. Additionally, please check that
    your computer has a publicly routable IP address and that no
    firewalls are preventing the server from communicating with the
    client. If you’re using the webroot plugin, you should also verify
    that you are serving files from the webroot path you provided.

2

Part 2

============================================================
certbot renew --dry-run --preferred-challenge http
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/cethanfulbright.com.conf

Cert is due for renewal, auto-renewing…
Plugins selected: Authenticator nginx, Installer nginx
Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for cethanfulbright.com
http-01 challenge for ethanfulbright.com
http-01 challenge for lament.ethanfulbright.com
http-01 challenge for lament.cethanfulbright.com
Waiting for verification…
Cleaning up challenges
Attempting to renew cert (cethanfulbright.com) from /etc/letsencrypt/renewal/cethanfulbright.com.conf produced an unexpected error: Failed authorization procedure. lament.cethanfulbright.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://lament.cethanfulbright.com/.well-known/acme-challenge/8TxetVsRhq0KUmBsflCsB2GKJRXJ7cXodoTzXhaqjBc: “\n404 Not Found\n

Not Found

\n

The requested URL was”. Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/cethanfulbright.com/fullchain.pem (failure)

** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates below have not been saved.)

All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/cethanfulbright.com/fullchain.pem (failure)
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates above have not been saved.)

1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

My web server is (include version): nginx 1.10.3

The operating system my web server runs on is (include version): Ubuntu Server 16.04.4 LTS

My hosting provider, if applicable, is: N/A, I think? It runs on an AWS instance.

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

3

Part 3 (Final part)
Ports 443 and 80 are both open on the firewall. The site itself is easily reachable from the web, and redirects properly from http to https, as far as I can tell. I’m a programmer, not a sysadmin, so I imagine I’ve made an error somewhere due to my lack of understanding.

I think the problem may be that I’m a complete idiot regarding nginx configuration and such - I’ll include my nginx config file as well. I’m assuming it contains some glaring error that will be completely obvious to anyone who knows what they’re doing. :smiley:

The basic setup is this - nginx proxies everything to gunicorn/flask. I don’t know if that’s relevant, but I’m trying to be thorough.

I’ve read several threads with problems similar to mine, but as I’m redirecting one domain to another and redirecting all http traffic to https, I’m not sure where the problem might be, or what I might be doing wrong. Any help or guidance would be greatly appreciated!

Here’s my nginx sites-enabled config for the site - I think this might be where the problem is, as some of it looks a little hinky, but I just don’t understand nginx well enough to tell.

server {
if ($host = ethanfulbright.com) {
return 301 https://$host$request_uri;
} # managed by Certbot

if ($host = cethanfulbright.com) {                                                                     
    return 301 https://$host$request_uri;                                                              
} # managed by Certbot                                                                                 
                                                                                                       
                                                                                                       
    # Listen on port 80 (http)                                                                         
    listen 80;                                                                                         
    server_name ethanfulbright.com cethanfulbright.com;                                                
    location / {                                                                                       
            # redirect any requests to the same URL but on https                                       
            #return 301 https://$host$request_uri;                                                     
            proxy_pass http://localhost:42000;                                                         
    }

}
server {
if ($host = lament.ethanfulbright.com) {
return 301 https://$host$request_uri;
} # managed by Certbot

if ($host = lament.cethanfulbright.com) {                                                              
    return 301 https://$host$request_uri;                                                              
} # managed by Certbot                                                                                 
                                                                                                       
                                                                                                       
    # Listen on port 80 for subdomain                                                                  
    listen 80;                                                                                         
    server_name lament.ethanfulbright.com lament.cethanfulbright.com;                                  
    location / {                                                                                       
            # Redirect any requests to the same subdomain+URL but on https                             
            # return 301 https://$host$request_uri;                                                    
            proxy_pass http://localhost:42000;                                                         
    }

}
server {
# Listen on port 443 for subdomain
listen 443 ssl;
server_name lament.ethanfulbright.com lament.cethanfulbright.com;
location / {
# Forward everything to gunicorn
proxy_pass http://localhost:42000;
}
location /static {
alias /home/lament/final_testing/portfolio/static;
}

ssl_certificate /etc/letsencrypt/live/cethanfulbright.com/fullchain.pem; # managed by Certbot          
ssl_certificate_key /etc/letsencrypt/live/cethanfulbright.com/privkey.pem; # managed by Certbot

}
server {
# Listen on port 443 (https)
listen 443 ssl;
server_name ethanfulbright.com cethanfulbright.com;

    # Write access and error logs to /var/log                                                          
    access_log /var/log/portfolio_access.log;                                                          
    error_log /var/log/portfolio_error.log;                                                            
                                                                                                       
    location / {                                                                                       
            # Forward application requests to the gunicorn server                                      
            proxy_pass http://localhost:42000;                                                         
            proxy_redirect off;                                                                        
            proxy_set_header Host $host;                                                               
            proxy_set_header X-Real_IP $remote_addr;                                                   
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;                               
    }                                                                                                  
                                                                                                       
    location /static {                                                                                 
            alias /home/lament/final_testing/portfolio/static;                                         
            expires 30d;                                                                               
    }                                                                                                  
                                                                                                       
ssl_certificate /etc/letsencrypt/live/cethanfulbright.com/fullchain.pem; # managed by Certbot          
ssl_certificate_key /etc/letsencrypt/live/cethanfulbright.com/privkey.pem; # managed by Certbot

}

I’ve got about 7 days until my certs expire (given the authentication error I’m getting, I’m not sure why it worked when I got them in the first place, but it did), and I’m not sure what to do if that happens.

Please let me know if I’ve let anything out, or if I’m being colossally stupid or rude in some way. Thanks in advance for any help you can give.

4

Hi @Haplo

checking your two domain names: They have different ip addresses:

D:\temp>nslookup cethanfulbright.com.
Name: cethanfulbright.com
Address: 64.98.145.30

D:\temp>nslookup ethanfulbright.com.
Name: ethanfulbright.com
Address: 18.216.154.2

Does your server manage both ip addresses? If no, so that can't work.

So run one certbot at 64.98.145.30 with the cethan-Domain, the other at the other machine with the second domain.

And don't use "renew", add the domain names with the -d - option.

1 Like
5

Thanks for the quick response!

There should only be one IP, so that's strange. It's a single server with a static IP from AWS. The AWS server is the 18.* one. I think the other might be the redirect from my domain registrar?

I don't have access to the 64.* IP, because I'm pretty sure that's just Hover (my registrar) performing the redirect. Do I need to forward the domain manually in some way, so that it points properly at the 18.* IP and not Hover's redirect server?

And don’t use “renew”, add the domain names with the -d - option.

Not sure if I'm understanding correctly - I already have certificates for both domains, so isn't renew exactly what I need? I'm not quite sure why I would need to use -d. One domain just redirects to the other. There's only a single site here.

Again, I apologize if I'm grossly misunderstanding some basic concepts here - I'm quite new to managing DNS and such. Thanks again!

6

This is a good question. Checked your 64*-domain:

D:\temp>download http://cethanfulbright.com/ -h
Transfer-Encoding: chunked
Connection: keep-alive
Status: 303 See Other
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Request-Id: f1222957-f141-4fc2-a3fc-9dc334694e21
X-Runtime: 0.005074
Cache-Control: no-cache
Content-Type: text/html; charset=utf-8
Date: Sat, 13 Oct 2018 11:16:12 GMT
Location: http://ethanfulbright.com
P3P: CP="IDC DSP COR ADM DEVi TAIi PSA PSD IVAi IVDi CONi HIS OUR IND CNT"
Server: nginx/1.6.2 + Phusion Passenger 4.0.53
X-Powered-By: Phusion Passenger 4.0.53

Status: 303 SeeOther

425,57 milliseconds
0,43 seconds

Same with a file under /.well-known/acme-challenge/

But I don't know how Letsencrypt handles this unusual 303 SeeOther - Redirect. Letsencrypt follows 301 and 302 - redirects and ignores wrong certificates. But searching the forum "303 SeeOther" - no other result.

Are you able to create a dns A-record cethanfulbright.com -> your 18-ip?

Normally, you need that. But if you create two certificates (one with cethan..., the other with ethan...), you can split the problem in two different parts. So you are able to reduce the problems.

You can also use the test system and certonly to test the validation part. There are own (higher) rate limits. So try something like

certbot certonly --nginx -d ethanfulbright.com -d lament.ethanfulbright.com 
--test-cert --preferred-challenges http

same with the other domain. Or only one single domain.

1 Like
7

Well, with your help, I seem to have finally solved it! Sorry it took me so long to reply - we had some bad storms in my part of Texas and my internet’s been out for about 36 hours.

It turns out that Hover uses 303 redirects instead of the industry standard, and they have no plans to change. Sigh.

I ended up creating a CNAME record for the cethan domain that redirects everything to the ethan domain. That allowed the validation to go through and my certs to renew!

I would have been utterly lost without your amazing help, @JuergenAuer! Also, I can’t imagine trying to give tech support in a second language, especially with my extremely limited German. Ihr Englisch ist fantastisch. Vielen dank! (Bitte vergib mir mein schlechtes Deustch!)

1 Like
8

Happy to read that it works now.

yes, I read that. Such things are more important then certificates.

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.