Certbot failed to authenticate some domains Invalid response

hi there.
I got this error when I try to simulate renewing
the saletoeurope.com is accessible you can try it

root@saletoeurope:~# sudo certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log


Processing /etc/letsencrypt/renewal/saletoeurope.com-0001.conf


Simulating renewal of an existing certificate for saletoeurope.com and www.saletoeurope.com

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: saletoeurope.com
Type: unauthorized
Detail: 46.101.112.67: Invalid response from Saletoeurope "<!doctype html><html lang="en"><meta charset="utf-8"/><link rel="icon" href="/logo.png"/><meta name="viewport" content="wi"

Domain: www.saletoeurope.com
Type: unauthorized
Detail: 46.101.112.67: Invalid response from Saletoeurope "<!doctype html><html lang="en"><meta charset="utf-8"/><link rel="icon" href="/logo.png"/><meta name="viewport" content="wi"

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Failed to renew certificate saletoeurope.com-0001 with error: Some challenges have failed.


Processing /etc/letsencrypt/renewal/saletoeurope.com.conf


Renewal configuration file /etc/letsencrypt/renewal/saletoeurope.com.conf is broken.
The error was: expected /etc/letsencrypt/live/saletoeurope.com/cert.pem to be a symlink
Skipping.


All simulated renewals failed. The following certificates could not be renewed:
/etc/letsencrypt/live/saletoeurope.com-0001/fullchain.pem (failure)

Additionally, the following renewal configurations were invalid:
/etc/letsencrypt/renewal/saletoeurope.com.conf (parsefail)


1 renew failure(s), 1 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

Welcome @Abdulmelik619

There are several problems here. You did not answer all of the questions on the form you were shown. Those answers would be helpful to assist you.

Are you using nginx v1.18 on Ubuntu?

If so, please show the output of this command. An upper case T is essential. The output will be long

sudo nginx -T
2 Likes

Without the answers to the questions you have erased...
I can only guess.
I guess that the HTTP vhost doesn't handle the ACME challenge requests [as it should] and it forwards all requests to HTTPS. And the HTTPS vhost also doesn't handle the ACME challenge requests.

1 Like

Agreed. Several problems likely

  • The webroot-path that Certbot uses doesn't match the root folder in their nginx
  • Redirecting the http challenge unnecessarily
  • The original Certbot cert profile is damaged
  • The folder their nginx uses for their ssl cert statements is likely wrong. That is, do they use the path from the damaged profile or the new one that isn't working?

We need to see the nginx config to sort this out.

2 Likes

thank you all for your answers here is the nginx conf file
root@saletoeurope:~# nginx -v
nginx version: nginx/1.18.0 (Ubuntu)
root@saletoeurope:~# sudo nginx -T
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

configuration file /etc/nginx/nginx.conf:

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
worker_connections 768;
# multi_accept on;
}

http {

    ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    types_hash_max_size 2048;
    # server_tokens off;

    # server_names_hash_bucket_size 64;
    # server_name_in_redirect off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # SSL Settings
    ##

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;

    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    ##
    # Gzip Settings
    ##

    gzip on;

    # gzip_vary on;
    # gzip_proxied any;
    # gzip_comp_level 6;
    # gzip_buffers 16 8k;
    # gzip_http_version 1.1;
    # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;

}

#mail {

# See sample authentication script at:

# Using a PHP Script on an Apache Server as the IMAP Auth Backend | NGINX

# auth_http localhost/auth.php;

# pop3_capabilities "TOP" "USER";

# imap_capabilities "IMAP4rev1" "UIDPLUS";

server {

listen localhost:110;

protocol pop3;

proxy on;

}

server {

listen localhost:143;

protocol imap;

proxy on;

}

#}

configuration file /etc/nginx/modules-enabled/50-mod-http-geoip2.conf:

load_module modules/ngx_http_geoip2_module.so;

configuration file /etc/nginx/modules-enabled/50-mod-http-image-filter.conf:

load_module modules/ngx_http_image_filter_module.so;

configuration file /etc/nginx/modules-enabled/50-mod-http-xslt-filter.conf:

load_module modules/ngx_http_xslt_filter_module.so;

configuration file /etc/nginx/modules-enabled/50-mod-mail.conf:

load_module modules/ngx_mail_module.so;

configuration file /etc/nginx/modules-enabled/50-mod-stream.conf:

load_module modules/ngx_stream_module.so;

configuration file /etc/nginx/modules-enabled/70-mod-stream-geoip2.conf:

load_module modules/ngx_stream_geoip2_module.so;

configuration file /etc/nginx/mime.types:

types {
text/html html htm shtml;
text/css css;
text/xml xml;
image/gif gif;
image/jpeg jpeg jpg;
application/javascript js;
application/atom+xml atom;
application/rss+xml rss;

text/mathml                           mml;
text/plain                            txt;
text/vnd.sun.j2me.app-descriptor      jad;
text/vnd.wap.wml                      wml;
text/x-component                      htc;

image/png                             png;
image/tiff                            tif tiff;
image/vnd.wap.wbmp                    wbmp;
image/x-icon                          ico;
image/x-jng                           jng;
image/x-ms-bmp                        bmp;
image/svg+xml                         svg svgz;
image/webp                            webp;

application/font-woff                 woff;
application/java-archive              jar war ear;
application/json                      json;
application/mac-binhex40              hqx;
application/msword                    doc;
application/pdf                       pdf;
application/postscript                ps eps ai;
application/rtf                       rtf;
application/vnd.apple.mpegurl         m3u8;
application/vnd.ms-excel              xls;
application/vnd.ms-fontobject         eot;
application/vnd.ms-powerpoint         ppt;
application/vnd.wap.wmlc              wmlc;
application/vnd.google-earth.kml+xml  kml;
application/vnd.google-earth.kmz      kmz;
application/x-7z-compressed           7z;
application/x-cocoa                   cco;
application/x-java-archive-diff       jardiff;
application/x-java-jnlp-file          jnlp;
application/x-makeself                run;
application/x-perl                    pl pm;
application/x-pilot                   prc pdb;
application/x-rar-compressed          rar;
application/x-redhat-package-manager  rpm;
application/x-sea                     sea;
application/x-shockwave-flash         swf;
application/x-stuffit                 sit;
application/x-tcl                     tcl tk;
application/x-x509-ca-cert            der pem crt;
application/x-xpinstall               xpi;
application/xhtml+xml                 xhtml;
application/xspf+xml                  xspf;
application/zip                       zip;

application/octet-stream              bin exe dll;
application/octet-stream              deb;
application/octet-stream              dmg;
application/octet-stream              iso img;
application/octet-stream              msi msp msm;

application/vnd.openxmlformats-officedocument.wordprocessingml.document    docx;
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet          xlsx;
application/vnd.openxmlformats-officedocument.presentationml.presentation  pptx;

audio/midi                            mid midi kar;
audio/mpeg                            mp3;
audio/ogg                             ogg;
audio/x-m4a                           m4a;
audio/x-realaudio                     ra;

video/3gpp                            3gpp 3gp;
video/mp2t                            ts;
video/mp4                             mp4;
video/mpeg                            mpeg mpg;
video/quicktime                       mov;
video/webm                            webm;
video/x-flv                           flv;
video/x-m4v                           m4v;
video/x-mng                           mng;
video/x-ms-asf                        asx asf;
video/x-ms-wmv                        wmv;
video/x-msvideo                       avi;

}

configuration file /etc/nginx/sites-enabled/default:

You should look at the following URL's in order to grasp a solid understanding

of Nginx configuration files in order to fully unleash the power of Nginx.

Getting Started | NGINX

Pitfalls and Common Mistakes | NGINX

Nginx/DirectoryStructure - Debian Wiki

In most cases, administrators will remove this file from sites-enabled/ and

leave it as reference inside of sites-available where it will continue to be

updated by the nginx packaging team.

This file will automatically load configuration files provided by other

applications, such as Drupal or Wordpress. These applications will be made

available underneath a path with that package name, such as /drupal8.

Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.

Default server configuration

server {
listen 80 default_server;
listen [::]:80 default_server;

    # SSL configuration
    #
    # listen 443 ssl default_server;
    # listen [::]:443 ssl default_server;
    #
    # Note: You should disable gzip for SSL traffic.
    # See: https://bugs.debian.org/773332
    #
    # Read up on ssl_ciphers to ensure a secure configuration.
    # See: https://bugs.debian.org/765782
    #
    # Self signed certs generated by the ssl-cert package
    # Don't use them in a production server!
    #
    # include snippets/snakeoil.conf;

    root /var/www/build;

    # Add index.php to the list if you are using PHP
    index index.html index.htm index.nginx-debian.html;

    server_name _;

    location / {
try_files $uri /index.html;

}

    # pass PHP scripts to FastCGI server
    #
    #location ~ \.php$ {
    #       include snippets/fastcgi-php.conf;
    #
    #       # With php-fpm (or other unix sockets):
    #       fastcgi_pass unix:/run/php/php7.4-fpm.sock;
    #       # With php-cgi (or other tcp sockets):
    #       fastcgi_pass 127.0.0.1:9000;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #       deny all;
    #}
    return 301 https://$host$request_uri;

}

Virtual Host configuration for example.com

You can move that to a different file under sites-available/ and symlink that

to sites-enabled/ to enable it.

#server {

listen 80;

listen [::]:80;

server_name example.com;

root /var/www/example.com;

index index.html;

location / {

try_files $uri $uri/ =404;

}

#}
server {
listen 443 ssl default_server;
listen [::]:443 ssl default_server;

    ssl_certificate /etc/nginx/sites-available/live/live/backend.saletoeurope.com/fullchain.pem;
    ssl_certificate_key /etc/nginx/sites-available/live/live/backend.saletoeurope.com/privkey.pem;

    server_name backend.saletoeurope.com;

location / {
    proxy_pass http://localhost:3000;  # Replace with the address of your Nest.js application
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
}

server_name example.com;

root /var/www/example.com;

index index.html;

location / {

try_files $uri $uri/ =404;

}

}

server {
listen 443 ssl;
listen [::]:443 ssl;

root /var/www/build;

ssl_certificate  /etc/letsencrypt/live/saletoeurope.com-0001/fullchain.pem;
ssl_certificate_key  /etc/letsencrypt/live/saletoeurope.com-0001/privkey.pem;

server_name saletoeurope.com www.saletoeurope.com;

location / {
try_files $uri /index.html;
}

# Additional configurations for your React app go here

# Additional SSL configurations go here

}

root@saletoeurope:

2 Likes

Thanks for that. Can you now also show the contents of this

/etc/letsencrypt/renewal/saletoeurope.com-0001.conf
3 Likes

here it is
root@saletoeurope:~# cat /etc/letsencrypt/renewal/saletoeurope.com-0001.conf

renew_before_expiry = 30 days

version = 2.8.0
archive_dir = /etc/letsencrypt/archive/saletoeurope.com-0001
cert = /etc/letsencrypt/live/saletoeurope.com-0001/cert.pem
privkey = /etc/letsencrypt/live/saletoeurope.com-0001/privkey.pem
chain = /etc/letsencrypt/live/saletoeurope.com-0001/chain.pem
fullchain = /etc/letsencrypt/live/saletoeurope.com-0001/fullchain.pem

Options used in the renewal process

[renewalparams]
account = 839661605883fd3bab8c4b86fc008ddc
authenticator = webroot
webroot_path = /root/live,
server = https://acme-v02.api.letsencrypt.org/directory
key_type = ecdsa
[[webroot_map]]
root@saletoeurope:~#

1 Like

The problem is the webroot_path in that renewal config does not match the root folder in your nginx config.

The quickest fix is to reconfigure your Certbot profile to use the correct path. This should do that:

sudo certbot reconfigure --cert-name saletoeurope.com-0001 --webroot -w /var/www/build

In addition to that, it would be better if you also added a dedicated server block for those two domains. You currently rely on the default server block which redirects the HTTP Challenge to your HTTPS server block. Having a dedicate server block for HTTP is more reliable and less likely to result in odd configuration issues.

It would look something like this:

server {
  listen 80;
  listen [::]:80;

  root /var/www/build;
  server_name saletoeurope.com www.saletoeurope.com;

  location /.well-known/acme-challenge {
     # even better is to make and use a unique path like /var/certbot
     # but then you would also reconfigure your certbot profile to use new path too
     root /var/www/build;   
  }

  location / {
    return 301 https://$host$request_uri;
  }
}

Oops. Corrected a missing }

3 Likes

Once the above is working let us know and we can instruct about deleting the broken Certbot renewal profile

3 Likes

I think the renew succeded. I'm incredibly grateful for your assistance. Here's the log.
root@saletoeurope:~# sudo certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log


Processing /etc/letsencrypt/renewal/saletoeurope.com-0001.conf


Simulating renewal of an existing certificate for saletoeurope.com and www.saletoeurope.com


Processing /etc/letsencrypt/renewal/saletoeurope.com.conf


Renewal configuration file /etc/letsencrypt/renewal/saletoeurope.com.conf is broken.
The error was: expected /etc/letsencrypt/live/saletoeurope.com/cert.pem to be a symlink
Skipping.


Congratulations, all simulated renewals succeeded:
/etc/letsencrypt/live/saletoeurope.com-0001/fullchain.pem (success)

Additionally, the following renewal configurations were invalid:
/etc/letsencrypt/renewal/saletoeurope.com.conf (parsefail)


0 renew failure(s), 1 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
root@saletoeurope:~#

1 Like

Yes, the -0001 cert profile looks good. Thanks for the thanks :slight_smile:

You should delete the broken profile.

Normally a sudo certbot delete --cert-name saletoeurope.com
would delete it but because it is broken you may need to manually delete this file:

/etc/letsencrypt/renewal/saletoeurope.com.conf
3 Likes

First I'd like to express my gratitude for your support and valuable suggestions.
my question is why it is saying only for saletoeurope.com and www.saletoeurope.com
Simulating renewal of an existing certificate for saletoeurope.com and www.saletoeurope.com

what about backend.saletoeurope.com

isnt renew command expected to renew all the cirtificates

thank you again

2 Likes

Yes, all the certs you got from Certbot anyway. What does this show

sudo certbot certificates

I saw the backend domain and server block in your nginx. But, the path to the ssl certificates looked like a location that was not related to Certbot. It was below which is a very strange place to keep certificates. And, not one Certbot ever would have used.

ssl_certificate /etc/nginx/sites-available/live/live/backend.saletoeurope.com/fullchain.pem;
ssl_certificate_key /etc/nginx/sites-available/live/live/backend.saletoeurope.com/privkey.pem;
3 Likes

here is the log
root@saletoeurope:~# sudo certbot certificates
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Renewal configuration file /etc/letsencrypt/renewal/saletoeurope.com.conf produced an unexpected error: expected /etc/letsencrypt/live/saletoeurope.com/cert.pem to be a symlink. Skipping.


Found the following certs:
Certificate Name: saletoeurope.com-0001
Serial Number: 4931757f6b16ce1ade6ba1364897ec5db8e
Key Type: ECDSA
Domains: saletoeurope.com www.saletoeurope.com
Expiry Date: 2024-04-15 17:33:08+00:00 (VALID: 1 day)
Certificate Path: /etc/letsencrypt/live/saletoeurope.com-0001/fullchain.pem
Private Key Path: /etc/letsencrypt/live/saletoeurope.com-0001/privkey.pem

The following renewal configurations were invalid:
/etc/letsencrypt/renewal/saletoeurope.com.conf


root@saletoeurope:~#

That confirms that Certbot does not know anything about a cert for your backend domain

I see that you got a Let's Encrypt cert for it last January. How did you get it back then? And, do you know why it might not be known on this machine?

3 Likes

I dont know if this is the reason
my client and me has separately generated these cirtificates
I generated backend
while my client generated the front end cirtificates.

my question is cant I use the cirtificate generated for the domain(saletoeurope.com)
for its subdomains(www.saletoeurope.com and backend.saletoeurope.com)

From certbot certificates we see this Certbot certificate profile is good for the root name and the www subdomain.

I recommend to make another certificate just for backend subdomain. Or, you can add backend subdomain to above cert but this sometimes gets harder to manage in future.

Each cert must have the domain name that will be used in the URL to reach it.

Just follow the same pattern of server blocks and Certbot commands you used for the above cert to get the one for backend.

Right now all 3 domains have the same IP address and point to the same nginx server. So, you can get and manage the certs for all 3 names there.

3 Likes

Thank you very much for your support; it's greatly appreciated. I will do as you suggested.

3 Likes

here I was generating a certificate
root@saletoeurope:~# sudo certbot certonly
--manual
--preferred-challenges=dns
--email abdulmelikambaw619@gmail.com
--server https://acme-v02.api.letsencrypt.org/directory
--work-dir=. --config-dir=. --logs-dir=.
--agree-tos
-d backend.saletoeurope.com
Saving debug log to /root/letsencrypt.log
Requesting a certificate for backend.saletoeurope.com


Please deploy a DNS TXT record under the name:

_acme-challenge.backend.saletoeurope.com.

with the following value:

C9j1NXf97MPPJGCNKNmXn79tT9r4IAVvHY8NyU7NPlo

Before continuing, verify the TXT record has been deployed. Depending on the DNS
provider, this may take some time, from a few seconds to multiple minutes. You can
check if it has finished deploying with aid of online tools, such as the Google
Admin Toolbox: Dig (DNS lookup).
Look for one or more bolded line(s) below the line ';ANSWER'. It should show the
value(s) you've just added.


Press Enter to Continue

Successfully received certificate.
Certificate is saved at: /root/live/backend.saletoeurope.com/fullchain.pem
Key is saved at: /root/live/backend.saletoeurope.com/privkey.pem
This certificate expires on 2024-07-13.
These files will be updated when the certificate renews.

NEXT STEPS:

  • This certificate will not be renewed automatically. Autorenewal of --manual certificates requires the use of an authentication hook script (--manual-auth-hook) but one was not provided. To renew this certificate, repeat this same certbot command before the certificate's expiry date.

but I noticead this
This certificate will not be renewed automatically

Why didn't you use the sudo certbot certonly --webroot ... method you used for your other domain?

In any case, I see you posted a new thread asking about the auto-renew with the DNS challenge manual method

2 Likes