Certbot 'invalid response from well-known acme challenge’

Help
#1

hi all,

I've been setting up an Nginx server to be able to access Ombi remotely (and securely) on a Raspberry Pi via my own domain. The domain's A Name has been modified - and resolves to - to my home ISP IP address. I can access the Nginx server and Ombi no problem over HTTP. Port 80 is open on my router and forwarding to the Pi IP address. Port 443 is open on the router and forwarding to the Pi IP address.

I previously installed a Let's Encrypt certificate using:

sudo certbot certonly --standalone -d mydomain.co.uk -d www.mydomain.co.uk

And it installed a certificate in /etc/letsencrypt

However, when I try to install a Let's Encrypt SSL certificate using this command:

sudo certbot certonly --webroot -w /var/www/mydomain -d my domain.co.uk -d www.mydomain.co.uk

I get this (I've anonymised the details):

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: mydomain.co.uk 
   Type:   unauthorized
   Detail: xx.xx.xx.xx: Invalid response from
   http://mydomain.co.uk/.well-known/acme-challenge/jkWODJ1CjEm03BVGkNXWZE3LifnhIbMx2brzk49oIsY:
   "<!DOCTYPE html><html lang=\"en\"><head><link rel=\"preconnect\"
   href=\"https://fonts.gstatic.com\" crossorigin=\"\">\n    <script
   type='t"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

My Nginx config for the domain looks like this (file name is mydomain.co.uk.conf):

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        listen 443 ssl;

        root /usr/share/nginx/html;
        index index.html index.htm;

        server_name mydomain.co.uk;

        ssl_certificate /etc/letsencrypt/live/mydomain.co.uk/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/mydomain.co.uk/privkey.pem;

        location / {
                try_files $uri $uri/ =404;
        }
}

Can anyone tell me what I'm doing wrong?

1 Like
#2

The reason for the failure with webroot is because the -w value must match the root value for that server. So, should be this folder:

Your combination of the http and https can easily lead to trouble. I recommend creating two server blocks. One for http and one for https. Your http server block will be nothing more than a redirect to https with a location block so it handles the acme challenge. Something like this:

    server {
      listen      80 default_server;
      listen      [::]:80 default_server;
      server_name mydomain.co.uk www.mydomain.co.uk;
      root        /usr/share/nginx/html;

      location /.well-known/acme-challenge {
         root /usr/share/nginx/html;
      }
      location / {
           return 301 https://$host$request_uri;
      }
}

Your https server would be as you showed but:

  • Add default_server to your listen 443 if that's what you really need
  • Add IPv6 listen for 443
  • Remove listen's for port 80
  • Add www.mydomain.co.uk to the server_name
8 Likes
#3

Hi there,

so, I changed the command to (I hope I got this right):

sudo certbot certonly --webroot -w /usr/share/nginx/html; -d mydoman.co.uk -d www.mydomain.co.uk

And got a result:

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/www.mydomain.co.uk/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/www.mydomain.co.uk/privkey.pem
   Your certificate will expire on 2022-09-30. To obtain a new or
   tweaked version of this certificate in the future, simply run
   certbot again. To non-interactively renew *all* of your
   certificates, run "certbot renew"

Which I'm assuming is good news.

I replaced the contents of the mydomin.co.uk.conf file with:

  server {
      listen      80 default_server;
      listen      [::]:80 default_server;
      server_name mydomain.co.uk www.mydomain.co.uk;
      root        /usr/share/nginx/html;

      location /.well-known/acme-challenge {
         root /usr/share/nginx/html;
      }
      location / {
           return 301 https://$host$request_uri;
      }
}

And that gets me the http version of my server on my domain.

However, I'm at a loss as to how to implement the https server. Does that go into the same my domain.co.uk.conf file or in a separate conf file? (You can tell I'm relatively new to this).

All help much appreciated.

1 Like
#4

Normally the semi-colon is left off. I'm surprised certbot did not give a syntax error.

Absolutely!

That should not be. The return 301 should redirect to your https server which you should be able to setup now you have your cert. You need to reload nginx after making changes and if changing ports (adding/deleting server blocks for example) you should restart.

You can put the second server block (for https) in the same conf with your http server block. There are many ways to organize them but (so far) yours looks simple enough to have both in same config file.

I don't have a link to a "how to" on general nginx config off-hand.

7 Likes
#5

Hi again,

thanks for this. I definitely restarted nginx after changing the conf and requesting the certificate. However, it's definitely not working on http and only works on http (notifying me that it's not secure).

Interestingly, when I tried the certbot command without the semi-colon, it failed. It works with it though.

I'm wary of messing with the conf file by trying to add the https server block, but I realise this isn't secure. I'll go read up on nginx server blocks.

1 Like
#6

Agh. I'm now getting the default nginx http html page and not the page I was expecting :frowning:

EDIT: seems it was just updating the server, so I am getting the correct page, but still only over http

#7

Can you share your domain name? It is hard to give specific advice without looking. Especially for someone just beginning. Any more details you can provide from the origin request form would be helpful (I removed questions you already answered).

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:

My web server is (include version):

The operating system my web server runs on is (include version):

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

7 Likes
#8

Here you go Mike (I truly appreciate the help):

My domain is: www.zoomworks.co.uk

My web server is (include version): nginx/1.18.0

The operating system my web server runs on is (include version): Raspbian GNU/Linux 11 (bullseye)

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): Plesk

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.12.0

1 Like
#9

Oh. If Plesk (or really any panel) is managing your config then you should work through the panel. Trying to update your config manually outside of Plesk can make a mess.

You should take a look at this

7 Likes
#10

Yeah. I manage a lot of websites via Plesk - but I’m not hosting the site/content with my reseller account (it’s being hosted on my Pi). Can you still issue the certificate via the Plesk panel?

Also - what should I do with th wold certificate stored on the Pi?

#11

IDK you answered you are using Plesk to manage this site. Are you?

Or did you install / setup nginx by hand so far?

7 Likes
#12

I mentioned at the top that the DNS A name record is resolved to my home ISP IP address and nginx is hosted manually on my Pi at home. So I’m only using Plesk to manage the records. It’s not hosted on my reseller.

#13

That's maybe not as clear as you think :slight_smile: The DNS records only I presume?

Anyway, can you show the output of this command? Please use 3 backticks before and after the output as it will be long. Will be much easier to read. I'm not sure where the backtick is on UK keyboard. But, like this:
```
output of sudo nginx -T
```
Thanks

7 Likes
#14

yes - just the DNS being managed with Plesk.

Here's the output:


nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# configuration file /etc/nginx/nginx.conf:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
	worker_connections 768;
	# multi_accept on;
}

http {

	##
	# Basic Settings
	##

	sendfile on;
	tcp_nopush on;
	types_hash_max_size 2048;
	# server_tokens off;

	# server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	##
	# SSL Settings
	##

	ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;

	##
	# Logging Settings
	##

	access_log /var/log/nginx/access.log;
	error_log /var/log/nginx/error.log;

	##
	# Gzip Settings
	##

	gzip on;

	# gzip_vary on;
	# gzip_proxied any;
	# gzip_comp_level 6;
	# gzip_buffers 16 8k;
	# gzip_http_version 1.1;
	# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

	##
	# Virtual Host Configs
	##

	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/sites-enabled/*;
}


#mail {
#	# See sample authentication script at:
#	# http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
#
#	# auth_http localhost/auth.php;
#	# pop3_capabilities "TOP" "USER";
#	# imap_capabilities "IMAP4rev1" "UIDPLUS";
#
#	server {
#		listen     localhost:110;
#		protocol   pop3;
#		proxy      on;
#	}
#
#	server {
#		listen     localhost:143;
#		protocol   imap;
#		proxy      on;
#	}
#}

# configuration file /etc/nginx/mime.types:

types {
    text/html                             html htm shtml;
    text/css                              css;
    text/xml                              xml;
    image/gif                             gif;
    image/jpeg                            jpeg jpg;
    application/javascript                js;
    application/atom+xml                  atom;
    application/rss+xml                   rss;

    text/mathml                           mml;
    text/plain                            txt;
    text/vnd.sun.j2me.app-descriptor      jad;
    text/vnd.wap.wml                      wml;
    text/x-component                      htc;

    image/png                             png;
    image/tiff                            tif tiff;
    image/vnd.wap.wbmp                    wbmp;
    image/x-icon                          ico;
    image/x-jng                           jng;
    image/x-ms-bmp                        bmp;
    image/svg+xml                         svg svgz;
    image/webp                            webp;

    application/font-woff                 woff;
    application/java-archive              jar war ear;
    application/json                      json;
    application/mac-binhex40              hqx;
    application/msword                    doc;
    application/pdf                       pdf;
    application/postscript                ps eps ai;
    application/rtf                       rtf;
    application/vnd.apple.mpegurl         m3u8;
    application/vnd.ms-excel              xls;
    application/vnd.ms-fontobject         eot;
    application/vnd.ms-powerpoint         ppt;
    application/vnd.wap.wmlc              wmlc;
    application/vnd.google-earth.kml+xml  kml;
    application/vnd.google-earth.kmz      kmz;
    application/x-7z-compressed           7z;
    application/x-cocoa                   cco;
    application/x-java-archive-diff       jardiff;
    application/x-java-jnlp-file          jnlp;
    application/x-makeself                run;
    application/x-perl                    pl pm;
    application/x-pilot                   prc pdb;
    application/x-rar-compressed          rar;
    application/x-redhat-package-manager  rpm;
    application/x-sea                     sea;
    application/x-shockwave-flash         swf;
    application/x-stuffit                 sit;
    application/x-tcl                     tcl tk;
    application/x-x509-ca-cert            der pem crt;
    application/x-xpinstall               xpi;
    application/xhtml+xml                 xhtml;
    application/xspf+xml                  xspf;
    application/zip                       zip;

    application/octet-stream              bin exe dll;
    application/octet-stream              deb;
    application/octet-stream              dmg;
    application/octet-stream              iso img;
    application/octet-stream              msi msp msm;

    application/vnd.openxmlformats-officedocument.wordprocessingml.document    docx;
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet          xlsx;
    application/vnd.openxmlformats-officedocument.presentationml.presentation  pptx;

    audio/midi                            mid midi kar;
    audio/mpeg                            mp3;
    audio/ogg                             ogg;
    audio/x-m4a                           m4a;
    audio/x-realaudio                     ra;

    video/3gpp                            3gpp 3gp;
    video/mp2t                            ts;
    video/mp4                             mp4;
    video/mpeg                            mpeg mpg;
    video/QuickTime                       mov;
    video/webm                            webm;
    video/x-flv                           flv;
    video/x-m4v                           m4v;
    video/x-mng                           mng;
    video/x-ms-asf                        asx asf;
    video/x-ms-wmv                        wmv;
    video/x-msvideo                       avi;
}

# configuration file /etc/nginx/sites-enabled/default:
##
# You should look at the following URL's in order to grasp a solid understanding
# of Nginx configuration files in order to fully unleash the power of Nginx.
# https://www.nginx.com/resources/wiki/start/
# https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/
# https://wiki.debian.org/Nginx/DirectoryStructure
#
# In most cases, administrators will remove this file from sites-enabled/ and
# leave it as reference inside of sites-available where it will continue to be
# updated by the nginx packaging team.
#
# This file will automatically load configuration files provided by other
# applications, such as Drupal or Wordpress. These applications will be made
# available underneath a path with that package name, such as /drupal8.
#
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
##

# Default server configuration
#
server {
	listen 80 default_server;
	listen [::]:80 default_server;

	# SSL configuration
	#
	# listen 443 ssl default_server;
	# listen [::]:443 ssl default_server;
	#
	# Note: You should disable gzip for SSL traffic.
	# See: https://bugs.debian.org/773332
	#
	# Read up on ssl_ciphers to ensure a secure configuration.
	# See: https://bugs.debian.org/765782
	#
	# Self signed certs generated by the ssl-cert package
	# Don't use them in a production server!
	#
	# include snippets/snakeoil.conf;

	root /var/www/html;

	# Add index.php to the list if you are using PHP
	index index.html index.htm index.nginx-debian.html;

	server_name _;

	location / {
		# First attempt to serve request as file, then
		# as directory, then fall back to displaying a 404.
		try_files $uri $uri/ =404;
	}

	# pass PHP scripts to FastCGI server
	#
	#location ~ \.php$ {
	#	include snippets/fastcgi-php.conf;
	#
	#	# With php-fpm (or other unix sockets):
	#	fastcgi_pass unix:/run/php/php7.4-fpm.sock;
	#	# With php-cgi (or other tcp sockets):
	#	fastcgi_pass 127.0.0.1:9000;
	#}

	# deny access to .htaccess files, if Apache's document root
	# concurs with nginx's one
	#
	#location ~ /\.hotel {
	#	deny all;
	#}
}


# Virtual Host configuration for example.com
#
# You can move that to a different file under sites-available/ and symlink that
# to sites-enabled/ to enable it.
#
#server {
#	listen 80;
#	listen [::]:80;
#
#	server_name example.com;
#
#	root /var/www/example.com;
#	index index.html;
#
#	location / {
#		try_files $uri $uri/ =404;
#	}
#}

# configuration file /etc/nginx/sites-enabled/zoomworks.co.uk.conf:
server {
	listen 80;
	server_name zoomworks.co.uk;
	location / {
	proxy_pass http://192.168.50.140:5000;
	}
}
#15

So, this is the only server block you have specifically for that domain name. Is that what you want it to do? That is, proxy to something at port 5000? Because that is not what you showed in your first post.

7 Likes
#16

Yeah. This is weird. This isn't what's in my zoomworks.co.uk.conf file - however, it is what I want to happen (securely forward to a service on port 5000 internally)

#17

If I do this:

sudo nano /etc/nginx/sites-enabled/zoomworks.co.uk.conf

I get this:

        
server {
        listen 80;
        server_name zoomworks.co.uk;
        location / {
        proxy_pass http://192.168.50.140:5000;
        }
}

If I do this:

sudo nano zoomworks.co.uk.conf

I get this:

 server {
      listen      80 default_server;
      listen      [::]:80 default_server;

      server_name zoomworks.co.uk www.zoomworks.co.uk;
      root        /usr/share/nginx/html;

      location /.well-known/acme-challenge {
         root /usr/share/nginx/html;
      }
      location / {
           return 301 https://$host$request_uri;
      }
}
#18

Hmmm. Well, nginx does not agree with you :slight_smile: The -T shows what nginx is using

Looks like you need to sort out your sites/enabled and sites/available folders. See this:

7 Likes
#19

In which directory is that file?
And what shows?:
ls -l /etc/nginx/sites-enabled/zoomworks.co.uk.conf

6 Likes
#20

That is a question I am trying to answer. Where on earth this rogue zoomworks.co.uk.conf file that I've been editing is.

If I do this:

ls -l /etc/nginx/sites-enabled/zoomworks.co.uk.conf

I get this:

lrwxrwxrwx 1 root root 47 Jul 2 12:40 /etc/nginx/sites-enabled/zoomworks.co.uk.conf -> /etc/nginx/sites-available/zoomworks.co.uk.conf

which is clearly the one I need to be editing. Essentially what I need is for it to be https and to proxy pass forward to port 5000 on the internal server.