Certbot not updating. Unauthorized

Posso ler respostas em inglês:
Sim

Meu nome de domínio é:
nextcloud.norbertoneto.com

Executei esse comando:
certbot renew

Produziu essa saída:
root@713d91a06c32:/# certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log


Processing /etc/letsencrypt/renewal/nextcloud.norbertoneto.com.conf


Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate for nextcloud.norbertoneto.com
Performing the following challenges:
http-01 challenge for nextcloud.norbertoneto.com
Waiting for verification...
Challenge failed for domain nextcloud.norbertoneto.com
http-01 challenge for nextcloud.norbertoneto.com
Cleaning up challenges
Failed to renew certificate nextcloud.norbertoneto.com with error: Some challenges have failed.


All renewals failed. The following certificates could not be renewed:
/etc/letsencrypt/live/nextcloud.norbertoneto.com/fullchain.pem (failure)


1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

Meu servidor web é (com versão):
root@713d91a06c32:/# nginx -v
nginx version: nginx/1.19.8
root@713d91a06c32:/#

O sistema operacional no meu servidor web é (com versão):
root@713d91a06c32:/# cat /etc/issue
Debian GNU/Linux 10 \n \l

root@713d91a06c32:/#

Its a docker with image staticfloat/nginx-certbot

O serviço de hospedagem do meu site (se aplicável) é:
N/A

Posso acessar um shell root na minha máquina (sim ou não, ou não sei):
Sim

Uso um painel de controle para administrar meu site (não, ou indique o nome e a versão do painel de controle):
Não

Saída nginx -T:

root@713d91a06c32:/# nginx -T
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# configuration file /etc/nginx/nginx.conf:

user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

# configuration file /etc/nginx/mime.types:

types {
    text/html                                        html htm shtml;
    text/css                                         css;
    text/xml                                         xml;
    image/gif                                        gif;
    image/jpeg                                       jpeg jpg;
    application/javascript                           js;
    application/atom+xml                             atom;
    application/rss+xml                              rss;

    text/mathml                                      mml;
    text/plain                                       txt;
    text/vnd.sun.j2me.app-descriptor                 jad;
    text/vnd.wap.wml                                 wml;
    text/x-component                                 htc;

    image/png                                        png;
    image/svg+xml                                    svg svgz;
    image/tiff                                       tif tiff;
    image/vnd.wap.wbmp                               wbmp;
    image/webp                                       webp;
    image/x-icon                                     ico;
    image/x-jng                                      jng;
    image/x-ms-bmp                                   bmp;

    font/woff                                        woff;
    font/woff2                                       woff2;

    application/java-archive                         jar war ear;
    application/json                                 json;
    application/mac-binhex40                         hqx;
    application/msword                               doc;
    application/pdf                                  pdf;
    application/postscript                           ps eps ai;
    application/rtf                                  rtf;
    application/vnd.apple.mpegurl                    m3u8;
    application/vnd.google-earth.kml+xml             kml;
    application/vnd.google-earth.kmz                 kmz;
    application/vnd.ms-excel                         xls;
    application/vnd.ms-fontobject                    eot;
    application/vnd.ms-powerpoint                    ppt;
    application/vnd.oasis.opendocument.graphics      odg;
    application/vnd.oasis.opendocument.presentation  odp;
    application/vnd.oasis.opendocument.spreadsheet   ods;
    application/vnd.oasis.opendocument.text          odt;
    application/vnd.openxmlformats-officedocument.presentationml.presentation
                                                     pptx;
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
                                                     xlsx;
    application/vnd.openxmlformats-officedocument.wordprocessingml.document
                                                     docx;
    application/vnd.wap.wmlc                         wmlc;
    application/x-7z-compressed                      7z;
    application/x-cocoa                              cco;
    application/x-java-archive-diff                  jardiff;
    application/x-java-jnlp-file                     jnlp;
    application/x-makeself                           run;
    application/x-perl                               pl pm;
    application/x-pilot                              prc pdb;
    application/x-rar-compressed                     rar;
    application/x-redhat-package-manager             rpm;
    application/x-sea                                sea;
    application/x-shockwave-flash                    swf;
    application/x-stuffit                            sit;
    application/x-tcl                                tcl tk;
    application/x-x509-ca-cert                       der pem crt;
    application/x-xpinstall                          xpi;
    application/xhtml+xml                            xhtml;
    application/xspf+xml                             xspf;
    application/zip                                  zip;

    application/octet-stream                         bin exe dll;
    application/octet-stream                         deb;
    application/octet-stream                         dmg;
    application/octet-stream                         iso img;
    application/octet-stream                         msi msp msm;

    audio/midi                                       mid midi kar;
    audio/mpeg                                       mp3;
    audio/ogg                                        ogg;
    audio/x-m4a                                      m4a;
    audio/x-realaudio                                ra;

    video/3gpp                                       3gpp 3gp;
    video/mp2t                                       ts;
    video/mp4                                        mp4;
    video/mpeg                                       mpeg mpg;
    video/quicktime                                  mov;
    video/webm                                       webm;
    video/x-flv                                      flv;
    video/x-m4v                                      m4v;
    video/x-mng                                      mng;
    video/x-ms-asf                                   asx asf;
    video/x-ms-wmv                                   wmv;
    video/x-msvideo                                  avi;
}

# configuration file /etc/nginx/conf.d/certbot.conf:
server {
    # Listen on plain old HTTP
    listen 80 default_server reuseport;
    listen [::]:80 default_server reuseport;

    # Pass this particular URL off to certbot, to authenticate HTTPS certificates
    location '/.well-known/acme-challenge' {
        default_type "text/plain";
        proxy_pass http://localhost:1337;
    }

    # Everything else gets shunted over to HTTPS
    location / {
        return 301 https://$http_host$request_uri;
    }
}

# configuration file /etc/nginx/conf.d/nextcloud.conf:
upstream php-handler {
    server app:9000;
}

server {
    listen       80;
    listen       443 ssl http2;
    server_name  nextcloud.norbertoneto.com;
    # Max allowed upload file in web page
    client_max_body_size 20G;
    if ($server_port !~ 443){
        rewrite ^(/.*)$ https://$host$1 permanent;
    }

    # SSL and Support TLSv1.3
    ssl_certificate    /etc/letsencrypt/live/nextcloud.norbertoneto.com/fullchain.pem;
    ssl_certificate_key    /etc/letsencrypt/live/nextcloud.norbertoneto.com/privkey.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers TLS13-AES-256-GCM-SHA384:TLS13-CHACHA20-POLY1305-SHA256:TLS13-AES-128-GCM-SHA256:TLS13-AES-128-CCM-8-SHA256:TLS13-AES-128-CCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE;
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 10m;
    error_page 497  https://$host$request_uri;

    # add header information to fix Nextcloud console warning messages
    add_header X-Content-Type-Options nosniff;
    add_header X-XSS-Protection "1; mode=block";
    add_header X-Robots-Tag none;
    add_header X-Download-Options noopen;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Permitted-Cross-Domain-Policies none;
    add_header Referrer-Policy no-referrer;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    fastcgi_hide_header X-Powered-By;

    root /var/www/html;

    location = /robots.txt {
        allow all;
        log_not_found off;
        access_log off;
    }

    location = /.well-known/carddav {
      return 301 $scheme://$host/remote.php/dav;
    }
    location = /.well-known/caldav {
      return 301 $scheme://$host/remote.php/dav;
    }

    # Enable gzip compression
    gzip on;
    gzip_vary on;
    gzip_comp_level 4;
    gzip_min_length 256;
    gzip_proxied expired no-cache no-store private no_last_modified no_etag auth;
    gzip_types application/atom+xml application/javascript application/json application/ld+json application/manifest+json application/rss+xml application/vnd.geo+json application/vnd.ms-fontobject application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/bmp image/svg+xml image/x-icon text/cache-manifest text/css text/plain text/vcard text/vnd.rim.location.xloc text/vtt text/x-component text/x-cross-domain-policy;

    # redirection and disable to access backend sensitive folders and resources
    location / {
        rewrite ^ /index.php$request_uri;
    }

    location ~ ^\/(?:build|tests|config|lib|3rdparty|templates|data)\/ {
        deny all;
    }
    location ~ ^\/(?:\.|autotest|occ|issue|indie|db_|console) {
        deny all;
    }

    # PHP configuration
    location ~ ^\/(?:index|remote|public|cron|core\/ajax\/update|status|ocs\/v[12]|updater\/.+|ocs-provider\/.+)\.php(?:$|\/) {
        fastcgi_split_path_info ^(.+?\.php)(\/.*|)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
        fastcgi_param HTTPS on;
        #Avoid sending the security headers twice
        fastcgi_param modHeadersAvailable true;
        fastcgi_param front_controller_active true;
        fastcgi_pass php-handler;
        fastcgi_intercept_errors on;
        fastcgi_request_buffering off;
    }

    location ~ ^\/(?:updater|ocs-provider)(?:$|\/) {
        try_files $uri/ =404;
        index index.php;
    }

    location ~ \.(?:css|js|woff2?|svg|gif)$ {
        try_files $uri /index.php$request_uri;
        add_header Cache-Control "public, max-age=15778463";
        add_header X-Content-Type-Options nosniff;
        add_header X-XSS-Protection "1; mode=block";
        add_header X-Robots-Tag none;
        add_header X-Download-Options noopen;
        add_header X-Permitted-Cross-Domain-Policies none;
        add_header Referrer-Policy no-referrer;
        access_log off;
    }

    location ~ \.(?:png|html|ttf|ico|jpg|jpeg)$ {
        try_files $uri /index.php$request_uri;
        # Optional: Don't log access to other assets
        access_log off;
    }
}

# configuration file /etc/nginx/fastcgi_params:

fastcgi_param  QUERY_STRING       $query_string;
fastcgi_param  REQUEST_METHOD     $request_method;
fastcgi_param  CONTENT_TYPE       $content_type;
fastcgi_param  CONTENT_LENGTH     $content_length;

fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
fastcgi_param  REQUEST_URI        $request_uri;
fastcgi_param  DOCUMENT_URI       $document_uri;
fastcgi_param  DOCUMENT_ROOT      $document_root;
fastcgi_param  SERVER_PROTOCOL    $server_protocol;
fastcgi_param  REQUEST_SCHEME     $scheme;
fastcgi_param  HTTPS              $https if_not_empty;

fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
fastcgi_param  SERVER_SOFTWARE    nginx/$nginx_version;

fastcgi_param  REMOTE_ADDR        $remote_addr;
fastcgi_param  REMOTE_PORT        $remote_port;
fastcgi_param  SERVER_ADDR        $server_addr;
fastcgi_param  SERVER_PORT        $server_port;
fastcgi_param  SERVER_NAME        $server_name;

# PHP only, required if PHP was built with --enable-force-cgi-redirect
fastcgi_param  REDIRECT_STATUS    200;

root@713d91a06c32:/#

Thank you for all you support!

1 Like

The standalone authenticator requires your nginx server to be stopped. It needs to use port 80 to process the incoming HTTP challenge.

Usually, when you have a web server like nginx you use the nginx plug-in or webroot instead. With these methods you do not have to stop nginx to renew your cert.

4 Likes

It's not a problem to stop my nginx server, but it didn't work too:

root@713d91a06c32:/# service nginx status
[ ok ] nginx is running.
root@713d91a06c32:/# service nginx stop
root@713d91a06c32:/# service nginx status
[FAIL] nginx is not running ... failed!
root@713d91a06c32:/# certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log


Processing /etc/letsencrypt/renewal/nextcloud.norbertoneto.com.conf


Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate for nextcloud.norbertoneto.com
Performing the following challenges:
http-01 challenge for nextcloud.norbertoneto.com
Waiting for verification...
Challenge failed for domain nextcloud.norbertoneto.com
http-01 challenge for nextcloud.norbertoneto.com
Cleaning up challenges
Failed to renew certificate nextcloud.norbertoneto.com with error: Some challenges have failed.


All renewals failed. The following certificates could not be renewed:
/etc/letsencrypt/live/nextcloud.norbertoneto.com/fullchain.pem (failure)


1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: nextcloud.norbertoneto.com
    Type: connection
    Detail: 144.22.216.143: Fetching
    http://nextcloud.norbertoneto.com/.well-known/acme-challenge/ZBmYawOnIuC9u5Rdnuf-Fmq0F_ck0qT36i1ELls-z-g:
    Error getting validation data

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A/AAAA record(s) for that domain
    contain(s) the right IP address. Additionally, please check that
    your computer has a publicly routable IP address and that no
    firewalls are preventing the server from communicating with the
    client. If you're using the webroot plugin, you should also verify
    that you are serving files from the webroot path you provided.
    root@713d91a06c32:/#

:frowning:

1 Like

I apologize. I did not look at your nginx config close enough. It looks like you have nginx pass the http challenge request to local port 1337. I'm guessing you run certbot standalone so it listens on that port? Is that right? If so, then nginx needs to be running to pass the request to certbot standalone.

That seems complicated. It is difficult to debug comms problems with such a setup. Is there a reason you cannot just handle the requests in nginx?

It must have worked at least once to get the cert using standalone. Otherwise the renew would not have tried that method. What changed since then?

4 Likes

This is very problematic:

5 Likes

Actually there isnt a local port 1337 running, nor in my docker:

root@713d91a06c32:/# netstat -ntpl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 307/nginx: master p
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 307/nginx: master p
tcp 0 0 127.0.0.11:46765 0.0.0.0:* LISTEN -
tcp6 0 0 :::80 :::* LISTEN 307/nginx: master p
root@713d91a06c32:/#

neither in my host:

shampoo@shampoo:~$ netstat -ntpl
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN -
tcp6 0 0 :::80 :::* LISTEN -
tcp6 0 0 :::22 :::* LISTEN -
tcp6 0 0 :::443 :::* LISTEN -
shampoo@shampoo:~$

Below are the 4 containers running to serve this nextcloud app:

shampoo@shampoo:~$ docker container ls -a | grep nextcloud
7ae7d327eeb1 mariadb "docker-entrypoint.s…" 2 months ago Up 44 hours 3306/tcp nextcloud_db_1
82baf56f7844 rcdailey/nextcloud-cronjob "tini -- /entrypoint…" 2 months ago Up 44 hours (healthy) nextcloud_cron_1
713d91a06c32 staticfloat/nginx-certbot "/bin/bash /scripts/…" 2 months ago Up 44 hours 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp nextcloud_proxy_1
17f5f5d940bc nextcloud:fpm "/entrypoint.sh php-…" 2 months ago Up 44 hours 9000/tcp nextcloud_app_1
shampoo@shampoo:~$

As you may see, there is one for the nextcloud itself, one for DB, one for proxy (this ones runs certbot) and another for cronjobs. Any of them has the port 1337 open.

I followed this tutorial to deploy it: https://blog.51sec.org/2021/01/install-nextcloud-docker-and-integrate.html

The first time, it get the certificate by itself as soon as you run the docker-compose up command.

Also, I'm running everything below a fast reverse proxy serving both 80 and 443 directly to my host. This frp is running in a Oracle Cloud host.

And there wouldn't be.
It is set aside for certbot to use while in --standalone mode [only].

How will that proxy serve the secure content?

3 Likes

Hello @rg305 , how are you doing?

May you tell me more about this is very problematic? I'm not a spec in webserver confs.

Thanks for all you help!

2 Likes

I think I am causing more confusion in this thread. Just want to show what was confusing me:

You have this default server (only part of config shown). See the location to redirect to something on localhost port 1337. But, your nextcloud domain has a combined server block for port 80 and 443 so this default server will not be used for that.

Instead, this server block will see the HTTP challenge request and redirect it to HTTPS with that if statement checking the server_port. The challenge will then come in on HTTPS but there is no location block to handle it. So, it looks like it would just get served as a regular file from the root folder /var/www/html (but I am not certain of this).

I don't understand why you would use standalone. I think you should use certonly webroot instead. Your port 80 and 443 server block would be cleaner to split into two server blocks (one for each port) and you could then more clearly handle the HTTP challenge in the HTTP server block.

4 Likes

Port 80 is for HTTP.
Port 443 is for HTTPS.
How can you serve HTTP & HTTPS from the same vhost?

3 Likes

I run frp server in a public host (with a public ip).
I run frp client in my private host binding port 80 and 443.
Every tcp packet which hits the public host, it's forward directly to my private host.

Here is the project for frp: Docker

OK, I got the idea but I just don't know how to accomplish that. Any clue how I can change it to try it out?

Is there any more information you need to check to help me on this?

Thank you!

1 Like

It worked. I tried some confs here and I found the problem.

For some reason, the nginx conf file really had both ports 80 and 443 for the same server.
I removed the port 80 for the nextcloud conf and then, when I ran certbot renew, it worked:

root@713d91a06c32:/var/www/html# certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log


Processing /etc/letsencrypt/renewal/nextcloud.norbertoneto.com.conf


Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate for nextcloud.norbertoneto.com
Performing the following challenges:
http-01 challenge for nextcloud.norbertoneto.com
Waiting for verification...
Cleaning up challenges


new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/nextcloud.norbertoneto.com/fullchain.pem



Congratulations, all renewals succeeded:
/etc/letsencrypt/live/nextcloud.norbertoneto.com/fullchain.pem (success)


root@713d91a06c32:/var/www/html#

Thank you all for all the support and tips.

2 Likes

Isn't that what @rg305 suggested above - where he asked you why you had both Port 80 & 443 in one vHost server block?

Certbot not updating. Unauthorized - #11 by rg305

Glad you took it out and got your cert afterward. :+1:

5 Likes

We are talking about two different meanings for vhost.
I'm talking about a virtual host within a web server serving two ports simultaneously:
[which is very difficult to do OR impossible]

You are talking about a VPS or Docker virtual instance serving two ports simultaneously.
[which is normal and expected]

3 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.