How should I setup my website for certbot renewal?

I’ve been trying to renew my certificate recently. I’ve done this before, and although it’s always a struggle since I can’t get autorenew to work, this time I’m really having issues, and I can’t get it done. Now, that’s not the reason for posting here, as you can see in the title. I’m perfectly fine with following instructions online, and there’s plenty of those for renewing your certificate.

What I can’t find online, is how I should setup my website for certbot renew to work. At the moment, when I try certbot renew, I get an error regarding lacking sufficient authorization, and an invalid response from .../acme-challenge/..., which then shows some HTML code for the default page of PiHole, which also runs on my Pi.

Obviously, something is setup wrong regarding either the webpage I’m hosting at port 80, or whatever I’ve got set up in Apache.

I’m probably being blind, as usual, so if somebody could point me to a tutorial that shows me how to setup apache / my webpage for renewal to work, that would be great.

Hi @Timmiej93

what’s your domain name?

What says

apachectl -S

Hi @JuergenAuer

My domain name is timderks.duckdns.org. Under normal circumstances, the main page shouldn’t show anything (probably shows an error message).

apachectl -S oddly tells me that there’s a syntax error, being that my cert.pem file doesn’t exist or is empty, which isn’t the case. The certificate has expired though, so maybe that’s causing the issue?

No, that’s not the problem.

Run it as root / sudo.

Checking your domain there you see the problem - https://check-your-website.server-daten.de/?q=timderks.duckdns.org

http + /.well-known/acme-challenge/random-filename - there is the wrong http status 200 (404 is expected) and the wrong content:

Open Source Ad Blocker Designed for Raspberry Pi timderks.duckdns.org 1 10 [π]: whitelist.txt:timderks.duckdns.org Monday 7:42 PM, March 30th. Pi-hole v4.3.2-0-ge41c4b5 (RPi3B-plus/192.168.0.254)

Read

and remove that Ad-Blocker, if the url starts with /.well-known/acme-challenge.

I always manage to overlook that, dumb me.
Here’s the output with sudo:

VirtualHost configuration:
*:80                   timderks.duckdns.org (/etc/apache2/sites-enabled/TimDerks.DuckDNS.org.conf:1)
*:8091                 localhost (/etc/apache2/sites-enabled/TimDerks.DuckDNS.org.conf:7)
*:443                  is a NameVirtualHost
         default server timderks.duckdns.org (/etc/apache2/sites-enabled/TimDerks.DuckDNS.org.conf:54)
         port 443 namevhost timderks.duckdns.org (/etc/apache2/sites-enabled/TimDerks.DuckDNS.org.conf:54)
         port 443 namevhost alskfjasilovnoiwehjfojioIOFJoifhosefjoseifj (/etc/apache2/sites-enabled/TimDerks.DuckDNS.org.conf:138)
                 wild alias *
ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www/html"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex mpm-accept: using_defaults
Mutex watchdog-callback: using_defaults
Mutex proxy-balancer-shm: using_defaults
Mutex rewrite-map: using_defaults
Mutex ssl-stapling-refresh: using_defaults
Mutex ssl-stapling: using_defaults
Mutex proxy: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/var/run/apache2/" mechanism=default
PidFile: "/var/run/apache2/apache2.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
User: name="www-data" id=33
Group: name="www-data" id=33

About the rest:
I know that PiHole puts a page on port 80 on my RPi, but when I try accessing it on its local IP:80, I get a default page from apache. I’ve disabled PiHole, tried again, but still get the same results. The URL of PiHole should definitely not have anything to do with the /.well-known/acme-challenge bit.

I’ve read about all the different challenges, and I (think I) know that the HTTP challenge is the only option available to me, since I’m using DuckDNS. What I’m unsure about is this ACME client. I assume that certbot is the ACME client, which should be putting something at the acme challenge address, but that’s about as far as my guessing can go.

Could you point me to something that shows how I should have my VirtualHost for port 80 setup? I’ve currently got a very simple setup, which probably isn’t doing anything useful, as far as that even matters:

<VirtualHost *:80>
    ServerName timderks.duckdns.org
    ServerAlias www.timderks.duckdns.org
</VirtualHost>

Your port 80 is ok. That

Open Source Ad Blocker Designed for Raspberry Pi

is the problem. That blocks. Disable that tool if it isn’t possible to add an exception.

can you show us this file?

Well that’s the thing: I have disabled it at the moment. I also don’t get that html page when I go to the local IP:80, I get an “Apache2 Debian Default Page”, so I really don’t understand how PiHole is getting in there.

<VirtualHost *:80>
	ServerName timderks.duckdns.org
	ServerAlias www.timderks.duckdns.org
</VirtualHost>

<VirtualHost *:8091>
    DocumentRoot /var/www/SprinklerServer

    ErrorLog ${APACHE_LOG_DIR}/port_8091_error.log
    CustomLog ${APACHE_LOG_DIR}/port_8091_access.log combined

    DirectoryIndex sprinklers.html

    Alias "/signin" "/var/www/SprinklerServer/signin"
    <Directory /var/www/SprinklerServer/signin>
        DirectoryIndex /signin/signin.html
    </Directory>

    Alias "/epochTime/" "/var/www/SprinklerServer/plugins/"

    <LocationMatch "^/$">
        AuthFormLoginRequiredLocation /sprinklers/signin/
        AuthFormLoginSuccessLocation /sprinklers/
        AuthFormProvider file
        AuthUserFile /var/www/SprinklerServer/.htpasswd
        AuthType form
        AuthName "Reserved Area"
       
        Session On
        SessionMaxAge 31540000
        SessionCookieName SPRSID path=/;domain=timderks.duckdns.org;httponly;secure;version=1;
        SessionCryptoPassphrase pt)id:)yO.JEJQqX'Pv?TeyE1v&w5Tl+%9{QvjcpUBBeT`r!Ep#$c_xyrN/gMx:
        require valid-user
    </LocationMatch>

    <Location "/dologin.html">
        SetHandler form-login-handler
        AuthFormLoginRequiredLocation /sprinklers/signin/
        AuthFormLoginSuccessLocation /sprinklers/
        AuthFormProvider file
        AuthUserFile /var/www/SprinklerServer/.htpasswd
        AuthType form
        AuthName "Reserved Area"

        Session On
        SessionMaxAge 31540000
        SessionCookieName SPRSID path=/;domain=timderks.duckdns.org;httponly;secure;version=1;
        SessionCryptoPassphrase pt)id:)yO.JEJQqX'Pv?TeyE1v&w5Tl+%9{QvjcpUBBeT`r!Ep#$c_xyrN/gMx:
    </Location>

</VirtualHost>

<VirtualHost *:443>
    ServerName timderks.duckdns.org

	ProxyErrorOverride On

    # Force traffic over HTTPS (after first connection?)
    Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains;"
    Header always set Content-Security-Policy "default-src 'self'; script-src 'self' https://* 'unsafe-inline'; connect-src 'self'; img-src 'self' data:; style-src 'self' https://* 'unsafe-inline'; base-uri 'self'; form-action 'self'; font-src 'self' fonts.gstatic.com;"
    Header set X-Content-Type-Options nosniff
    Header always append X-Frame-Options DENY
    Header set X-XSS-Protection "1; mode=block"
    Header always set Expect-CT "enforce, max-age=300"
    Header always set Feature-Policy "geolocation 'none'; midi 'none'; camera 'none'; usb 'none'; magnetometer 'none'; accelerometer 'none'; vr 'none'; speaker 'none'; ambient-light-sensor 'none'; gyroscope 'none'; microphone 'none'"
    Header set Access-Control-Allow-Origin "https://timderks.duckdns.org"
    Header set Access-Control-Allow-Credentials true
    Header set Referrer-Policy "same-origin"

    # Add 'path=/; ' to the DMZSID clearing cookie
    Header edit Set-Cookie "(^DMZSID=none;\s?HttpOnly;\s?)(.*)" "$1path=/; $2"

    # Removing cookies from requests to domoticz coming from sprinklers, just to be sure
    RequestHeader unset Cookie "expr=%{HTTP_REFERER} =~ m#.*/sprinklers/.*# && %{REQUEST_URI} =~ m#.*domoticz/json.htm.*#"
    RequestHeader set Test "test" "expr=%{HTTP_REFERER} =~ m#.*/sprinklers/.*# && %{REQUEST_URI} =~ m#.*domoticz/json.htm.*#"

    # Log locations for this port
    ErrorLog ${APACHE_LOG_DIR}/port_443_error.log
    CustomLog ${APACHE_LOG_DIR}/port_443_access.log combined

    # All SSL stuff
    SSLEngine On
    SSLCertificateFile      /etc/letsencrypt/live/timderks.duckdns.org/cert.pem
    SSLCertificateKeyFile   /etc/letsencrypt/live/timderks.duckdns.org/privkey.pem
    SSLCertificateChainFile	/etc/letsencrypt/live/timderks.duckdns.org/chain.pem
    SSLProtocol -all +TLSv1.2 +TLSv1.3
    SSLHonorCipherOrder On

    # Enable stuff
    SSLProxyEngine On
    ProxyPreserveHost On
    ProxyRequests Off
    RewriteEngine On

    # Stuff to get websocket working
    # When Upgrade:websocket header is present, redirect to ws
    # Using NC flag (case-insensitive) as some browsers will pass Websocket
    RewriteCond %{HTTP:Upgrade} websocket [NC]
    RewriteCond %{HTTP:Connection} upgrade [NC]
    RewriteRule \/sprinklers\/node\/(.*) ws://127.0.0.1:8000/$1  [P]

    # Add trailing slash if necessary
    RewriteRule ^/domoticz$ /domoticz/ [R=307]
    RewriteRule ^/sprinklers/node$ /sprinklers/node/ [R=307]
    RewriteRule ^/sprinklers$ /sprinklers/ [R=307]

    # Proxy the requests to the right internal address
    ProxyPassMatch      /domoticz/(.*)          https://127.0.0.1:444/$1 keepalive=On
    ProxyPassReverse    /domoticz/(.*)          https://127.0.0.1:444/$1 keepalive=On
    ProxyPassMatch      /sprinklers/node/(.*)   http://127.0.0.1:8000/$1 keepalive=On
    ProxyPassReverse    /sprinklers/node/(.*)   http://127.0.0.1:8000/$1 keepalive=On
    ProxyPassMatch      /sprinklers/(.*)        http://127.0.0.1:8091/$1 keepalive=On
    ProxyPassReverse    /sprinklers/(.*)        http://127.0.0.1:8091/$1 keepalive=On

    <Location />
    	Deny from all
	</Location>

	<LocationMatch "^/(sprinklers|domoticz|errorpages|misc).*">
		Allow from all
	</LocationMatch>

	DocumentRoot /var/www/SprinklerServer
	ErrorDocument 401 /errorpages/401error.html
	ErrorDocument 403 /errorpages/403error.html
	ErrorDocument 404 /errorpages/404error.html
	ErrorDocument 500 /errorpages/500error.html
	ErrorDocument 501 /errorpages/500error.html
	ErrorDocument 502 /errorpages/500error.html
	ErrorDocument 503 /errorpages/500error.html
	ErrorDocument 504 /errorpages/500error.html
	ErrorDocument 505 /errorpages/500error.html
   
</VirtualHost>

# The bit below prevents access to the domain from any URL other than timderks.duckdns.org
<VirtualHost *:443>
    ServerName alskfjasilovnoiwehjfojioIOFJoifhosefjoseifj
    ServerAlias *
    <Location />
        AllowOverride None
        Order deny,allow
        deny from all
    </Location>
</VirtualHost>

Your main page has the Default page.

Your

http://timderks.duckdns.org/.well-known/acme-challenge/1234

has

Open Source Ad Blocker Designed for Raspberry Pi

Letsencrypt checks /.well-known/acme-challenge/random-filename.

oh, come on, why are people doing this. you can define a default virtualhost in easier ways.

<VirtualHost *:80>
	ServerName _
        DocumentRoot /var/www/html
</VirtualHost>
<VirtualHost *:443>
	ServerName _
        DocumentRoot /var/www/html
        SslEngine On
        # here you need to import a selfsigned cert, snakeoil they call them.
        # maybe they're imported already.
</VirtualHost>

and then all the rest.

I’d replace <VirtualHost *:8091> with <VirtualHost 127.0.0.1:8091> if you want port 8091 only reachable from localhost.

That’s odd, when I tried going to a similar page earlier I got an error page from Domoticz (EDIT: Upon retrying, I get that errorpage when I’m not in incognito mode. When I am in incognito mode, I get the PiHole page. Something to do with cookies probably.). Now I’m indeed getting a PiHole page, but it’s saying that “This page has been whitelisted. Please flush your DNS cache and/or restart your browser”. This whitelist entry has been there for quite some time, so I’m a bit confused as to why it’s still blocking it. Do you also see the notice that the site has been whitelisted?

Because people don’t know everything, people use google, and people find a working solution. For experts, this may look stupid, but for a novice, if it works, it works.

I’m very confused by this part. What’s the reason for ServerName _? Why change DocumentRoot to something I’m not using? Why do I need to import a selfsigned certificate? What do you mean by all the rest? Like I said, for an expert, things may seem obvious(ly wrong), but for novices, things are nowhere near as obvious.

By ‘only reachable from localhost’, do you mean only reachable from the device that’s hosting it, or only on the local network?

to have an invalid servername on purpose, that virtualhost is the default one because it’s the first one.

because you don’t, but certbot might, if you want to use --webroot

to throw errors at people using invalid addresses :wink:

the former.

@9peppe, I’m still very confused about the point of what you’ve described above, so for now, I’m sticking with what I had. That works, while my implementation of your suggestion doesn’t (which I can guarantee is down to my implementation, due to lack of knowledge).

More on topic though: I finally managed to get renewal to work. I did some searching on why PiHole was showing up on that URL. Turned out it did have something to do with PiHole hosting some pages at port 80. I changed the ports for lighttpd from 80 to 800 (for others, be aware that this breaks the pi.hole/admin page, you’ll need to use IP:PORT/admin from now on. It also probably resets on a PiHole update), tried renewing again, and it worked just fine.

I know it’s a patch-job, but for now I’m happy with it. Thanks for your help getting me on the right path, much appreciated.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.