Root domain redirected via DNS to Hetzner, cert issuing woes

I did some additions to Hetzners config yesterday and have rebooted the server now.
Will test a certbot dry run in a bit...

I am most grateful you guys look out for the cert functionality and spot these things.

May I ask what you use to ping IPv6, a web service..perhaps ? I tend to use Pingtools for IPv4, but they have no support for IPv6.
If I use the same I would be more confident in the result.

FYI now the server is down, unable to connect in any way and the site it is hosting is also down.

:nauseated_face:

The server is not down, it is up. Kinda. The OS is running. Awaaaaay... :neutral_face:

The problem seems to be a misconfiguration somehow of IPv6 and I am inclined to believe that somehow the Hetzner network is prioritizing IPv6, since it is available, but neither HTTP nor SSH is able to find the target server. Not even using SSH with the -4 switch can reach it.

When I setup the server I disregarded IPv6 and added it later when troubleshooting LetsEncrypt. So the server has two IP entries now, when two days ago it only had IPv4. That addition of IPv6 also required a manual configuration in a file called /etc/network/interfaces.d/50-cloud-init adding those entries to this - the BOLD ones:

network:
 version: 2
 ethernets:
   eth0:
    addresses:
    **- 2a01:4f8:1c1c:2b42::1/64**
       **routes:**
**       - to: default**
**         via: fe80::1**
     match:
       macaddress: "96:00:03:c8:96:e1"
     dhcp4: true
     set-name: "eth0"

Waiting for some response from the support, but I think that if I remove the AAAA entries from the DNS and also remove the IPv6 IP address from the server the connectivity might work again.

I use a server with both IPv4 and IPv6 support. But, you could try https://letsdebug.net or perhaps SSL Labs: SSL Server Test (Powered by Qualys SSL Labs)

Be sure to be testing or using it regularly. Perhaps your mobile phone uses it when on the cellular network (not wifi). Just go to any "what's my IP" site like: What is my IP address location? Find out here | NordVPN

Note ping uses UDP and not TCP so is not quite the same test as HTTP(s). Often the results will match but this is not certain.

3 Likes

Thanks for those links, I will use them to test more and see if I will be able to make the IPv6 conf work on this server. Or remove it entirely.

Not sure why you mentioned phone though, I do not use my phone for these things. (I do have a very capable Galaxy S24 despite that... )

To look up my IP I can check in Unifi from my UDM Pro... as a matter of fact I think it has not changed for a month or so.

1 Like

I mentioned a phone as a possible way to test IPv6 HTTP(s) connections from the public internet. When using your carrier's network (not wifi) using the methods I described will show what kind of connection you are using.

3 Likes

@MikeMcQ
Ah, yes, sorry, of course that is a possibility. But have comps on other networks and carriers I can test with.

1 Like

Depending on your location, your phone company might not support IPv6 :sob:

1 Like

No, it might not. Which is why I said "possible" and "perhaps"

2 Likes

I'm especially disappointed in the telecom operators in The Netherlands: only 1 out of 3 operators supports IPv6 :confused:

2 Likes

So I fixed it. Well i still need to check if a reliable renewal path is available. I will probably have to remove the IPv6 all together, both DNS entries and the IP in server.

IPv4 seems ok

That was before the IPv6 DNS removal
This is after

All OK!
:ok:
OK

No issues were found with homeworldlore.net. If you are having problems with creating an >SSL certificate, please visit the Let's Encrypt Community forums and post a question there.

(I was not able to insert a second screenshot...

But I will still do a dry run...

root@hwlore-bsa-ubuntu-4gb-nbg1-1:~# sudo certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/homeworldlore.net.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Simulating renewal of an existing certificate for homeworldlore.net

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all simulated renewals succeeded:
  /etc/letsencrypt/live/homeworldlore.net/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Everything is A OK and much thanks for your help, even if some was not LetsEncrypt related.

1 Like

@MikeMcQ
I did run the Qualys test as well and it came up with some errors. Is that something you guys can discuss and perhaps help with here, maybe in a new thread, or maybe you don't have support for that...?

I suppose that depends entirely on the nature of the error(s).
Please more specific about any such error(s).

2 Likes

I see that the cert in use doesn't cover the "www":
SSL Server Test: homeworldlore.net (Powered by Qualys SSL Labs)

2 Likes

Well, I thought to be polite and ask before asking. :slight_smile:

But yes, that might be it

Hmm, I will have to reissue the certbot then...

Also ... wait, what, not it gave me an A. Yesterday it was ... don't know... well a lot less.

ok I guess all is good then.

And only now do I realize I lacked the DNS entry for that.

I had www only, and the domain only but I needed one www . the site. net added also in order to get all to work.

I did not realize that until I reviewed some internal paths where I had a mix of non-www and www... and suddenly got a cert error. :face_with_monocle:

Do I need to redo the certbot now?

Yes, if you want a new cert you must request it with the names you need. Your cert history shows you have gotten certs with various combinations. From https://crt.sh see below pic

At one time you even got a wildcard cert which covered your apex name and its subdomains.

I also note your DNS for some domains (like apex and www) have just an IPv4 address but others (like mail and ftp) have both IPv4 and IPv6

A good place to start is by showing us output of this.

sudo certbot certificates

Your cert history from crt.sh

2 Likes

That is because I redirect only IPv4. Had lots of problems with IPv6, as you see above.

The domain host is also a web host, so that is why they have a default range of entries in the DNS list, not created by me. When I redirect to another host, where I have a VPS, I only redirect the IPv4 address since I do not use mail or any other things but the domain, or sub domain as such. The reason for multiple entries was a couple of test sites, those have mostly been removed, with one prod site and one "mw.homew..." site left.

So why this "mess"? Well, the webhost do not have support for certain functionality as shared web host, that I need for the sites and only seem to be able to run as VPS.
But I am doing some tests to see if that really is the case.

Ah, the output

Found the following certs:
  Certificate Name: homeworldlore.net
    Serial Number: 3729d7236949ec73c388b70b607aa4fb68b
    Key Type: ECDSA
    Domains: homeworldlore.net
    Expiry Date: 2025-02-12 09:43:40+00:00 (VALID: 79 days)
    Certificate Path: /etc/letsencrypt/live/homeworldlore.net/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/homeworldlore.net/privkey.pem
# certbot
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Which names would you like to activate HTTPS for?
We recommend selecting either all domains, or all domains in a VirtualHost/server block.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: homeworldlore.net
2: www.homeworldlore.net
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel):

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
You have an existing certificate that contains a portion of the domains you
requested (ref: /etc/letsencrypt/renewal/homeworldlore.net.conf)

It contains these names: homeworldlore.net

You requested these names for the new certificate: homeworldlore.net,
www.homeworldlore.net.

Do you want to expand and replace this existing certificate with the new
certificate?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(E)xpand/(C)ancel: e
Renewing an existing certificate for homeworldlore.net and www.homeworldlore.net

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/homeworldlore.net/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/homeworldlore.net/privkey.pem
This certificate expires on 2025-02-23.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

Deploying certificate
Successfully deployed certificate for homeworldlore.net to /etc/apache2/sites-enabled/bookstack-le-ssl.conf
Successfully deployed certificate for www.homeworldlore.net to /etc/apache2/sites-enabled/bookstack-le-ssl.conf
Failed redirect for homeworldlore.net
Unable to set the redirect enhancement for homeworldlore.net.

NEXT STEPS:
- The certificate was saved, but could not be installed (installer: apache). After fixing the error shown below, try installing it again by running:
  certbot install --cert-name homeworldlore.net

Unable to find corresponding HTTP vhost; Unable to create one as intended addresses conflict; Current configuration does not support automated redirection
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

I guess I got the errors because I chose to "expand" so the original non-www entry did not renew.

# certbot certificates
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Found the following certs:
  Certificate Name: homeworldlore.net
    Serial Number: 3171f700c6a2a57c786f4118c96d803ef02
    Key Type: ECDSA
    Domains: homeworldlore.net www.homeworldlore.net
    Expiry Date: 2025-02-23 05:54:23+00:00 (VALID: 89 days)
    Certificate Path: /etc/letsencrypt/live/homeworldlore.net/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/homeworldlore.net/privkey.pem

I am not sure what you mean by that. You now have a cert with both names and your server is using that so all looks good. Do you still have a problem?

I think your earlier problems were your incorrect IPv6 configuration. You have since removed the AAAA record.

"Renew" is a confusing term. We use it with different meanings. Technically, no cert is "renewed". Once a cert is issued it is never changed. It continues to exist "forever" but will eventually expire. You can see your cert history at a site like https://crt.sh

We often say "renew" to mean you got a new cert with the identical set of domain names in it. We also say renew when talking about the Certbot "renew" command.

3 Likes