Subdomain certs failing, TLDs work fine

[paul_schrader]

Hi all, thanks so much for your help.

I've been successfully using LE certs for years using Serverpilot's AutoSSL. I've added many domains and subdomains to a server via the Serverpilot GUI and everything has been happy. These domains are part of a WordPress Multisite install.

Yesterday the subdomains started throwing errors. Deleting them and adding them back in still results in failure. That said, I tested and added in a TLD just fine.

Let'sDebug says everything is fine, but I noticed a 404 for acme-challenge.

Is this a DNS issue? Hover is my registrar and I'm using their name servers.

Any ideas? Thanks!!

crt.sh | madison.prefabpower.com

Test result for madison.prefabpower.com: OK

My domain is: madison.prefabpower.com

I ran this command: n/a -- issuing via Serverpilot AutoSSL

My web server is (include version): Nginx as a reverse proxy in front of Apache

The operating system my web server runs on is (include version): Ubuntu 18.04

My hosting provider, if applicable, is: DO

I can login to a root shell on my machine (yes or no, or I don't know): Yes.

I'm using a control panel to manage my site: no control panel, but Serverpilot for managing the server

[rg305]

Hi @paul_schrader, and welcome to the LE community forum

@paul_schrader

Yesterday the subdomains started throwing errors.

Please provide more specific detail(s) on those errors.

Also, since you are using Apache [somewhere in there], please show us the output of:
sudo apachectl -t -D DUMP_VHOSTS

[MikeMcQ]

Also, have you tried asking ServerPilot support? This sounds more like a config issue of some kind with that product and your servers. They say this on their main page. Sounds promising anyway

Support

Get real answers from our U.S. based support team of hosting industry veterans.

Contact Support - ServerPilot

[paul_schrader]

Ah, yeah. The error, I should have specific, is that the sites weren't loading over HTTPS. Getting a basic NET::ERR_CERT_COMMON_NAME_INVALID -- SSL / Certificate name mismatch

Severpilot installs a custom Apache so apachectl doesn't work, but I pasted in the .conf file from the app, which I think is the same as what you wanted?

Thanks!

<VirtualHost 127.0.0.1:81>
    Define DOCUMENT_ROOT /srv/users/prefabpower/apps/prefabpower/public
    Define PHP_PROXY_URL unix:/srv/users/prefabpower/run/prefabpower.php-fpm.sock|fcgi://loca$

    ServerAdmin webmaster@
    DocumentRoot ${DOCUMENT_ROOT}
    ServerName arrtreads.org
    ServerAlias carolinelibrary.org
    ServerAlias eplva.org
    ServerAlias espl.org
    ServerAlias finditva.com
    ServerAlias hamnerlibrary.org
    ServerAlias heritagepubliclibrary.org
    ServerAlias lynchburgpubliclibrary.org
    ServerAlias madison.prefabpower.com
    ServerAlias oconeelibrary.org
    ServerAlias ocplva.org
    ServerAlias pcplib.org
    ServerAlias prefabpower.com
    ServerAlias rappahannock.prefabpower.com
    ServerAlias sparkpa.org
    ServerAlias sparksupport.prefabpower.com
    ServerAlias support.sparkpa.org
    ServerAlias usablelibrary.org
    ServerAlias www.arrtreads.org
    ServerAlias www.carolinelibrary.org
    ServerAlias www.eplva.org
    ServerAlias www.espl.org
    ServerAlias www.finditva.com
    ServerAlias www.hamnerlibrary.org
    ServerAlias www.heritagepubliclibrary.org
    ServerAlias www.lynchburgpubliclibrary.org
    ServerAlias www.oconeelibrary.org
    ServerAlias www.ocplva.org
    ServerAlias www.pcplib.org
    ServerAlias www.sparkpa.org
    ServerAlias www.usablelibrary.org
    ServerAlias www.wythegrayson.lib.va.us
    ServerAlias wythegrayson.lib.va.us

    ErrorLog "/srv/users/prefabpower/log/prefabpower/prefabpower_apache.error.log"
    CustomLog "/srv/users/prefabpower/log/prefabpower/prefabpower_apache.access.log" common

    RewriteEngine on
    RewriteCond %{HTTP:X-Forwarded-Proto} !=https
    RewriteRule /(.*) https://%{HTTP_HOST}/$1 [R=301,L]

    RemoteIPHeader X-Real-IP
    SetEnvIf X-Forwarded-SSL on HTTPS=on
    SetEnvIf X-Forwarded-Proto https HTTPS=on

    SuexecUserGroup prefabpower prefabpower

    IncludeOptional /etc/apache-sp/vhosts.d/prefabpower.d/*.conf
</VirtualHost>

[paul_schrader]

Hi MikeMcQ -- thanks, yes I did and I did receive a quick reply from them. They suggested this.

This is the error we're getting from Let's Encrypt (this is for madison.prefabpower.com, the error is the same for the other domain):

DNS problem: SERVFAIL looking up CAA for madison.prefabpower.com - the domain's
nameservers may be malfunctioning

We're not seeing any errors when querying CAA records for these subdomain or the registered domains. -- Checking both is relevant because, for CAA records, if there's no CAA record for the exact domain, querying continues up each level to the registered domain.

I contacted Hover and they passed the buck to LE. So now I'm here.

[MikeMcQ]

Oh, that CAA issue is definitely a problem. I can easily reproduce it using https://unboundtest.com (which looks up DNS like Let's Encrypt does).

Your base name works fine prefabpower.com but the madison subdomain of that does not. You can try both names on unboundtest yourself to see the SERVFAIL

It also only fails using unbound 1.18 which was a recent change in Let's Encrypt servers. Using 1.16 gave a proper response.

The TL;DR is yes you need to contact Hover and have them fix the SERVFAIL. Even show them the unboundtest site.

I don't know DNS as well as others here (such as @rg305) but that domain fails at a
EDNS test site that we often use. This is likely the reason for the SERVFAIL

https://ednscomp.isc.org/ednscomp/b2c16ffef4

[petercooperjr]

Can also see the error on DNSViz (once you use the advanced options to check the CAA record)

https://dnsviz.net/d/madison.prefabpower.com/dnssec/?rr=257&a=all&ds=all&ta=.&tk=

madison.prefabpower.com/CAA (NODATA): No SOA RR was returned with the NODATA response. (64.98.148.13, 216.40.47.26, UDP_-_EDNS0_4096_D_KN)

This is a bug in the DNS server software being used. (And something that the newer Unbound that Let's Encrypt is using now seems to be pickier about.)

[rg305]

@paul_schrader

    RewriteEngine on
    RewriteCond %{HTTP:X-Forwarded-Proto} !=https
    RewriteRule /(.*) https://%{HTTP_HOST}/$1 [R=301,L]

That HTTP config redirects everything to HTTPS.
Do you have that config?

Actually, it doesn't seem like a server config issue - more like a DNS issue (at this time)
Let's put this search on hold for now.

[rg305]

Have you thought about adding some DNS redundancy to your domain?:
[did you even know that you could do that?]

prefabpower.com nameserver = ns1.hover.com
prefabpower.com nameserver = ns2.hover.com

Where those two nameservers only resolve to two IPv4 addresses:

ns1.hover.com   internet address = 216.40.47.26
ns2.hover.com   internet address = 64.98.148.13

[paul_schrader]

Thanks thanks thanks. I have initiated with Hover again and will report back in case anyone else has this issue.

Seems like this is something that should be doable so hopefully they have a fix of other solution.

[paul_schrader]

Well, Hover does not support CAA records. And I guess if things with LE changed recently, that explains why my subdomains failed.

So the solution is to transfer the domain to a different registrar, or, perhaps less severe of a move, use a third party like Cloudflare for DNS.

Thanks all.

[MikeMcQ]

They don't have to support caa records although they should anyway. But they need to reply with an appropriate 'not found'. They are not.

They also only have the two ip addresses and they do not support IPV 6 for their DNS servers. All in all this may be a good time to switch to a more modern and capable dns provider

[webprofusion]

Adding cross reference to other post that mentions hover.com DNS issues - CAA requests resulting in SERVFAIL since Dec 12th - #11 by webprofusion