I have an apache server with a fairly complex setup and several subdomains. Certbot used to work when getting the certificate with the --apache plugin and when renewing once.
The second renewal now fails. The challenges get a 404 for all domains. So it seems that the rewrite rules for .well-known are not active in my apache.
I started certbot with --debug-challenges and stopped it. The pre and post files are created in /etc/apache2.
Now I don’t know how to continue looking for the problem. I could not find the point where the pre/post files are inserted into the Apache config. Where is this supposed to happen?
Or do you have any suggestions on how to continue analysis?
i considered using the --webroot plugin instead, but because of the considerable number of subdomains with different directories that would be rather difficult to get right.
The changes are added (and then removed) within the port 80 VirtualHost that is identified for each domain.
One way this can screw up is if you have port 80 VirtualHosts that are duplicates for a single ServerName. Certbot just uses the first one that seems to match, which is sometimes not the one that is active when Apache responds to that domain.
Posting the output of this command here can help identify this one:
apachectl -t -D DUMP_VHOSTS
The second way it can fail is if the VirtualHost configuration itself is incompatible with the change that Certbot makes. This is usually not the case, but can happen with some mod_proxy or mod_authz_* configurations and other weirdness. Or really wild redirects.
That's a little harder to discern, if you can post the full contents of a port 80 VirtualHost that exhibits this problem, we can try to reproduce it.
I have 7 vhosts listed for port 80. Do I understand you correctly that I should find changes by --apache plugin in the first one, which is named “default” in the output of apachectl?
Thank you. I have 7 individual hosts. I found the includes in the first vhost, which was marked as "default". So the mechanism bascially works.
I have tried many times now. There are 14 challenges for subdomains in total. It noted that not all challenges fail all the time. Most of the times, the list of fails is so long that it looked like all failed. But many times, a large part succeeds and only 2 or 3 of the challenges fail. And it's not the same ones either.
What might cause this mixed behaviour? Could it be a timing problem? That the apache server has not fully loaded the changed configs so part of them fail with a 404? i would not assume that the rewrite engine works so erratically?
When only a few fail, they are usually subdomains out of the main vhost which is pretty low on the list output by apachectl. But again, not all subdomains, but a seemingly random selection. (Couldn't try the theory exhaustively because I keep running into the rate limit. )
)