Certbot goes through all of the existing lineages to determine whether there is one that already covers toastmastervoice.com, or whether it needs to create a new lineage.
While it is going through the lineages, it finds that some of them are broken (config file is bad, or symlinks are messed up, or whatever). When Certbot encounters a broken lineage, it logs the error, skips it and moves onto the next one.
Finally, it reaches the end, and it hasn’t found a matching lineage, so it decides to create a new one for toastmastervoice.com. In this log, it does find an existing one, and asks the user whether they want to re-issue the existing certificate or re-install it (the latter choice is made).
At this point, the problems with the other lineages are probably quite irrelevant. We’re not using them, we don’t care about them. One broken lineage isn’t going to (at least, doesn’t IME) make the rest of Certbot blow up.
Ah. Thanks for the great explanation. Makes sense. This is why I’m a big fan of divorcing the various aspects of the process from one another. I guess in my eyes I’d rather deal with a different machine (with a known-good configuration) handling the transaction process and clerical work. I’ve seen too many cases lately of configuration changes made by certbot conflicting with particular setups. Not that that’s necessarily what has happened in this topic. Guess I feel like there’s a difference between getting certificates properly issued and configuring a system to properly use https.
Things don’t work out in the following certbot function:
I don’t actually fully understand the code exactly, but perhaps (and that’s a big “perhaps”) it’s because the VirtualHost includes three ServerAlias options which are not included in the command you’ve used? You only specified toastmastervoice.com while the VirtualHost also specifies www.toastmastervoice.com, toastmastersvoice.com and www.toastmastersvoice.com.
You could try including all four hostnames (with four times the -d option).
I haven’t had ServerAlias cause difficulties before, and I don’t really suspect it now.
I tried, for a sanity check, to comment out all the ServerAliases leaving only the ServerName entry. The behavior when I tried to reinstall the existing certificate (instead of the #2 option of Request and Renew) seemed to be identical.
I might be able to get the desired behavior by adapting and installing the certificate according to the pattern of the other sites-enabled entries, but that’s a losing solution.
Debugging Augeas (the library responsible for parsing the Apache configuration files) is certainly possible, but would entail putting all kinds of debug statements in the Python code and see what’s going on. Without an Apache configuration which can reproduce this error, that’s hard to do…
I can try to trick certbot by generating a self signed certificate for your domains, so it won’t try to get new ones, but will try to generate a new TLS VirtualHost if it sees it hasn’t been installed yet. Not sure if certbot gets tricked by self signed certs though
No, this is actually very, VERY simple debugging… Just add breakpoint() after every line in the function, do a print(variable) statement on the debugger command line (where variable is the variable you want to check, such as new_matches from the Python code) and you compare the output of those variable prints to a configuration you know is working.
I noticed new_matches returns a single Augeas reference to the newly generated VirtualHost with a working Apache configuration, but returned empty with this VirtualHost. So I just removed something from the erroneous configuration file and checked if it returned a “match” that time. And kept removing stuff, until I finally DID get a match! That was after I removed all the trailing comments… Then I added the comments back again and tried to remove only pieces of it… But it kept failing, until I removed all the indentation. Then I tried a few scenario’s with #Foo and #Bar and stuff and noticed the indentation was key to the failing Augeas library.
As you can see, fairly simple and the timestamps of the posts say it can be done within 20 minutes
The “tricky” part was to keep it as “real” as possible, I wanted certbot to accept a self signed certificate with @CJSHayward s host, so I could use his configuration file without editing it, but that was easily done by commenting out this line.
...then something horrifying. Glad it worked though. I've found when editing PHP files that I sometimes have to backspace and recreate newlines when copying things to get rid of the extra characters. Sometimes native nbsp and other whitespace gobbledygook interferes. My historical favorite was forgetting to remove an unneeded parameter to a printf statement that had no corresponding place in the format string. I spent four hours commenting out chunks of code while assuming "nah, couldn't be the printf" until finally I showed it to a colleague. As soon as he pulled it up on his screen, my eyes immediately fell on the printf and I slapped myself in the forehead.
That wasn’t supposed to be an insult to you. Your approach was very effective and discovered one of the more elusive types of bugs. My face was because what you went through reminded me of many of my own experiences of commenting and if(0)'ing chunks of code like a mad surgeon looking for the cause of something crazy. I found it quite funny. I suppose python can be pretty picky about its indentation. I’ve only used it to any extent for network flow / optimization problems.
By the by, on another topic you mentioned my time being an issue with a problem I was identifying. I mentioned Samsung Internet and you suggested it might have been updated. You may very well be right. I explored it further and found that Samsung internet now suggests changing your time when it encounters an outdated certificate. I assumed that the cert in question was a fake that was set to start in the future, which I’ve seen for a few placeholders. Previously that browser simply stated that the cert was outdated (or had invalid dates if it starts after it ends). Its behavior now is really deceptive and annoying.