Debugging Augeas (the library responsible for parsing the Apache configuration files) is certainly possible, but would entail putting all kinds of debug statements in the Python code and see what’s going on. Without an Apache configuration which can reproduce this error, that’s hard to do…
I can try to trick certbot by generating a self signed certificate for your domains, so it won’t try to get new ones, but will try to generate a new TLS VirtualHost if it sees it hasn’t been installed yet. Not sure if certbot gets tricked by self signed certs though
I can post a tarball of sites-enabled/, but that’s quite a lot of generosity for a reading assignment.
It seems to get a certificate, it just doesn’t know where to install it. The certificate is there and is seen by runs of certbot-auto; it’s just not fully installed enough to serve the site via HTTPS.
That's why I'm hoping certbot accepts a self signed cert so I can test the installation part. Hopefully I'll get an error with your VirtualHost too and see where it goes wrong.
Ha! Success! At least, I’m getting the same error as you Had to hack my certbot just a tiny little bit (comment out some check I hope I can find back and un-comment it…)
No, this is actually very, VERY simple debugging… Just add breakpoint() after every line in the function, do a print(variable) statement on the debugger command line (where variable is the variable you want to check, such as new_matches from the Python code) and you compare the output of those variable prints to a configuration you know is working.
I noticed new_matches returns a single Augeas reference to the newly generated VirtualHost with a working Apache configuration, but returned empty with this VirtualHost. So I just removed something from the erroneous configuration file and checked if it returned a “match” that time. And kept removing stuff, until I finally DID get a match! That was after I removed all the trailing comments… Then I added the comments back again and tried to remove only pieces of it… But it kept failing, until I removed all the indentation. Then I tried a few scenario’s with #Foo and #Bar and stuff and noticed the indentation was key to the failing Augeas library.
As you can see, fairly simple and the timestamps of the posts say it can be done within 20 minutes
The “tricky” part was to keep it as “real” as possible, I wanted certbot to accept a self signed certificate with @CJSHayward s host, so I could use his configuration file without editing it, but that was easily done by commenting out this line.
...then something horrifying. Glad it worked though. I've found when editing PHP files that I sometimes have to backspace and recreate newlines when copying things to get rid of the extra characters. Sometimes native nbsp and other whitespace gobbledygook interferes. My historical favorite was forgetting to remove an unneeded parameter to a printf statement that had no corresponding place in the format string. I spent four hours commenting out chunks of code while assuming "nah, couldn't be the printf" until finally I showed it to a colleague. As soon as he pulled it up on his screen, my eyes immediately fell on the printf and I slapped myself in the forehead.
That wasn’t supposed to be an insult to you. Your approach was very effective and discovered one of the more elusive types of bugs. My face was because what you went through reminded me of many of my own experiences of commenting and if(0)'ing chunks of code like a mad surgeon looking for the cause of something crazy. I found it quite funny. I suppose python can be pretty picky about its indentation. I’ve only used it to any extent for network flow / optimization problems.
By the by, on another topic you mentioned my time being an issue with a problem I was identifying. I mentioned Samsung Internet and you suggested it might have been updated. You may very well be right. I explored it further and found that Samsung internet now suggests changing your time when it encounters an outdated certificate. I assumed that the cert in question was a fake that was set to start in the future, which I’ve seen for a few placeholders. Previously that browser simply stated that the cert was outdated (or had invalid dates if it starts after it ends). Its behavior now is really deceptive and annoying.
Had to hack my certbot just a tiny little bit (comment out some check I hope I can find back and un-comment it…)
Glad it worked though. I've found when editing PHP files that I sometimes have to backspace and recreate newlines when copying things to get rid of the extra characters. Sometimes native nbsp and other whitespace gobbledygook interferes. My historical favorite was forgetting to remove an unneeded parameter to a printf statement that had no corresponding place in the format string. I spent four hours commenting out chunks of code while assuming "nah, couldn't be the printf" until finally I showed it to a colleague. As soon as he pulled it up on his screen, my eyes immediately fell on the printf and I slapped myself in the forehead.
Your approach was very effective and discovered one of the more elusive types of bugs. My 
face was because what you went through reminded me of many of my own experiences of commenting and if(0)'ing chunks of code like a mad surgeon looking for the cause of something crazy. I found it quite funny. 
I suppose python can be pretty picky about its indentation. I’ve only used it to any extent for network flow / optimization problems.
Props to