Why can't I certbot -d toastmastervoice.com and create a new VirtualHost and certificate?

Yeah, it was a wild guess… :roll_eyes:

Debugging Augeas (the library responsible for parsing the Apache configuration files) is certainly possible, but would entail putting all kinds of debug statements in the Python code and see what’s going on. Without an Apache configuration which can reproduce this error, that’s hard to do…

I can try to trick certbot by generating a self signed certificate for your domains, so it won’t try to get new ones, but will try to generate a new TLS VirtualHost if it sees it hasn’t been installed yet. Not sure if certbot gets tricked by self signed certs though :stuck_out_tongue:

2 Likes

I can post a tarball of sites-enabled/, but that’s quite a lot of generosity for a reading assignment.

It seems to get a certificate, it just doesn’t know where to install it. The certificate is there and is seen by runs of certbot-auto; it’s just not fully installed enough to serve the site via HTTPS.

2 Likes

Quite.

That’s why I’m hoping certbot accepts a self signed cert so I can test the installation part. Hopefully I’ll get an error with your VirtualHost too and see where it goes wrong.

2 Likes

Ha! Success! At least, I’m getting the same error as you :smiley: Had to hack my certbot just a tiny little bit (comment out some check I hope I can find back and un-comment it…)

Now time for some debugging…

2 Likes

OK, I’ve figured it out:

The Augeas parser doesn’t like indented comments at the end of the configuration file…

For example, this is fine (for clarity, I left out the top most of the file):

(...)
</VirtualHost>
#Foo

An extra blank line doesn’t matter, this is also fine:

(...)
</VirtualHost>

#Foo

However, when you try adding spaces or tabs, it doesn’t like that! These configuration files return the error!

(...)
</VirtualHost>
 #Foo

(Notice the subtle space before the comment.)

(...)
</VirtualHost>

     #Foo

Not good too!

So: remove the trailing comments (not required anyway) and you’re good to go!

4 Likes

That’s some awesome debugging. :smiley:

1 Like

No, this is actually very, VERY simple debugging… :stuck_out_tongue: Just add breakpoint() after every line in the function, do a print(variable) statement on the debugger command line (where variable is the variable you want to check, such as new_matches from the Python code) and you compare the output of those variable prints to a configuration you know is working.

I noticed new_matches returns a single Augeas reference to the newly generated VirtualHost with a working Apache configuration, but returned empty with this VirtualHost. So I just removed something from the erroneous configuration file and checked if it returned a “match” that time. And kept removing stuff, until I finally DID get a match! That was after I removed all the trailing comments… Then I added the comments back again and tried to remove only pieces of it… But it kept failing, until I removed all the indentation. Then I tried a few scenario’s with #Foo and #Bar and stuff and noticed the indentation was key to the failing Augeas library.

As you can see, fairly simple and the timestamps of the posts say it can be done within 20 minutes :smiley:

The “tricky” part was to keep it as “real” as possible, I wanted certbot to accept a self signed certificate with @CJSHayward s host, so I could use his configuration file without editing it, but that was easily done by commenting out this line. :stuck_out_tongue:

I’ve opened an issue on the certbot Github.

3 Likes

Looks like a good start…

…then something horrifying. :nauseated_face: Glad it worked though. I’ve found when editing PHP files that I sometimes have to backspace and recreate newlines when copying things to get rid of the extra characters. Sometimes native nbsp and other whitespace gobbledygook interferes. My historical favorite was forgetting to remove an unneeded parameter to a printf statement that had no corresponding place in the format string. I spent four hours commenting out chunks of code while assuming “nah, couldn’t be the printf” until finally I showed it to a colleague. As soon as he pulled it up on his screen, my eyes immediately fell on the printf and I slapped myself in the forehead.

:face_with_raised_eyebrow:

1 Like

@Osiris

That wasn’t supposed to be an insult to you. :smile: Your approach was very effective and discovered one of the more elusive types of bugs. My :nauseated_face: face was because what you went through reminded me of many of my own experiences of commenting and if(0)'ing chunks of code like a mad surgeon looking for the cause of something crazy. I found it quite funny. :joy: I suppose python can be pretty picky about its indentation. I’ve only used it to any extent for network flow / optimization problems.

By the by, on another topic you mentioned my time being an issue with a problem I was identifying. I mentioned Samsung Internet and you suggested it might have been updated. You may very well be right. I explored it further and found that Samsung internet now suggests changing your time when it encounters an outdated certificate. I assumed that the cert in question was a fake that was set to start in the future, which I’ve seen for a few placeholders. Previously that browser simply stated that the cert was outdated (or had invalid dates if it starts after it ends). Its behavior now is really deceptive and annoying.

1 Like

Wow. Thanks!

This resolved my issue completely.

3 Likes

Glad everything worked out. :slightly_smiling_face: Props to @Osiris for his diligent digging and debugging.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.