Trying to renew expired certificate

[mruga]

Hello everyone,

After browsing through most of the forum topics related to my issue, and not being able to figure out a solution for my exact problem (or my exact problem), I have decided to make this post! All help is greatly appreciated.

My domain is:
mru9aksheepalwe.com

My subdomain is:
finalproject.mru9aksheepalwe.com

I ran this command:
sudo certbot --force-renewal -d mru9aksheepalwe.com -d finalproject.mru9aksheepalwe.com

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
tls-sni-01 challenge for finalproject.mru9aksheepalwe.com
http-01 challenge for mru9aksheepalwe.com
Waiting for verification…
Cleaning up challenges
Failed authorization procedure. mru9aksheepalwe.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://mru9aksheepalwe.com/.well-known/acme-challenge/b3XfZ1BP0avgOuV2oPBr8Nne4diFf5BV77ljc3gajGY: “\n\n404 Not Found\n\n

Not Found

\n<p”

IMPORTANT NOTES:

• The following errors were reported by the server:

  Domain: mru9aksheepalwe.com
  Type: unauthorized
  Detail: Invalid response from
  http://mru9aksheepalwe.com/.well-known/acme-challenge/b3XfZ1BP0avgOuV2oPBr8Nne4diFf5BV77ljc3gajGY:
  “\n\n404 Not
  Found\n\n

  Not Found

  \n<p”

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.

My web server is (include version):
Apache/2.4.18 (Ubuntu)

The operating system my web server runs on is (include version):
Ubuntu 16.04

My hosting provider, if applicable, is:
domain host: fatcow
server host: AWS

I can login to a root shell on my machine (yes or no, or I don’t know):
Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):
No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):
certbot 0.26.1

[JuergenAuer]

Hi @mruga

@mruga

Domain: mru9aksheepalwe.com
Type: unauthorized
Detail: Invalid response from
http://mru9aksheepalwe.com/.well-known/acme-challenge/b3XfZ1BP0avgOuV2oPBr8Nne4diFf5BV77ljc3gajGY:
“\n\n404 Not
Found\n\n

I see, you have already tested your domain ( https://check-your-website.server-daten.de/?q=mru9aksheepalwe.com ).

The important things are ok. Your port 80 answers, your non-www has no redirect, your finalproject has one (http -> https), but that shouldn't be problem. non-www and finalproject have the same ip. Your www has another ip address and a different server header, but you don't request one certificate with both domain names, so this isn't relevant.

Please check your config to find your DocumentRoot in your VirtualHost. Then use it.

certbot certonly -a webroot -w yourDocumentRoot -d mru9aksheepalwe.com -d finalproject.mru9aksheepalwe.com --preferred-challenges http

If that works, use

certbot run -a webroot -i apache -w yourDocumentRoot -d mru9aksheepalwe.com -d finalproject.mru9aksheepalwe.com --preferred-challenges http

to create a new certificate.

[rg305]

As suggested, the --webroot method could be the simplest/quickest way to obtain a cert.
But that may mask an underlying “problem”:
The certbot Apache plugin is unable to correctly parse and modify the config to handle the authentication requests.

You can choose to ignore that and go on with “business-as-usual”.
Or take a moment to investigate a little into how/why this is happening.

My most likely guesses are:

• There are overlapping ServerName/ServerAlias (active or even commented out).
• There are VirtualHosts nested in unhandled <If"Something"> blocks.
• There are overriding/confusing rewrite statements (active or even commented out).

[there may be other reasons and/or multiple reasons - but are worth looking into IMHO]

[mruga]

Thank you @JuergenAuer
The commands worked perfectly, and now I'm wondering what part of the problem I was missing.

Yes, I did indeed run some of the debug tests but couldn't interpret their results the way you did:

The important things are ok. Your port 80 answers, your non-www has no redirect, your finalproject has one (http -> https), but that shouldn’t be problem. non-www and finalproject have the same ip. Your www has another ip address and a different server header, but you don’t request one certificate with both domain names, so this isn’t relevant.

Thank you for this!

Now I will work on understanding why it worked as @rg305 suggested!

[mruga]

Thank you for your response to this thread @rg305. I am definitely interested in investigating why/how the -webroot solution worked.

I don't understand what this means! Could you please paraphrase:

The certbot Apache plugin is unable to correctly parse and modify the config to handle the authentication requests.

Why is the process of renewing a certificate after its expiry so complex? The solution that worked was not related to what the error was giving me. I was under the impression that my acme challenges weren't found/generated.

P.S. I do not know what a challenge is (going to google it now). But I thought certbot couldn't find the right file, hence the 404.

What piece of knowledge am I missing?

Thank you for you time!

[JuergenAuer]

Happy to read that it have worked.

Rechecking your subdomain there is now the new certificate with both domain names:

CN=mru9aksheepalwe.com
	01.02.2019
	02.05.2019
	finalproject.mru9aksheepalwe.com, mru9aksheepalwe.com - 2 entries

@mruga

The commands worked perfectly, and now I’m wondering what part of the problem I was missing.

The switch to webroot is the key. There are some configurations the apache plugin doesn't understand. But every running webserver has a webroot, so that should always work.

[JuergenAuer]

@mruga

Yes, I did indeed run some of the debug tests but couldn’t interpret their results the way you did:

Yep, sometimes I think it's time to create a help page. The tool is new, startet end 2018-10, now it's growing. And there are a lot of different things, some not important if someone wants to create a new certificate.

[rg305]

The only reason this happened:

@mruga

The client lacks sufficient authorization :: Invalid response from http://mru9aksheepalwe.com/.well-known/acme-challenge/b3XfZ1BP0avgOuV2oPBr8Nne4diFf5BV77ljc3gajGY: “\n\n404 Not Found\n\n

is because

@rg305

The certbot Apache plugin is unable to correctly parse and modify the config to handle the authentication requests.

If it had been able to properly handle it, you would have got a cert using that plugin.

@mruga

Plugins selected: Authenticator apache, Installer apache

[mruga]

I see, so if I understand correctly, does that mean there needs to be an update to certbot’s Apache plugin in order for it to handle the authentication request?

[rg305]

Perhaps an update could make it “smarter”.
But from my experience, Apache is all too forgiving.
It will attempt to run at all cost.
So it tends to be very lenient and allows things that sometimes really just don’t make any sense.
That said, I can’t be certain either way - I have not seen your entire Apache config (nor any part of it).
So you may have a perfectly correct config and the plugin is to blame.
But I would NOT bet on that.