.well-known directory deleted causing renew problem

My domain is: smoothgift.com

I can login to a root shell on my machine (yes ):

I'm using a control panel to manage my site (no):
see below for more details

My certificates are due to expire 3/10/2020. I've received email notification to that effect.

Operating System is Linux Opensuse Leap 15.1, Web Server is Apache, e-commerce software is Oscommerce phoenix, certbot version 1.4.0, DNS reg with Godaddy.

I have reinstalled the certs a few times in the past with
"sudo certbot -1 apache -w /srv/www/htdocs -d www.smoothgift.com -d smoothgift.com" without error.

A couple of weeks ago, I changed IP address and had to reinstall Oscommerce Phoenix.
Site DNS is good. But I forgot about Letsencrypt's .well known directory and deleted it along with the old Oscommerce Phoenix files (I think :thinking: ).
I also cannot remember the password I used in setting up the certbot certifcates.

What I would like is to be able to reinstall certificates and use a new password in the process.
Is this possible? Or may be wait for 3/10/2020 (certs expiration) and then issue new ones?
Other than these, what can I do to resolve this problem please?

Thank you

2 Likes

Hi @Rich64

read your output: There is no answer.

A working port 80 / http is required if you want to use http validation.

Works http internal?

curl http://smoothgift.com/

from that machine?

That's the first problem you have to fix.

1 Like

Thanks or your response.
The .well-known directory does not exist, was deleted. Isn't that why fetching it timed out?
Http port 80 is open which is why it could be redirected to https:443 or not? Do you mean that I have to stop redirect first to validate?

2 Likes

Yes, it does now.
And port 80 is open.

What I would like is to be able to reinstall certificates and use a new password in the process.
Is this possible? Or may be wait for 3/10/2020 (certs expiration) and then issue new ones?
Other than these, what can I do to resolve this problem please?

Thank you

2 Likes

But I can't load your domain with my browser. Same with https://check-your-website.server-daten.de/?q=smoothgift.com

Domainname Http-Status redirect Sec. G
http://smoothgift.com/ 86.86.183.62 -14 10.044 T
Timeout - The operation has timed out
http://www.smoothgift.com/ 86.86.183.62 -14 10.027 T
Timeout - The operation has timed out
https://smoothgift.com/ 86.86.183.62 -14 10.047 T
Timeout - The operation has timed out
https://www.smoothgift.com/ 86.86.183.62 -14 10.027 T
Timeout - The operation has timed out

So if it works internal, your router is wrong, you have a blocking firewall or you have regional filters.

Or your ISP blocks port 80.

Completely unrelevant.

1 Like

Beats me! Well, I have now taken it off permanent redirect. Please let me know if site is still not loadable.

So permanent redirect is a good practice, I read and moreso, I need to have site redirected permanently to https for obvious reasons. Now that I have both http and https available, does that mean that I will always have to leave http and https available in order for certbot to be able to renew/reissue/reinstall certificates? Do I understand that correctly? Please advise.

I have looked into these but found non of them to be the case. Is there something else I could look into to help clarify these assumptions please?

Could you please explain why this is irrelevant?
Like I explained, my reason for wanting a new certificate is to be able to have a password that can be used when exporting/deploting letsencrypt certificate to be used elsewhere other than the webroot-path. I cannot do this at the moment. What's the way out?

Many thanks for your help.

2 Likes

Please use online tools to check that. It's your job to have a working configuration. No, it doesn't work.

You have a timeout, that's relevant.

1 Like

Hello Rich :slightly_smiling_face:

I too am unable to access your site in any of these ways:
http://smoothgift.com
http://www.smoothgift.com
https://smoothgift.com
https://www.smoothgift.com

I tried quite a few different tools, but without success. Are you able to access your site from a mobile device? It looks like you were able to successfully get certificates a few months ago (https://crt.sh/?q=smoothgift.com). In changing your IP address did you also change hosting providers?

To clarify, you need ports 80 (http) and 443 (https) open and operating in order for the addresses above to function. You can (and should) have a redirect from http to https (and from non-www to www or vice versa). That applies to having your website function correctly, not just for getting a certificate from Let's Encrypt.

To which password do you refer? The private encryption key for the certificate perhaps?

For your certbot command, I think you may have some typos. I have cleaned it up and enhanced it based on what I believe you want:
sudo certbot run --cert-name smoothgift.com -a webroot -w /srv/www/htdocs -d smoothgift.com,www.smoothgift.com -i apache

Edit:
You could always use dns-01 challenges instead of the http-01 challenges you've been using since your DNS records are publicly visible. It would start out as a manual process, but it could probably be automated later with something like acme-dns. Here's the command, which will require you to manually create two TXT records in your DNS:
sudo certbot run --cert-name smoothgift.com -a manual --preferred-challenges dns -d smoothgift.com,www.smoothgift.com -i apache

1 Like

Hello Griffin :slightly_smiling_face:

Totally my bad to have assumed that site was loadable externally without checking it outside of it's internal IP range. Related telnet, netstat etc commands were telling me that ports were open and I did not hang around long enough to see that connections were interrupted and closed like 20 secs afterwards by foreign host. Culprit was of course my modem port forwarding. Sorted out now. Ports are open.

griffin, I cannot thank you enough!

Also, I appreciate your efforts in clarifying things for me and pointing me in the right direction.

Using the dns-01 challenges would have been the better way to go now except I would need to read up first on acme-dns to know how to automate renewal (unless you have a lead for me there). So I will stick to the http-01 challenges (compliments of cronjob for renewal) but do not know how to go about the password issue. This still leaves me unable to answer your questions "To which password do you refer? The private encryption key for the certificate perhaps?" I am talking about the password you have to give ONLY the FIRST time you requested Certbot for certificates & key. For example, if I would like to use certificates issued by Certbot for Pentaho server, it would need to be implemented via openssl keystore. The keystore of course will request for the password used in creating the Certbot certficate. Hope you understand the 'password' I'm going on about now. I have forgotten the password so I need brand new certificates. If I request for a new certificate now Certbot will just give me one but will not ask me for a password input which I need.

I guess I could put back the http to https redirect as it would not be in the way of Certbot reissue/renewal.

Many thanks griffin!

3 Likes

My apologies Juergen. The ports are open but my modem port forwarding was not working, so technically, you were correct. Site is loadable now I believe.

So how do I deal with my situation now?

Thanks.

2 Likes

Now your check worked - Grade I, not good, but /.well-known/acme-challenge works - https://check-your-website.server-daten.de/?q=smoothgift.com

Good: All checks /.well-known/acme-challenge/random-filename without redirects answer with the expected http status 404 - Not Found. Creating a Letsencrypt certificate via http-01 challenge should work. If it doesn't work: Check your vHost configuration (apachectl -S, httpd -S, nginx -T). Every combination of port and ServerName / ServerAlias (Apache) or Server (Nginx) must be unique. Merge duplicated entries in one vHost. If you use an IIS, extensionless files must be allowed in the /.well-known/acme-challenge subdirectory. Create a web.config in that directory. Content: <system.webServer></system.webServer>. If you have a redirect http ⇒ https, that's ok, Letsencrypt follows such redirects to port 80 / 443 (same or other server). There must be a certificate. But the certificate may be expired, self signed or with a not matching domain name. Checking the validation file Letsencrypt ignores such certificate errors.

What's now the result?

1 Like

The next step would be for you to try to generate a new certificate. :slightly_smiling_face: As for the lost openssl keystore password issue, I'm not really sure. I'll do some research and get back to you there.

2 Likes

I have been able to use both http-01 challenges and dns-01 challenges using the commands given by griffin. Both worked. Certs are now valid till 24/12. I used both option 1 (reinstall) and option 2 (renew) for both commands. That's great! BUT BOTH still did not give me the option to input password which really defeats my original purpose. It makes Letsencrypt issued certs not usable for other applications (using openssl keystore) like Pentaho server, ofbiz server, etc

In case of the dns-01(sudo certbot run --cert-name smoothgift.com -a manual --preferred-challenges dns -d smoothgift.com,www.smoothgift.com -i apache) I did not get a chance to manually create two TXT records in your DNS contrary to expectation but certs were issued though.

Is it safe to conclude now that Letsencrypt's certbot don't do proper reissue or renew (with challenges and password)? Please advise.

Thanks a lot!

1 Like

That's not required if you have an Apache.

That's completely wrong. These applications may need other formats, may be pfx. So use OpenSsl to create the required pfx file. Then you can add a password.

You are mixing things that are completely separated.

1 Like

Hey griffin,

Getting somewhere :slightly_smiling_face: Been able to get certs issued using both of your commands. pls read more about it above @ [JuergenAuer]. Still stuck with the password issue. Don't know what to do as it is. Will hold on though to see if you could come up with a solution. Grateful!

Thanks.

Cheers

3 Likes

Ok. I will go and look into your suggestion. My experience so far is that I would need a password for the keystore itself (like you said) but it wants the password used @ certbot as well to be able to copy certs to keystore which makes sense for security sake. Came across another way to use it directly (without keystore) but the password is also required.

"That's not required if you have an Apache." But I am using apache.

2 Likes

No. If you want a keystore, you use a Bitnami. That's completely different.

And the Bitnami documentation has tons of samples how to add the certificate to the required keystore.

PS: Or another system. But a standard Apache doesn't use a keystore.

1 Like

Ok. Just want letsencrypt. But what can I do to start a fresh request in certbot so that I can get to input challenges answers and password?

1 Like

There is no password. Certbot creates a certificate without a password, nothing else.

If you want to use the certificate files with an additional password, you have to do additional things. That's your thing.

If you don't mind my asking, Rich, can you please describe your password scenario specifically so we can address it in one place. It's not too common around here, so we want to make sure we understand clearly. :slightly_smiling_face:

2 Likes