Certificate generation failing because of 404 error

Help
1

My domain is: mydomain[.]com (mydomain is just an example)

I ran this command: sudo certbot certonly --webroot -d mydomain[.]com -d www.mydomain[.]com -w /home/xxx/public_html/ROOT

It produced this output: It's returning 404. http://www[.]mydomain[.]com/.well-known/acme-challenge/5wzAPQb1WKUHfnW4DNSdqMPYi7srmPvBhkg7fckQid8: 404

My web server is (include version): Tomcat

The operating system my web server runs on is (include version): Linux

My hosting provider, if applicable, is: NA

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 2.5.0

I have a struts2 application deployed inside tomcat. When I run the certbot command given above, it gives me 404 error which fails the certificate generation. Though I can see the directory(.well-known/acme-challenge) get's created at the time of running the command but it still returns 404. After doing some search what I found is, the link http://www[.]mydomain[.]com/.well-known/acme-challenge/5wzAPQb1WKUHfnW4DNSdqMPYi7srmPvBhkg7fckQid8 is pointing to a file which doesn't have an extension. If I create a similar directory but with extension(for eg http://www[.]mydomain[.]com/.well-known/acme-challenge/5wzAPQb1WKUHfnW4DNSdqMPYi7srmPvBhkg7fckQid8.txt) then I can download the file but without file extension it's giving me 404 status even on local as well. Could anybody please help me on this. Let me know if any other info needed.

2

Welcome @VineetK

Maybe some other volunteer will know what is wrong with Tomcat. We see Tomcat problems here regularly but I have never seen a problem with a missing extension.

You should try a Tomcat forum or stackoverflow and ask why Tomcat cannot serve a file that has no extension.

As a test just make any test file without an extension and try retrieving it. Just put this test file in your webroot folder. That way you eliminate any question of it being related to Let's Encrypt.

If you have any comms device in front of nginx that might be inspecting URLs you should check that too. Do you see the HTTP Challenge requests show up in the Tomcat access logs as 404? Maybe Tomcat isn't the place where it is being rejected.

3 Likes
3

That is expected to not have an extension.

2 Likes
4

Thank you for the response @MikeMcQ . I did some local test by creating a file without extension and then tried to access it, It's showing 404 error. While if i give the extension to filename, then it works fine.

The access log was not showing any 404 error for this challenge but in catalina.out log it was showing some error like below.

17-Apr-2023 03:58:47.262 SEVERE [https-jsse-nio-443-exec-9] org.apache.struts2.dispatcher.Dispatcher.error Exception occurred during processing request: There is no Action mapped for action name rbokEuh0B62sDF7UhPjrXGSR7lqDAlD6P3jzVVuOLfA

1 Like
5

Ok, so don't we have any provision/configuration by which some extension is appended to this dynamic file ?.

6

No. This is how ACME Let's Encrypt works. You should find out how to have struts allow access.

3 Likes
7

For your (and OPS info): ZeroSSL works with the .txt extension for the challenge file when you try to issue a cert manually.

4 Likes
8

I have found some way from struts side. Just one more thing while generating the certificate I use to see "Successfully received certificate" but it also shows some message like
"Certbot has set up a scheduled task to automatically renew this certificate in the background"

What actually this scheduled task is and what it does when I have already got the message of "successfully received certificate".

9

This is either a cron or systemd timer to auto-renew your cert. You can read details of this here:
https://eff-certbot.readthedocs.io/en/stable/using.html#renewing-certificates
and here:
https://eff-certbot.readthedocs.io/en/stable/using.html#automated-renewals

3 Likes
10

Now I am able to renew the certificate but I have another related query, I am using --deploy-hook parameter to run a script after successful renewal using certbot renew command but the script is not getting executed. The renew command is something like this ..

sudo certbot renew --force-renewal --deploy-hook /home/abc/script/deploy-hook.sh

I tried by placing the deploy hook script in /etc/letsencrypt/renewal-hook/deploy directory as well but no success. The letsencrypt log shows something like this in the end :

2023-05-05 04:14:01,690:INFO:certbot.compat.misc:Running deploy-hook command: /etc/letsencrypt/renewal-hooks/deploy/deploy-hook.sh

What could be the reason for this ?

11

We should try to avoid using:

We should use reconfigure instead.
Try:
certbot help reconfigure
[to get an understanding of what it can do]

3 Likes
12

That looks like the deploy hook did run. Why do you think it did not?

I like to put something in my hook scripts to track activity with something like:

echo $(date) 'Starting MyDeployHook' >>/var/log/MyHooks.log

Sometimes the shell is different in a hook script than in a command prompt. So check that your .sh script sets up your path and such as you intend.

3 Likes
13

Please show file:

and
find / -name deploy-hook.sh

2 Likes
15

Here is the deploy-hook.sh file(some values are mock only)

#!/bin/sh

  echo "Deploying the renewed certificate to load balancer"
  DOMAIN="mydomain[.]com"
  CERT_PATH="/etc/letsencrypt/live/mydomain.com"
  LOAD_BALANCER_ID="111111111111"
  HTTPS_LISTENER_NAME="my_listener"

 oci lb certificate create {...some parameters..}

  oci lb listener update --force  {...some parameters..}
16

I already have echo statements in deploy-hook.sh but it's not get printed in terminal while running. If I call the deploy-hook.sh directly from renew.sh then it get's called but through --deploy-hook option of certbot command it didn't.

Can you please give some more detail about your below point :

"Sometimes the shell is different in a hook script than in a command prompt. So check that your .sh script sets up your path and such as you intend."

17

From your log message it looks like Certbot tries to run your deploy-hook but it doesn't run. Could something like Selinux be blocking it?

Or, could your echo to the terminal just be lost and some other problem causing the load balancer commands to not run?

And, what do you mean it runs ok from renew.sh?

Here is what my letsencrypt.log looks like after doing a renew command of the same format as yours. My hook script only has only the one line I showed earlier (echo $(date) ...) and it runs as expected. I did have to mark the script file as executable.

Log reformatted slightly for readability

2023-05-06 14:02:59,807:
DEBUG:certbot._internal.cli:Var deploy_hook=/home/ubuntu/MyHook.sh (set by user).
DEBUG:certbot._internal.cli:Var renew_hook={'deploy_hook'} (set by user).
DEBUG:certbot._internal.storage:Writing new config /etc/letsencrypt/(...)
DEBUG:certbot._internal.storage:Deleting /etc/letsencrypt/archive/(...) and related items during clean up
INFO:certbot.compat.misc:Running deploy-hook command: /home/ubuntu/MyHook.sh
3 Likes
18

Some people don't add the shebang in their script so inherit the shell of the caller which may be different than when they test the script from the command prompt. You have that so probably not the problem.

3 Likes
19

What is that about?

2 Likes
20

I think it's because they are trying to add the deploy-hook but it only runs when the cert is actually renewed. They could probably test using --staging and --break-my-certs but that can create its own problems. The --dry-run won't run deploy-hook

3 Likes
21

--force is a parameter to forcefully load the certificate for load balancer