Error 403 when Creating Certificate with USG profile

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:sens.wtf

I ran this command: sudo certbot --apache -v

It produced this output:

2024-12-21 20:16:44,564:DEBUG:certbot._internal.display.obj:Notifying user:
Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
  Domain: sens.wtf
  Type:   unauthorized
  Detail: [redacted ip]: Invalid response from http://sens.wtf/.well-known/acme-challenge/2PggW3Ktc2xCVBQq87hBKkcwuqOn5oSXMdc2ZME0MDM: 403

  Domain: www.sens.wtf
  Type:   unauthorized
  Detail: [redacted ip]: Invalid response from http://www.sens.wtf/.well-known/acme-challenge/IqVuY72GhwzYJjJt8k-TOTBNPzHVCvC72xHPXm9tFVQ: 403

Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.

2024-12-21 20:16:44,568:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 90, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 178, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2024-12-21 20:16:44,568:DEBUG:certbot._internal.error_handler:Calling registered functions
2024-12-21 20:16:44,569:INFO:certbot._internal.auth_handler:Cleaning up challenges
2024-12-21 20:16:45,032:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 33, in <module>
    sys.exit(load_entry_point('certbot==1.21.0', 'console_scripts', 'certbot')())
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 15, in main
    return internal_main.main(cli_args)
  File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line 1574, in main
    return config.func(config, plugins)
  File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line 1287, in run
    new_lineage = _get_and_save_cert(le_client, config, domains,
  File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line 133, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "/usr/lib/python3/dist-packages/certbot/_internal/client.py", line 459, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
  File "/usr/lib/python3/dist-packages/certbot/_internal/client.py", line 389, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/usr/lib/python3/dist-packages/certbot/_internal/client.py", line 439, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 90, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 178, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.
2024-12-21 20:16:45,041:ERROR:certbot._internal.log:Some challenges have failed.

My web server is (include version): Apache/2.4.52

The operating system my web server runs on is (include version):
Ubuntu 22.04 LTS Server with USG profile (cis_level1_server)

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.21.0

Hello friends!

I suspect this is because of the overly restrictive permissions set by the USG profile.
When I try to create a file in my /var/www/sens.wtf/ directory.
(ex. sudo touch /var/www/sens.wtf/test.html)
I also get a 403 error when visiting http://sens.wtf/test.html

  File: /var/www/sens.wtf/test.html
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: 1dh/29d Inode: 224041      Links: 1
Access: (0644/-rw-r--r--)  Uid: (   33/www-data)   Gid: (   33/www-data)
Access: 2024-12-21 19:12:12.940870038 -0500
Modify: 2024-12-21 19:12:12.940870038 -0500
Change: 2024-12-21 19:18:08.078956903 -0500
 Birth: 2024-12-21 19:12:12.940870038 -0500

I have tried running the command umask 0022 to no avail.

Any ideas?

You must have changed something because I can see your test file just fine

curl -i http://sens.wtf/test.html
HTTP/1.1 200 OK
Server: Apache/2.4.52 (Ubuntu)

If you still want help, what is your latest command and failure message?

1 Like

My apologies, I modified it while troubleshooting earlier. As a demonstration I've created another test file http://sens.wtf/letsencrypt.html with the command sudo touch /var/www/sens.wtf/letsencrypt.html

These are my most recent commands.

467 sudo certbot --apache -v
468 sudo touch /var/www/sens.wtf/letsencrypt.html

This is the output from sudo certbot --apache -v

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache

Which names would you like to activate HTTPS for?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: sens.wtf
2: www.sens.wtf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 
Requesting a certificate for sens.wtf and www.sens.wtf
Performing the following challenges:
http-01 challenge for sens.wtf
http-01 challenge for www.sens.wtf
Enabled Apache rewrite module
Waiting for verification...
Challenge failed for domain sens.wtf
Challenge failed for domain www.sens.wtf
http-01 challenge for sens.wtf
http-01 challenge for www.sens.wtf

Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
  Domain: sens.wtf
  Type:   unauthorized
  Detail: [ip redacted]: Invalid response from http://sens.wtf/.well-known/acme-challenge/9xpyTSGlzQfcY7bcZmynYuOiZCOF0eQ8degi7hR1A_Y: 403

  Domain: www.sens.wtf
  Type:   unauthorized
  Detail: [ip redacted]: Invalid response from http://www.sens.wtf/.well-known/acme-challenge/1ot3qWVyEZjxgt27AmCf8pad2cMxwiiKQDvvUh6lccc: 403

Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.

Cleaning up challenges
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

Thank you a lot for taking the time out of your day to help.

1 Like

I see the 403 for your letsencrypt.html file. Interesting, I do get a default "home" page from the index.html file on your server. Is the letsencrypt.html test file in this same folder as index.html? Are the permissions on the files the same?

Apache should explain the reason for the 403 in its ErrorLog (see: Log Files - Apache HTTP Server Version 2.4). You might even have a default error log already active.

You can post the related error log entries here and we might be able to help.

1 Like

Yes, they are in the same folder.

$ ls -la /var/www/sens.wtf/

total 4
drwxr-xr-x 1 www-data www-data  70 Dec 22 00:02 .
drwxr-xr-x 1 root     root      24 Dec 21 07:22 ..
-rw-r--r-- 1 www-data www-data 170 Dec 21 07:23 index.html
-rw-r----- 1 root     root       0 Dec 21 23:55 letsencrypt.html
-rw-r--r-- 1 www-data www-data   0 Dec 21 19:12 test.html
[Sun Dec 22 17:38:10.613058 2024] [core:error] [pid 963:tid 281473130950848] (13)Permission denied: [client [ip redacted]:38800] AH00132: file permissions deny server access: /var/www/sens.wtf/letsencrypt.html
[Sun Dec 22 17:53:16.710793 2024] [core:error] [pid 963:tid 281473139404992] (13)Permission denied: [client [ip redacted]:45762] AH00132: file permissions deny server access: /var/www/sens.wtf/letsencrypt.html
[Sun Dec 22 17:53:32.615573 2024] [core:error] [pid 963:tid 281473380249792] (13)Permission denied: [client [ip redacted]:41012] AH00132: file permissions deny server access: /var/www/sens.wtf/letsencrypt.html
[Sun Dec 22 18:18:28.321014 2024] [mpm_event:notice] [pid 961:tid 281473406963744] AH00493: SIGUSR1 received.  Doing graceful restart
[Sun Dec 22 18:18:28.354414 2024] [mpm_event:notice] [pid 961:tid 281473406963744] AH00489: Apache/2.4.52 (Ubuntu) configured -- resuming normal operations
[Sun Dec 22 18:18:28.354494 2024] [core:notice] [pid 961:tid 281473406963744] AH00094: Command line: '/usr/sbin/apache2'
[Sun Dec 22 18:18:31.556417 2024] [core:error] [pid 102207:tid 281473388507328] (13)Permission denied: [client 23.178.112.107:50023] AH00035: access to /.well-known/acme-challenge/nXgZGvgXZ4O8NZ5dY1vLJdXiXX6molj5FxWN7AXXdas denied (fil>
[Sun Dec 22 18:18:31.696248 2024] [core:error] [pid 102207:tid 281473298985152] (13)Permission denied: [client 23.178.112.107:50029] AH00035: access to /.well-known/acme-challenge/hWaICY88cSdXFvtC4ObN0AfTBBHeVJXUChN5y2xLnts denied (fil>
[Sun Dec 22 18:18:33.080315 2024] [mpm_event:notice] [pid 961:tid 281473406963744] AH00493: SIGUSR1 received.  Doing graceful restart
[Sun Dec 22 18:18:33.106103 2024] [mpm_event:notice] [pid 961:tid 281473406963744] AH00489: Apache/2.4.52 (Ubuntu) configured -- resuming normal operations
[Sun Dec 22 18:18:33.106180 2024] [core:notice] [pid 961:tid 281473406963744] AH00094: Command line: '/usr/sbin/apache2'
[Sun Dec 22 18:57:14.177733 2024] [mpm_event:notice] [pid 961:tid 281473406963744] AH00492: caught SIGWINCH, shutting down gracefully
[2024-12-22 18:57:14.328681] [mpm_event:notice] AH00489: Apache/2.4.52 (Ubuntu) configured -- resuming normal operations
[2024-12-22 18:57:14.329146] [core:notice] AH00094: Command line: '/usr/sbin/apache2'
[2024-12-22 18:57:38.359959] [C:jb4IpaQ6o6E] remote [ip redacted]:58012 local [ip redacted]:80
[2024-12-22 18:57:38.360208] [R:ybwIpaQ6o6E] Request 0 on C:jb4IpaQ6o6E pid:162241 tid:281473417998528
[2024-12-22 18:57:38.360236] [R:ybwIpaQ6o6E] UA:'Mozilla/5.0 (X11; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0'
[2024-12-22 18:57:38.360263] [core:error] [R:ybwIpaQ6o6E] (13)Permission denied: AH00132: file permissions deny server access: /var/www/sens.wtf/letsencrypt.html

I ran another sudo certbot --apache and sent several requests. I've also modified the ErrorLogFormat to include more information.

I'm still learning how to mess around with the error logs so if you have any direct advice on how to make this easier for you or anyone else, please let me know.

Yes, in same folder but not same permissions. You require root to read the letsencrypt.html file but anyone can read the other two which work. The error log clearly says this.

A similar error is shown for the incoming request with /.well-known/acme-challenge. These are coming from the Let's Encrypt Server to validate your cert request.

You have some fundamental issues with your Apache setup. I'd recommend using the --webroot option instead of --apache for now. It will be clearer what happens. The --apache option writes files into folders determined by Certbot. With --weboot you also provide a --webroot-path and can choose the path.

Use that along with --debug-challenges -v to help debug how your system sets permissions on newly created files.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.