Urgent: Certbot 4.0.0 Renewal Suddenly Failing on Existing Production Nginx Server (Ubuntu 24.04)

Help
1

Hello everyone,

I have an existing production Ubuntu server hosting multiple websites that have successfully renewed Let's Encrypt certificates for well over a year. As of July 1, all certificate renewals suddenly began failing, causing every website on the server to go offline. The sites have now been down for approximately 72 hours.

I have never relied on automatic renewals. I have always manually executed Certbot renewals every 30–45 days, and they have always completed successfully until now.

Environment

  • Ubuntu 24.04.4 LTS
  • Nginx 1.28.3
  • Certbot 4.0.0
  • Multiple production websites hosted on the same VPS
  • IPv4 and IPv6 configured
  • DNS A and AAAA records verified

Existing Configuration

The Nginx virtual host configurations have remained essentially unchanged since they were created.

  • Three sites have had the same configuration since November 2024.
  • One additional site was added in December 2025 and has also renewed successfully since its deployment.
  • No recent changes have been made to the Nginx virtual hosts.
  • nginx -t returns:
syntax is ok
test is successful

The active Nginx configuration (nginx -T) confirms that each virtual host explicitly contains both the apex domain and the www hostname in the server_name directive.

Current Problem

Running:

certbot renew --dry-run -v

fails for every website.

The Certbot logs repeatedly contain:

Generated server block:
[]

followed by:

AuthorizationError: Some challenges have failed

Additional Testing

I tested using the webroot authenticator instead of the Nginx plugin.

Testing only the apex domain succeeds:

certbot certonly --webroot -w <webroot> -d example.com --dry-run

However, requesting both the apex domain and the www hostname fails:

certbot certonly --webroot -w <webroot> -d example.com -d www.example.com

The failure is:

Challenge failed for domain www.example.com

Type: unauthorized

Detail:
Invalid response from
http://www.example.com/.well-known/acme-challenge/<token>

404

The challenge directory exists and Certbot creates the challenge file correctly.

Testing with curl confirms the same behavior:

curl -4 http://www.example.com/.well-known/acme-challenge/test

returns:

404

and

curl -6 http://www.example.com/.well-known/acme-challenge/test

also returns:

404

What Has Been Verified

  • DNS A records are correct.
  • DNS AAAA records are correct.
  • IPv6 is configured and active.
  • Nginx configuration is valid.
  • The renewal configuration has not changed and continues to use:
authenticator = nginx
installer = nginx

Request for Assistance

At this point, I am looking for the fastest path to restoring service.

Specifically:

  1. What would cause the Certbot Nginx plugin to report:
Generated server block: []

on an existing production configuration that has successfully renewed certificates for over a year?

  1. Why would the HTTP-01 challenge consistently return a 404 for the www hostname even though both hostnames are explicitly defined in the active Nginx server block?
  2. Is there a recommended method to restore certificate issuance immediately without rebuilding or significantly modifying a production Nginx configuration that has been stable for over a year?
  3. Has anyone experienced this behavior with Certbot 4.0.0 on Ubuntu 24.04, and if so, what was the resolution?

I can provide complete Certbot logs, the relevant Nginx virtual host configuration, renewal configuration, and any additional diagnostics that would help identify the issue.

Any guidance is greatly appreciated. My immediate priority is restoring HTTPS service across all hosted websites.

2

That is reported when there's already an existing HTTP (not HTTPS) vhost for the challenge's domain.

Check for a server block listening on port 80 and check any www specific rewriting/redirection. Without specific configurations, there's not much else I can suggest.

Without a working Nginx based renewal, your options are mostly to

a. Temporarily stop Nginx and use the standalone renewal HTTP server or
b. Use DNS-01.

While I don't know of any prior occurances of that, Certbot 4.0.0 is outdated. The current version is Certbot 5.6.0 and can be installed using pip or snap.

3

@thebadagency I moved your thread to the Help section. It isn't an "Issuance" issue but an ACME Client problem. No worries - just explaining. You would have also been shown a form asking basic info but you provided most of it anyway and offered the rest so we're good :slight_smile:

This strongly indicates you have a "rogue" server block handling some requests. I have seen this before when someone does not have consistent listen statements for IPv4 and IPv6 in all their server blocks and thus nginx doesn't process the request as you expect. Other things can cause this.

After a webroot test with both names check the nginx access log for the server block you think should handle it. That works best if you use unique file names for each server block (or use some other id in a common log).

HOWEVER, the nginx authenticator works a lot different than webroot. The nginx authenticator makes temp changes to your nginx server block(s), reloads nginx, requests the certs, and then removes the temp changes reloading nginx yet again. These changes handle the challenge without creating a file

I think the webroot test shows there is something "off" in your nginx config. It just manifests slightly different between webroot and --nginx.

You can see the actual code changes with --nginx in the Certbot log.

I'd be happy to look at your nginx config. If you could upload output from below that would be great

sudo nginx -T >~/nginx.txt

I think you know this but the upper case T is essential for this