Error retrieving account in Staging error 500 with new domain request

Help
1

I previously registered other domains successfully using the same commands but with other domain names. Today when trying to add a new domain name I'm receiving an error in trying to retrieve account information.

My domain is: staging.saikoctf.org

I ran this command: certbot certonly -d staging.saikoctf.org --webroot -vvv --dry-run

Note this is executed using containerized certbot and nginx. There is a shared webroot folder so that the /.well-known/acme-challenge/ context is writable from the certbot container and accessible via the nginx container. This configuration and process worked last week with dev.saikoctf.org.

It produced this output:

2024-07-18 22:04:02,379:DEBUG:certbot._internal.main:certbot version: 2.10.0
2024-07-18 22:04:02,380:DEBUG:certbot._internal.main:Location of certbot entry point: /usr/local/bin/certbot
2024-07-18 22:04:02,380:DEBUG:certbot._internal.main:Arguments: ['-d', 'staging.saikoctf.org', '--webroot', '--dry-run', '-vvv']
2024-07-18 22:04:02,380:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2024-07-18 22:04:02,390:DEBUG:certbot._internal.log:Root logging level set at 0
2024-07-18 22:04:02,391:DEBUG:certbot._internal.plugins.selection:Requested authenticator webroot and installer None
2024-07-18 22:04:02,391:DEBUG:certbot._internal.plugins.selection:Single candidate plugin: * webroot
Description: Saves the necessary validation files to a .well-known/acme-challenge/ directory within the nominated webroot path. A seperate HTTP server must be running and serving files from the webroot path. HTTP challenge only (wildcards not supported).
Interfaces: Authenticator, Plugin
Entry point: EntryPoint(name='webroot', value='certbot._internal.plugins.webroot:Authenticator', group='certbot.plugins')
Initialized: <certbot._internal.plugins.webroot.Authenticator object at 0x7fdeef143d40>
Prep: True
2024-07-18 22:04:02,392:DEBUG:certbot._internal.plugins.selection:Selected authenticator <certbot._internal.plugins.webroot.Authenticator object at 0x7fdeef143d40> and installer None
2024-07-18 22:04:02,392:INFO:certbot._internal.plugins.selection:Plugins selected: Authenticator webroot, Installer None
2024-07-18 22:04:02,435:DEBUG:certbot._internal.main:Picked account: <Account(RegistrationResource(body=Registration(key=None, contact=(), agreement=None, status=None, terms_of_service_agreed=None, only_return_existing=None, external_account_binding=None), uri='https://acme-staging-v02.api.letsencrypt.org/acme/acct/148475964', new_authzr_uri=None, terms_of_service=None), 6bf802353e9bc27aec5c263934333e47, Meta(creation_dt=datetime.datetime(2024, 5, 16, 23, 45, 9, tzinfo=<UTC>), creation_host='97685444d6e7', register_to_eff=None))>
2024-07-18 22:04:02,436:DEBUG:acme.client:Sending GET request to https://acme-staging-v02.api.letsencrypt.org/directory.
2024-07-18 22:04:02,438:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org:443
2024-07-18 22:04:02,525:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 820
2024-07-18 22:04:02,526:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Thu, 18 Jul 2024 22:04:02 GMT
Content-Type: application/json
Content-Length: 820
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "2obshh25W4s": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-staging-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.4-April-3-2024.pdf",
    "website": "https://letsencrypt.org/docs/staging-environment/"
  },
  "newAccount": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order",
  "renewalInfo": "https://acme-staging-v02.api.letsencrypt.org/draft-ietf-acme-ari-03/renewalInfo",
  "revokeCert": "https://acme-staging-v02.api.letsencrypt.org/acme/revoke-cert"
}
2024-07-18 22:04:02,534:DEBUG:certbot._internal.display.obj:Notifying user: Simulating a certificate request for staging.saikoctf.org
2024-07-18 22:04:02,536:DEBUG:acme.client:Requesting fresh nonce
2024-07-18 22:04:02,536:DEBUG:acme.client:Sending HEAD request to https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce.
2024-07-18 22:04:02,565:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "HEAD /acme/new-nonce HTTP/1.1" 200 0
2024-07-18 22:04:02,566:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Thu, 18 Jul 2024 22:04:02 GMT
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: zuRnlMmX4Our2Z54Z015V4xFLp9XMPmEdylXCe8nhGI26TLnaNE
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800


2024-07-18 22:04:02,567:DEBUG:acme.client:Storing nonce: zuRnlMmX4Our2Z54Z015V4xFLp9XMPmEdylXCe8nhGI26TLnaNE
2024-07-18 22:04:02,567:DEBUG:acme.client:JWS payload:
b'{\n  "identifiers": [\n    {\n      "type": "dns",\n      "value": "staging.saikoctf.org"\n    }\n  ]\n}'
2024-07-18 22:04:02,570:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/new-order:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8xNDg0NzU5NjQiLCAibm9uY2UiOiAienVSbmxNbVg0T3VyMlo1NFowMTVWNHhGTHA5WE1QbUVkeWxYQ2U4bmhHSTI2VExuYU5FIiwgInVybCI6ICJodHRwczovL2FjbWUtc3RhZ2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL25ldy1vcmRlciJ9",
  "signature": "qssPiD89ZNEZDpKOMVk5YM5v6nITEpWtziWnJmwkQIrTd3CTWDTuPhnKVcIUKTM0gRpaBriHQvMRihvu9Yk5ChmwkmtInKXgCU7pfmIa2fs5g2ksCB-wT2MtcqTIpOQSrXreQ8pp3CTKKRC7X9I0O2iCcM93jPVTkr4my2_UHu9jz1BACHqRivjbKLQRqtjkGj3yX1Q2A9t96BDHPmXN2T7IRBsl26C1DeAbMqbfqllR_GGGC69AVHKihgqojS2atPCX_ftB9DZ5-yhTKOSWLlUS2A9fQAV769kqbrw29qGvTdmQBsu83cT73o3n0OZncqf9C1KT1c6rI8fM-Gew6g",
  "payload": "ewogICJpZGVudGlmaWVycyI6IFsKICAgIHsKICAgICAgInR5cGUiOiAiZG5zIiwKICAgICAgInZhbHVlIjogInN0YWdpbmcuc2Fpa29jdGYub3JnIgogICAgfQogIF0KfQ"
}
2024-07-18 22:04:02,600:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/new-order HTTP/1.1" 500 183
2024-07-18 22:04:02,601:DEBUG:acme.client:Received response:
HTTP 500
Server: nginx
Date: Thu, 18 Jul 2024 22:04:02 GMT
Content-Type: application/problem+json
Content-Length: 183
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: ROLR2NAwUKhLsom51dkdNYlFtQshU7ymGbtMFK2n2NZDYpUiWGo

{
  "type": "urn:ietf:params:acme:error:serverInternal",
  "detail": "Error retrieving account \"https://acme-staging-v02.api.letsencrypt.org/acme/acct/148475964\"",
  "status": 500
}
2024-07-18 22:04:02,601:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/local/bin/certbot", line 33, in <module>
    sys.exit(load_entry_point('certbot', 'console_scripts', 'certbot')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/src/certbot/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/src/certbot/certbot/_internal/main.py", line 1894, in main
    return config.func(config, plugins)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/src/certbot/certbot/_internal/main.py", line 1600, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/src/certbot/certbot/_internal/main.py", line 143, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/src/certbot/certbot/_internal/client.py", line 517, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/src/certbot/certbot/_internal/client.py", line 428, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/src/certbot/certbot/_internal/client.py", line 478, in _get_order_and_authorizations
    orderr = self.acme.new_order(csr_pem)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/src/acme/acme/client.py", line 137, in new_order
    response = self._post(self.directory['newOrder'], order)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/src/acme/acme/client.py", line 365, in _post
    return self.net.post(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/src/acme/acme/client.py", line 738, in post
    return self._post_once(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/src/acme/acme/client.py", line 751, in _post_once
    response = self._check_response(response, content_type=content_type)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/src/acme/acme/client.py", line 602, in _check_response
    raise messages.Error.from_json(jobj)
acme.messages.Error: urn:ietf:params:acme:error:serverInternal :: The server experienced an internal error :: Error retrieving account "https://acme-staging-v02.api.letsencrypt.org/acme/acct/148475964"
2024-07-18 22:04:02,605:ERROR:certbot._internal.log:An unexpected error occurred:
2024-07-18 22:04:02,606:ERROR:certbot._internal.log:Error retrieving account "https://acme-staging-v02.api.letsencrypt.org/acme/acct/148475964"

My web server is (include version): nginx 1.25.5

The operating system my web server runs on is (include version): Debian Bookworm (dockerized NGINX)

My hosting provider, if applicable, is: private datacenter

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 2.10.0

2

I change the title to highlight the specific error. Several people have reported this so it looks like there is an active problem. Hopefully staff provides status update soon

3

Thanks.

Is there system status page somewhere?

4
5

Yes, and there should also be a banner in this forum once the incident gets posted

6

I believe this failure is resolved now. :slightly_smiling_face:

7

Yes, we had a brief database problem, which has been resolved.