OpenResty with resty.auto-ssl failing with curl error 3

ashleyk · April 5, 2023, 9:53pm

My domain is: events.shongololo.xyz

I ran this command:

env HOOK_SECRET=REDACTED HOOK_SERVER_PORT=8999 /usr/local/bin/resty-auto-ssl/dehydrated --cron --accept-terms --no-lock --domain events.shongololo.xyz --challenge http-01 --config /etc/resty-auto-ssl/letsencrypt/config --hook /usr/local/bin/resty-auto-ssl/letsencrypt_hooks

It produced this output:

INFO: Using main config file /etc/resty-auto-ssl/letsencrypt/config
startup_hook
Processing events.shongololo.xyz
 + Signing domains...
 + Generating private key...
 + Generating signing request...
 + Requesting new certificate order from CA...
 + Received 1 authorizations URLs from the CA
 + Handling authorization for events.shongololo.xyz
 + Found valid authorization for events.shongololo.xyz
 + 0 pending challenge(s)
 + Requesting certificate...
 err: ERROR: Problem connecting to server (post for ; curl returned with 3)
, context: ssl_certificate_by_lua*, client: 159.203.63.67, server: 0.0.0.0:443

My web server is (include version): OpenResty 1.21.4.1-2~jammy1 with resty.auto-ssl

The operating system my web server runs on is (include version): Ubuntu 22.04 LTS

My hosting provider, if applicable, is: AWS

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): N/A

jvanasco · April 5, 2023, 10:00pm

You configured something incorrectly. My guess is that you did not configure which ACME server to use. I'm not familiar with that OpenResty plugin, but if you share it I can take a look.

the curl error code 3 is for a malformed url, and the debug log message suggests you posted to an undefined url.

my best guess is that library set the ACME url to nil, and you are supposed to define an ACME url when you import that plugin or execute it in the lua block.

ashleyk · April 5, 2023, 10:02pm

This was working, it only stopped working today, and I have not changed any configuration.

imfranklin · April 5, 2023, 10:04pm

OpenResty relies on the dehydrated ACME client, and it doesn't look like dehydrated supports this new async business being enabled for eight hours today: Enabling Asynchronous Order Finalization - #5 by aarongable

aarongable · April 5, 2023, 10:08pm

Yeah, it looks to me like the client tried to unconditionally extract the Certificate URL from the Finalize response, and when it wasn't there, made a request for the empty-string url "" (which "resolved" to 0.0.0.0:443) which obviously didn't work.

Instead, it should poll (with exponential backoff!) the Order URL until the "certificate" field appears in it, or it transitions to state "invalid", and only attempt to download the certificate then.

ashleyk · April 5, 2023, 10:12pm

I believe the 0.0.0.0:443 is just my OpenResty web server which listens on port 443 on all interfaces and not what dehydrated client is actually connecting to.

aarongable · April 5, 2023, 10:21pm

It looks to me like dehydrated does the right thing for asynchronous finalization already:

github.com

dehydrated-io/dehydrated/blob/e3ef43c816f73d443f32410862d9253d35cf3f99/dehydrated#L1276-L1296


      
          result="$(signed_request "${finalize}" '{"csr": "'"${csr64}"'"}' | jsonsh)"
          while :; do
            orderstatus="$(echo "${result}" | get_json_string_value status)"
            case "${orderstatus}"
            in
              "processing" | "pending")
                echo " + Order is ${orderstatus}..."
                sleep 2;
                ;;
              "valid")
                break;
                ;;
              *)
                _exiterr "Order in status ${orderstatus}"
                ;;
            esac
            result="$(signed_request "${order_location}" "" | jsonsh)"
          done
          
          
resheaders="$(_mktemp)"
          certificate="$(echo "${result}" | get_json_string_value certificate)"

This loop makes a finalize request, the polls the order object until it transitions into either state "valid" or state "invalid", and only attempts to extract the "certificate" url once it's in state "valid".

This change was made almost exactly three years ago: Don't assume order status to be valid · dehydrated-io/dehydrated@58bd926 · GitHub

Does Resty use an older version of dehydrated than that? A quick search didn't immediately reveal to me the dependency chain from Resty to dehydrated.

ashleyk · April 5, 2023, 10:32pm

It uses version 0.6.5 and not the latest 0.7.1 release, and hasn't had a new release in more than 3 years (https://github.com/auto-ssl/lua-resty-auto-ssl).

I will try copying the new dehydrated client over 0.6.5 manually.

ashleyk · April 5, 2023, 10:40pm

Thank you! I copied the 0.7.1 dehydrated client over /usr/local/bin/resty-auto-ssl/dehydrated and its working again.

mcpherrinm · April 5, 2023, 10:44pm

We've disabled the brownout early because of problems like this

ashleyk · April 5, 2023, 10:58pm

Any chance that the rate limits can be reset for affected certs? We have a lot of clients who are holding events who are impacted by this.

jcjones · April 5, 2023, 11:30pm

I'm afraid we do not have the technical means in Boulder to reset rate limits in that sort of way.

system · May 5, 2023, 11:31pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.