Ubuntu Certbot migration for

gkaufmann · January 15, 2020, 4:04pm

I’m trying to migrate certbot to acme-v2 for standalone mode running behind HA-Proxy for auto-renewal Ha-Proxy certs.

After upgrading (using apt ppa) I’m running this certbot version:
certbot 0.31.0

With acme-v1 renewal still works. Currently my cli.ini (/etc/letsencrypt/cli.ini) is pointing to ‘server = https://acme-v01.api.letsencrypt.org/directory’. It doesn’t seem to work upgrading it to ‘server = https://acme-v02.api.letsencrypt.org/directory’.

With acme-v1 I noticed this new warnings (but still works):
Attempting to parse the version 0.26.1 renewal configuration file found at /etc/letsencrypt/renewal/my-domain.de.conf with version 0.10.2 of Certbot. This might not work.
(does produce new certs and works anyway)

Trying to run without 'server = ’ in cli.conf the upgraded client still uses acme-v1. Replacing with acme-v2 URL it doesn’t seem to work at all saying:
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Attempting to renew cert from /etc/letsencrypt/renewal/gitlab.typoworx.de.conf produced an unexpected error: ‘Directory field not found’. Skipping.
(producing no new cert at all)

I did not found a migration guide yet. So what’s wrong here? Is the given version too old and there’s no version for Ubuntu Xenial supporting Acme2 or how can I get this work?

I should say that I’m nut ‘simply’ running certbot --standalone, but also some deploy-hook workflow for Ha-Proxy (prepare PEM-file for Ha-Proxy and softly restart Ha-Proxy afterwards).

Is this also possible with acme.sh if I would have to migrate to this solution?

Osiris · January 15, 2020, 6:09pm

What do you see when you do curl https://acme-v02.api.letsencrypt.org/directory from the server where you run certbot?

gkaufmann · January 15, 2020, 6:43pm

Thanks for your help @Osiris

I see this response:

curl https://acme-v02.api.letsencrypt.org/directory

{
"HcPU9ibhw7A": "Adding random entries to the directory",
"keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",
"meta": {
"caaIdentities": [
"letsencrypt.org"
],
"termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
"website": "https://letsencrypt.org"
},
"newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",
"newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",
"newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",
"revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"

bmw · January 15, 2020, 7:46pm

You should get the same behavior whether you delete the server entry in cli.ini or set it to the ACMEv2 server.

Can you provide the complete log of the failure you get when trying to use ACMEv2? Log files are included in /var/log/letsencrypt by default.

Osiris · January 15, 2020, 8:05pm

@bmw Is it perhaps useful for these kinds of problems to include the directory entry which is missing in the error message? As far as I can tell, this code:

github.com

certbot/certbot/blob/master/acme/acme/messages.py#L250-L254


def __getitem__(self, name):
    try:
        return self._jobj[self._canon_key(name)]
    except KeyError:
        raise KeyError('Directory field not found')

Is triggered when acme tries to “access” a directory item (field in the code?) which isn’t included in the /directory output? One might think an error message like “Directory field ‘new-implementation’ not found” (where a directory field ‘new-implementation’ isn’t provided by Boulder, I just made it up )

Another something I just saw, looking at the different outputs of the /directory paths between the two API versions: the directory field are widely different. Could it be the ACME client of certbot wasn’t updated when certbot itself was updated? I.e., certbot might be ACME v2 compatible, at least, have a compatible version number, but perhaps the ACME client used by certbot wasn’t upgraded? I know Gentoo has two different packages for app-crypt/certbot and app-crypt/acme… Lemme check for Ubuntu Xenial.

Xenial seems to require the PPA. @gkaufmann What is the version of your python-acme package?

mnordhoff · January 15, 2020, 9:30pm

Where did this message with version 0.10.2 come from?

bmw · January 15, 2020, 10:08pm

Nice catch @mnordhoff.

Osiris:

@bmw Is it perhaps useful for these kinds of problems to include the directory entry which is missing in the error message? As far as I can tell, this code:

github.com

certbot/certbot/blob/master/acme/acme/messages.py#L250-L254
def __getitem__(self, name):
    try:
        return self._jobj[self._canon_key(name)]
    except KeyError:
        raise KeyError('Directory field not found')
Is triggered when acme tries to “access” a directory item (field in the code?) which isn’t included in the /directory output? One might think an error message like “Directory field ‘new-implementation’ not found” (where a directory field ‘new-implementation’ isn’t provided by Boulder, I just made it up )

I agree. Thanks for looking into this. I created Include missing directory field in error message · Issue #7683 · certbot/certbot · GitHub.

gkaufmann · January 16, 2020, 8:16am

You should get the same behavior whether you delete the server entry in cli.ini or set it to the ACMEv2 server.
If I remove the server-entry, certbot seems to fallback to ACME v1 server.

I renamed the previous log to letsencrypt.log~ and reset server to ACME v2 server URL. After running certbot renew this is the content of the letsencrypt.log file.

tail /var/log/letsencrypt/letsencrypt.log

File "/usr/bin/certbot", line 11, in
load_entry_point('certbot==0.10.2', 'console_scripts', 'certbot')()
File "/usr/lib/python2.7/dist-packages/certbot/main.py", line 849, in main
return config.func(config, plugins)
File "/usr/lib/python2.7/dist-packages/certbot/main.py", line 655, in renew
renewal.handle_renewal_request(config)
File "/usr/lib/python2.7/dist-packages/certbot/renewal.py", line 430, in handle_renewal_request
len(renew_failures), len(parse_failures)))
Error: 2 renew failure(s), 0 parse failure(s)

gkaufmann · January 16, 2020, 8:22am

The message regarding configuration issue with version 0.10.2 comes while running certbot renew. With ACME v1 server it works never the less. But I can't figure out what the real reason behind this error is.

Is it maybe just a warning about the 'version = 0.10.2' value in the configuration file and not a real problem at all?

btw. this is one of my renewal domain configurations:

version = 0.10.2
cert = /etc/letsencrypt/live/gitlab.typoworx.de/cert.pem
privkey = /etc/letsencrypt/live/gitlab.typoworx.de/privkey.pem
chain = /etc/letsencrypt/live/gitlab.typoworx.de/chain.pem
fullchain = /etc/letsencrypt/live/gitlab.typoworx.de/fullchain.pem
archive_dir = /etc/letsencrypt/archive/gitlab.typoworx.de

Options used in the renewal process

[renewalparams]
authenticator = standalone
account = 9186ac39ed6c6e1b5a8ccd2fadde0c45
standalone_supported_challenges = http-01
http01_port = 50080
tls_sni_01_port = 54321
pref_challs = http-01,
installer = None

I configured certbot to use port 50080 as certbot is running on a server running HA-Proxy (at port 80, 443) and I'm routing certbot-request URIs to the local certbot instance listening at port 50080.

gkaufmann · January 16, 2020, 8:30am

By the way… just in parallel to my questions above. I noticed acme.sh may be an interesting option as replacement for certbot. I noticed acme.sh seems being able to somehow interact with Cloudflare API (https://github.com/Neilpang/acme.sh/wiki/dnsapi). I’m almost using Cloudflare+SSL already or plan to migrate to this service.

Does this work for complete renewal process with Cloudflare using DNS-Challenge? I’m searching for some docs on how to migrate my deploy scripts currently used for certbot. The scripts fetch the Domain-Name being able to locate the certs and create a PEM-File for Ha-Proxy and then reload the service (to recognice the new certs).

If this would be able with acme.sh without too much time, I would also think about this option, too.

mnordhoff · January 16, 2020, 3:17pm

No -- you're actually running Certbot 0.10.2. (Or at least parts of Certbot are version 0.10.2.)

You have another configuration file that contains "version = 0.26.1".

You apparently have, or had, Certbot 0.26.1 installed as well.

What does "dpkg -l '*acme*' '*certbot*'" show?

What does "sudo apt update" show?

What does "apt list --upgradeable" show?

Certbot 0.10.2 does not support ACMEv2. You need to upgrade it.

The latest versions of Certbot available for Ubuntu Xenial -- 0.27.0 in Ubuntu's repositories, or 0.31.0 in the Certbot PPA -- will work. But you somehow have 0.10.2 instead.

I think you may have multiple Certbot installations -- where did 0.26.1 come from? How did you use 0.26.1 and later use 0.10.2? -- but it's hard to say.

(Did you copy /etc/letsencrypt/ files from a computer with 0.26.1?)

Yes, that would work.

Certbot also has a Cloudflare DNS plugin, though. If you're using the PPA, you can sudo apt install python3-certbot-dns-cloudflare. (It's also available in Ubuntu's repositores starting in Bionic.)

gkaufmann · January 17, 2020, 9:39am

You are absolutely right - sorry, my mistake. I'm running HA-Proxy on Debian and there I were unable to update to recent cerbot-version at some point (because of missing deps). I created a chroot (using schroot-package) with debootstrap'ed ubuntu-minimal for certbot.

For some reason I forgot to remove some waist parts of certbot in the main-system (debian). I have linked certbot to the chroot-binary by using mount-feature of schroot (/etc/letsencrypt) and a shell-script in /usr/local/bin delegating all certbot-calls into the chroot-system. I know this is not ideal, but in matter of time it was the best idea I had at that time.

I also gave acme.sh a try. I noticed they already have a bunch of deploy-scripts (including ha-proxy). But for some reason the cloudflare-API authentication/access seems not to work for me currently. I'm not shure yet if it is an issue of permission-level or some API changes at cloudflare. Acme.sh currently gives me this error trying to issue a cert:

export CF_Account_ID="xxxxxxxxxxxxxxxxxxxxxxxxx"
export CF_Token="xxxxxxxxxxxxxxxxxxxxxxxxx"
acme.sh --issue --dns dns_cf --test -d my-domain.de

invalid domain
Error add txt for domain:_acme-challenge.my-domain.de

Of course the given domain already exists at my cloudflare-account! I've been trying to use the global API with this permissions:

So acme.sh is no "plug & play" solution for me at the moment

system · February 16, 2020, 9:39am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Acmev1 deprecated and certbot unavailable Help	10	1911	August 21, 2021
Directory field not found': new_authz Help	4	2818	July 16, 2021
Migrate from certbot to acme.sh: painless? Help	7	2980	February 17, 2023
Update your client software to continue using Let's Encrypt Help	6	861	May 14, 2020
Upgrade certbot 0.14.2 on Ubuntu 14.04 Help	3	1928	June 27, 2020

Ubuntu Certbot migration for

curl https://acme-v02.api.letsencrypt.org/directory

Options used in the renewal process

Related topics