Problem creating new certificate with http-01 validation

Help
1

Hi, I have a similar problem with our debian 8 apache server.
I used certbot 0.9.3 before and it worked properly using the default tls-sni challenge.

Now I removed certbot and installed certbot-auto 0.30.2 but the domain validation with http-01 doesn’t work.
When I force certbot 0.30.2 to use tls-sni it fortunately still works but apparently soon it will no more.

root@trasis:/home/certbot# ./certbot-auto --apache --dry-run renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/www.trasis.com.conf

Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator apache, Installer apache
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for my.trasis.com
http-01 challenge for trasis.com
http-01 challenge for www.trasis.com
Waiting for verification…
Cleaning up challenges
Attempting to renew cert (www.trasis.com) from /etc/letsencrypt/renewal/www.trasis.com.conf produced an unexpected error: Failed authorization procedure. www.trasis.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.trasis.com/.well-known/acme-challenge/boiCprgCwx4rgijZQqSsfK2svp6MU7s9R16w3R7nlf8: "\n<html lang=“en” dir=“ltr” prefix=“content: http://purl.org/rss/1.0/modules/content/ dc: http://purl.org/dc/ter”, trasis.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://trasis.com/.well-known/acme-challenge/6fZpyQru1Fuq8kMJ_UHUbllzVO1XqGKzoLhjDdOhqYs: "\n<html lang=“en” dir=“ltr” prefix=“content: http://purl.org/rss/1.0/modules/content/ dc: http://purl.org/dc/ter”. Skipping.

Under IMPORTANT NOTES is shows Type: unauthorized and Detail: Invalid response from…

(sorry, I wanted to put the full output but because I’m a new user there is a link restriction of 20 and I can’t upload a file neither … )

I tried to put a file under .well-known/acme-challenge and it seems to be accessible (by the way trasis.com is redirected to www.trasis.com).

root@trasis:/home/certbot# curl -X GET -I http://trasis.com/.well-known/acme-challenge/1234
HTTP/1.1 301 Moved Permanently
Date: Fri, 01 Feb 2019 13:52:41 GMT
Server: Apache
X-Content-Type-Options: nosniff
Location: http://www.trasis.com/.well-known/acme-challenge/1234
Content-Length: 261
Content-Type: text/html; charset=iso-8859-1

root@trasis:/home/certbot# curl -X GET -I http://www.trasis.com/.well-known/acme-challenge/1234
HTTP/1.1 200 OK
Date: Fri, 01 Feb 2019 13:52:48 GMT
Server: Apache
X-Content-Type-Options: nosniff
Last-Modified: Fri, 01 Feb 2019 10:51:26 GMT
ETag: “5-580d2ecbd7551”
Accept-Ranges: bytes
Content-Length: 5
X-Content-Type-Options: nosniff
X-Frame-Options: sameorigin

I’ve basic knowledge of apache configuration and I was happy in the past to use certbot for our https access. It was indeed a peace of cake to configure it, but now I’m a little bit lost and some help would really be appreciated.
Yes, I tried to find some answers in other posts but nothing worked, I’m really stuck :frowning:

2

Hi @aveithen

I've splittet your post to create a new topic, it's easier if you have your own thread.

That

looks good, so use your webroot instead of apache as authenticator:

certbot certonly -a webroot -w yourWebRoot -d www.trasis.com -d trasis.com -d my.trasis.com --dry-run

I see, you have already checked your domain ( https://check-your-website.server-daten.de/?q=trasis.com ). That looks ok, a redirect http + non-www to http, then a http status 404.

3

Hi @JuergenAuer

Thank you for your fast reply.
I tried the command below and I get the following error:

root@trasis:/home/certbot# ./certbot-auto certonly -a webroot -w /home/data/trasis.com/wwwroot/ -d www.trasis.com -d trasis.com -d my.trasis.com --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None

Traceback (most recent call last):
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/crypto_util.py”, line 334, in _load_cert_or_req
return load_func(typ, cert_or_req_str)
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/OpenSSL/crypto.py”, line 1661, in load_certificate
_raise_current_error()
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/OpenSSL/_util.py”, line 48, in exception_from_error_queue
raise exception_type(errors)
Error: [(‘PEM routines’, ‘PEM_read_bio’, ‘no start line’)]
An unexpected error occurred:
Error: [(‘PEM routines’, ‘PEM_read_bio’, ‘no start line’)]
Please see the logfiles in /var/log/letsencrypt for more details.

Does not look so good.

4 
root@trasis:/home/certbot# cat /var/log/letsencrypt/letsencrypt.log

2019-02-01 16:19:10,605:DEBUG:certbot.main:certbot version: 0.30.2
2019-02-01 16:19:10,606:DEBUG:certbot.main:Arguments: [’-a’, ‘webroot’, ‘-w’, ‘/home/data/trasis.com/wwwroot/’, ‘-d’, ‘www.trasis.com’, ‘-d’, ‘trasis.com’, ‘-d’, ‘my.trasis.com’, ‘–dry-run’]
2019-02-01 16:19:10,606:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2019-02-01 16:19:10,624:DEBUG:certbot.log:Root logging level set at 20
2019-02-01 16:19:10,624:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2019-02-01 16:19:10,625:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2019-02-01 16:19:10,627:DEBUG:certbot.plugins.selection:Single candidate plugin: * webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot.plugins.webroot:Authenticator
Initialized: <certbot.plugins.webroot.Authenticator object at 0x7f66d1cb9c10>
Prep: True
2019-02-01 16:19:10,628:DEBUG:certbot.plugins.selection:Selected authenticator <certbot.plugins.webroot.Authenticator object at 0x7f66d1cb9c10> and installer None
2019-02-01 16:19:10,628:INFO:certbot.plugins.selection:Plugins selected: Authenticator webroot, Installer None
2019-02-01 16:19:10,630:DEBUG:certbot.main:Picked account: <Account(RegistrationResource(body=Registration(status=None, terms_of_service_agreed=None, agreement=u’https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf’, only_return_existing=None, contact=(), key=JWKRSA(key=<ComparableRSAKey(<cryptography.hazmat.backends.openssl.rsa._RSAPublicKey object at 0x7f66d4772210>)>), external_account_binding=None), uri=u’https://acme-staging.api.letsencrypt.org/acme/reg/853332’, new_authzr_uri=u’https://acme-staging.api.letsencrypt.org/acme/new-authz’, terms_of_service=u’https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf’), 0f2b9fba836efcf617172c73d6bfdb04, Meta(creation_host=u’trasis.com’, creation_dt=datetime.datetime(2017, 2, 6, 14, 37, 12, tzinfo=)))>
2019-02-01 16:19:10,631:DEBUG:acme.client:Sending GET request to https://acme-staging-v02.api.letsencrypt.org/directory.
2019-02-01 16:19:10,632:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org:443
2019-02-01 16:19:10,808:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 “GET /directory HTTP/1.1” 200 724
2019-02-01 16:19:10,809:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 724
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Fri, 01 Feb 2019 15:19:10 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Fri, 01 Feb 2019 15:19:10 GMT
Connection: keep-alive

{
“0JhtefgFS1A”: “Adding random entries to the directory”,
“keyChange”: “https://acme-staging-v02.api.letsencrypt.org/acme/key-change”,
“meta”: {
“caaIdentities”: [
“letsencrypt.org”
],
“termsOfService”: “https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf”,
“website”: “https://letsencrypt.org/docs/staging-environment/”
},
“newAccount”: “https://acme-staging-v02.api.letsencrypt.org/acme/new-acct”,
“newNonce”: “https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce”,
“newOrder”: “https://acme-staging-v02.api.letsencrypt.org/acme/new-order”,
“revokeCert”: “https://acme-staging-v02.api.letsencrypt.org/acme/revoke-cert”
}

5

2019-02-01 16:19:10,812:ERROR:certbot.crypto_util:
Traceback (most recent call last):
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/crypto_util.py”, line 334, in _load_cert_or_req
return load_func(typ, cert_or_req_str)
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/OpenSSL/crypto.py”, line 1661, in load_certificate
_raise_current_error()
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/OpenSSL/_util.py”, line 48, in exception_from_error_queue
raise exception_type(errors)
Error: [(‘PEM routines’, ‘PEM_read_bio’, ‘no start line’)]
2019-02-01 16:19:10,812:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
File “/opt/eff.org/certbot/venv/bin/letsencrypt”, line 11, in
sys.exit(main())
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/main.py”, line 1364, in main
return config.func(config, plugins)
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/main.py”, line 1242, in certonly
should_get_cert, lineage = _find_cert(config, domains, certname)
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/main.py”, line 286, in _find_cert
action, lineage = _find_lineage_for_domains_and_certname(config, domains, certname)
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/main.py”, line 313, in _find_lineage_for_domains_and_certname
return _find_lineage_for_domains(config, domains)
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/main.py”, line 257, in _find_lineage_for_domains
ident_names_cert, subset_names_cert = cert_manager.find_duplicative_certs(config, domains)
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/cert_manager.py”, line 167, in find_duplicative_certs
return _search_lineages(config, update_certs_for_domain_matches, (None, None))
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/cert_manager.py”, line 388, in _search_lineages
rv = func(candidate_lineage, rv, *args)
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/cert_manager.py”, line 155, in update_certs_for_domain_matches
candidate_names = set(candidate_lineage.names())
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/storage.py”, line 878, in names
return crypto_util.get_names_from_cert(f.read())
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/crypto_util.py”, line 382, in get_names_from_cert
csr, crypto.load_certificate, typ)
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/crypto_util.py”, line 362, in _get_names_from_cert_or_req
loaded_cert_or_req = _load_cert_or_req(cert_or_req, load_func, typ)
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/crypto_util.py”, line 334, in _load_cert_or_req
return load_func(typ, cert_or_req_str)
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/OpenSSL/crypto.py”, line 1661, in load_certificate
_raise_current_error()
File “/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/OpenSSL/_util.py”, line 48, in exception_from_error_queue
raise exception_type(errors)
Error: [(‘PEM routines’, ‘PEM_read_bio’, ‘no start line’)]
2019-02-01 16:19:10,813:ERROR:certbot.log:An unexpected error occurred:

6

Curious. Looks like Certbot tries to find current certificates and finds certificate files without content.

Did you delete older certificates?

7

Yesterday at some point https didn’t work any more on our website (because I tried some stuff) so I tried to manually change the symbolic links in the live folder to point them to the previous files in archive folder.
Of course that didn’t work and finally I was lucky that forcing the tls-sni challenge made it work again.

I just checked the archive folder and for trasis.com the live files point to some empty .pem files …
Strange, because the https access workes properly for trasis.com and www.trasis.com

root@trasis:/etc/letsencrypt/archive/trasis.com# ls -lt …/…/archive/trasis.com/
total 224
-rw-r–r-- 1 root root 0 jan 31 16:40 privkey15.pem
-rw-r–r-- 1 root root 0 jan 31 16:40 chain15.pem
-rw-r–r-- 1 root root 0 jan 31 16:40 fullchain15.pem
-rw-r–r-- 1 root root 0 jan 31 16:40 cert15.pem
-rw-r–r-- 1 root root 1899 jan 31 16:05 cert14.pem

Did I mess it up ? :thinking:
How can we repair it ?

8

That's bad.

Perhaps use

certbot update_symlinks

to repair.

9

Perhaps running certbot-auto as root user can help.

10

In every CLI copy/paste, I see root@. How did you figure he doesn’t run certbot as root?

11

Ok so your eyes are still 20/20 :wink:

What about running ./certbot-auto from the directory where it is installed?
[perhaps a PATH related issue…]

12

I’ll try the update_symlinks command as soon as I return to office (end to the week).
There I’m out because of a cold… :snowflake::frowning_face:

13

I tried the update_symlinks command but it doesn’t change the links.
The “live” links (for trasis.com) are still pointing to the version 15 files that are empty.

root@trasis:/etc/letsencrypt/live/trasis.com# ls -l
total 0
lrwxrwxrwx 1 root root 35 fév 7 10:18 cert.pem -> …/…/archive/trasis.com/cert15.pem
lrwxrwxrwx 1 root root 36 fév 7 10:18 chain.pem -> …/…/archive/trasis.com/chain15.pem
lrwxrwxrwx 1 root root 40 fév 7 10:18 fullchain.pem -> …/…/archive/trasis.com/fullchain15.pem
lrwxrwxrwx 1 root root 38 fév 7 10:18 privkey.pem -> …/…/archive/trasis.com/privkey15.pem

root@trasis:/etc/letsencrypt/archive/trasis.com# ls -lt
total 224
-rw-r–r-- 1 root root 0 jan 31 16:40 privkey15.pem
-rw-r–r-- 1 root root 0 jan 31 16:40 chain15.pem
-rw-r–r-- 1 root root 0 jan 31 16:40 fullchain15.pem
-rw-r–r-- 1 root root 0 jan 31 16:40 cert15.pem
-rw-r–r-- 1 root root 1899 jan 31 16:05 cert14.pem
-rw-r–r-- 1 root root 1647 jan 31 16:05 chain14.pem
-rw-r–r-- 1 root root 3546 jan 31 16:05 fullchain14.pem
-rw-r–r-- 1 root root 1704 jan 31 16:05 privkey14.pem
-rw-r–r-- 1 root root 1895 jan 28 12:55 cert13.pem
-rw-r–r-- 1 root root 1647 jan 28 12:55 chain13.pem
-rw-r–r-- 1 root root 3542 jan 28 12:55 fullchain13.pem
-rw-r–r-- 1 root root 1704 jan 28 12:55 privkey13.pem

Couldn’t I change manually the links so they point to a previous versions ?

14

Yes.

Take a backup first. :slightly_smiling_face:

It might also be necessary to delete the broken 15.pem files. I don't know how well Certbot would react to their presence next the the certificate is renewed or whatever. It might be fine, though.

15

Looks promising now :slight_smile:

root@trasis:/home/certbot# ./certbot-auto certonly -a webroot -w /home/data/trasis.com/wwwroot/ -d www.trasis.com -d trasis.com -d my.trasis.com --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Cert not due for renewal, but simulating renewal for dry run
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for my.trasis.com
http-01 challenge for trasis.com
http-01 challenge for www.trasis.com
Using the webroot path /home/data/trasis.com/wwwroot for all unmatched domains.
Waiting for verification…
Cleaning up challenges

IMPORTANT NOTES:
** - The dry run was successful.**

So, now I simply remove the --dry-run and it will work with the new certificates ?

I used a cronjob before to check every day if the certificates need to be renewed or not.
Can I simply do the same “certbot-auto renew” or do I have to specify the webroot and domain arguments like above ?

16

It should, yes. 

17

I run the command wihout --dry-run and it passed, but when I go to the website the expiration date of the certificates is still the old one 2019-05-01 instead of 2019-05-08 like shown in the program output:

root@trasis:/home/certbot# ./certbot-auto certonly -a webroot -w /home/data/trasis.com/wwwroot/ -d www.trasis.com -d trasis.com -d my.trasis.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Cert not yet due for renewal

You have an existing certificate that has exactly the same domains or certificate name you requested and isn’t close to expiry.
(ref: /etc/letsencrypt/renewal/www.trasis.com.conf)

What would you like to do?

1: Keep the existing certificate for now
2: Renew & replace the cert (limit ~5 per 7 days)

Select the appropriate number [1-2] then [enter] (press ‘c’ to cancel): 2
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for my.trasis.com
http-01 challenge for trasis.com
http-01 challenge for www.trasis.com
Using the webroot path /home/data/trasis.com/wwwroot for all unmatched domains.
Waiting for verification…
Cleaning up challenges

IMPORTANT NOTES:

  • Congratulations! Your certificate and chain have been saved at:
    /etc/letsencrypt/live/www.trasis.com/fullchain.pem
    Your key file has been saved at:
    /etc/letsencrypt/live/www.trasis.com/privkey.pem
    Your cert will expire on 2019-05-08. To obtain a new or tweaked
    version of this certificate in the future, simply run certbot-auto
    again. To non-interactively renew all of your certificates, run
    “certbot-auto renew”

  • If you like Certbot, please consider supporting our work by:

    Donating to ISRG / Let’s Encrypt: https://letsencrypt.org/donate
    Donating to EFF: https://eff.org/donate-le

Should I remove the certonly argument or how will the new certificates be applied (or is it normal) ?

In my cronjob, can is simply put “certbot-auto renew” ?

18

I think it works fine now. Many thanks for your help !

root@trasis:/home/certbot# ./certbot-auto renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/www.trasis.com.conf

Cert not yet due for renewal

Processing /etc/letsencrypt/renewal/my.trasis.com.conf

Cert not yet due for renewal

Processing /etc/letsencrypt/renewal/trasis.com.conf

Cert not yet due for renewal

The following certs are not due for renewal yet:
/etc/letsencrypt/live/www.trasis.com/fullchain.pem expires on 2019-05-08 (skipped)
/etc/letsencrypt/live/my.trasis.com/fullchain.pem expires on 2019-04-28 (skipped)
/etc/letsencrypt/live/trasis.com/fullchain.pem expires on 2019-05-01 (skipped)
No renewals were attempted.

1 Like
19

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.