Help with certbot on the new "Amazon Linux 2"


#1

I can’t seem to get certbot to run on the new “Amazon Linux 2”

When I try this:

 wget https://dl.eff.org/certbot-auto
 chmod a+x certbot-auto
./certbot-auto'

I get this:

Sorry, I don't know how to bootstrap Certbot on your operating system!

When I try this:

yum install pip
yum install python-pip
pip install cryptography 
pip install certbot
yum install python-urllib3
yum install augeas
/usr/bin/certbot

I get this:

Traceback (most recent call last):
  File "/usr/bin/certbot", line 7, in <module>
    from certbot.main import main
  File "/usr/lib/python2.7/site-packages/certbot/main.py", line 19, in <module>
    from certbot import client
  File "/usr/lib/python2.7/site-packages/certbot/client.py", line 11, in <module>
    from acme import client as acme_client
  File "/usr/lib/python2.7/site-packages/acme/client.py", line 34, in <module>
    import urllib3.contrib.pyopenssl  # pylint: disable=import-error
  File "/usr/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py", line 50, in <module>
    from ndg.httpsclient.ssl_peer_verification import SUBJ_ALT_NAME_SUPPORT
ImportError: No module named ndg.httpsclient.ssl_peer_verification

Any help would be of great help!


Amazon Linux 2 certbot error
#2

Hi @markengelhardt,

I’m sorry to say that we have not been able to make Certbot reliable on Amazon Linux so far. The environment is a bit different from other Linux distributions and we have not been able to find folks to help coordinate on the Amazon side.

If you search this forum, you can see other people who have found certain packaging workarounds which often allowed them to run on Amazon Linux—but I can’t guarantee that those workarounds are applicable to your situation.

If you take a look at

you can see that there are a lot of other client applications, most of which have fewer dependencies than Certbot and hence are easier to use on a wider range of OSes. Particularly the shell-based clients tend to be extremely portable. The main limitation is that most client applications, unlike Certbot, don’t try to edit your web server configuration for you in order to install the certificate after it’s been obtained. Therefore, when using another client, you’ll have to edit your web server configuration yourself once the certificate has been issued.


#3

Schoen,

I have had good luck with Certbot on Amazon Linux… I have learned to never let it auto update after I get it working though. BUT… this is Amazon Linux 2 its all new and looks more like Centos 7. I would think it should not be that hard to get it going. Right now it gives up before it even tries.

Let me know if you want me to test something for you. Or if you have a suggestion I can try.


#4

@bmw, any thoughts about the new Amazon Linux?


#5

certbot-auto uses /etc/issue and various /etc/*release files to determine the system it’s on. On Amazon Linux 2, certbot-auto doesn’t recognize the layout as it has changed from previous versions.

I’ve included instructions of how to make certbot-auto try installation on Amazon Linux 2 below, however, if you’re able to enable the EPEL7 repo as this article suggests, I’d encourage you install Certbot from there. Recent versions of Certbot are packaged in EPEL7 and using this repository should be preferred over certbot-auto.

If you want to use certbot-auto though, the problem is that previously for Amazon Linux /etc/issue contained the string “Amazon Linux” but it doesn’t anymore. To make certbot-auto try the Amazon Linux bootstrapping, you can add “Amazon Linux” to /etc/issue or create the file /etc/redhat-release. Red Hat and Amazon Linux bootstrapping are identical.


#6

Just filed this ticket with a working patch


#7

Someone asked me for more details on the cert installation procedure, so I’m sharing it with the community here.

1. Launch EC2 with Amazon Linux 2 and relevant Security Groups (I won’t go into the details here in order to focus on certbot-auto)

2. I don’t know whether you’re starting from scratch or adapting your existing web server setup to HTTPS. Although I’m going to use an empty environment just for this walkthrough, in case you don’t want to stop your existing setup, I’m setting up a toy HTTP server in Python and pretend that’s my production setup which shouldn’t be interfered by certbot-auto.

[ec2-user@ip-172-31-11-240 ~]$ ls
bak
[ec2-user@ip-172-31-11-240 ~]$ mkdir www
[ec2-user@ip-172-31-11-240 ~]$ ls -lt
total 0
drwxrwxr-x 2 ec2-user ec2-user  6 Feb  6 15:21 www
drwxrwxr-x 3 ec2-user ec2-user 37 Feb  6 15:21 bak
[ec2-user@ip-172-31-11-240 ~]$ cd www/
[ec2-user@ip-172-31-11-240 www]$ cat <<EOS > index.html
> <!DOCTYPE html>
> <html>
> <head>
> <title>Testing Let's Encrypt</title>
> </head>
> <body>
> <h1>We are testing Let's Encrypt</h1>
> </body>
> </html>
> EOS
[ec2-user@ip-172-31-11-240 www]$ wc index.html
  9  16 136 index.html
[ec2-user@ip-172-31-11-240 www]$ python2 -V
Python 2.7.5
[ec2-user@ip-172-31-11-240 www]$ sudo python2 -m SimpleHTTPServer 80 &
[1] 3373
[ec2-user@ip-172-31-11-240 www]$ Serving HTTP on 0.0.0.0 port 80 ...

The front page should be accessible from the Internet at this point.

$ curl http://2718.ml/
<!DOCTYPE html>
<html>
<head>
<title>Testing Let's Encrypt</title>
</head>
<body>
<h1>We are testing Let's Encrypt</h1>
</body>
</html>

(So we’re going to use the http-01 challenge type. Cf. https://certbot.eff.org/docs/using.html#getting-certificates-and-choosing-plugins )

3. Download the installer

[ec2-user@ip-172-31-11-240 www]$ cd
[ec2-user@ip-172-31-11-240 ~]$ curl -OL https://dl.eff.org/certbot-auto
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 61939  100 61939    0     0  61939      0  0:00:01 --:--:--  0:00:01  203k

4. Adapt it to Amazon Linux 2 (and compare it with the original version)

[ec2-user@ip-172-31-11-240 ~]$ cp certbot-auto certbot-auto.orig
[ec2-user@ip-172-31-11-240 ~]$ vim certbot-auto
... hack hack ...
[ec2-user@ip-172-31-11-240 ~]$ diff -U3 certbot-auto.orig certbot-auto
--- certbot-auto.orig   2018-02-06 15:34:06.915133686 +0000
+++ certbot-auto        2018-02-06 15:40:17.978336144 +0000
@@ -837,7 +837,8 @@
     DeprecationBootstrap "macOS" BootstrapMac
   }
   BOOTSTRAP_VERSION="BootstrapMac $BOOTSTRAP_MAC_VERSION"
-elif [ -f /etc/issue ] && grep -iq "Amazon Linux" /etc/issue ; then
+elif grep -i "Amazon Linux" /etc/issue > /dev/null 2>&1 || \
+    grep 'cpe:.*:amazon_linux:2' /etc/os-release > /dev/null 2>&1; then
   Bootstrap() {
     ExperimentalBootstrap "Amazon Linux" BootstrapRpmCommon
   }

5. Now it’s time to run it and get a certificate!
Again, we’re using the http-01 challenge type (--webroot) as that’s the only way in this web server setup; neither TLS-SNI nor DNS are under our control.
Also, certbot doesn’t (can’t) support automatic SSL cert configuration (certbot run) for this type of web server (in Python!) so it can only obtain certs (certbot certonly.)

I got a throw-away domain name 2718.ml for free for this walkthrough.
Note a single cert can support both of www.2718.ml and 2718.ml (as many as you wish.)

[ec2-user@ip-172-31-11-240 ~]$ sudo ./certbot-auto certonly --webroot -w ~/www/ -d 2718.ml -d www.2718.ml
FATAL: Amazon Linux support is very experimental at present...
if you would like to work on improving it, please ensure you have backups
and then run this script again with the --debug flag!
Alternatively, you can install OS dependencies yourself and run this script
again with --no-bootstrap.

6. OK. I see.

[ec2-user@ip-172-31-11-240 ~]$ sudo ./certbot-auto certonly --debug --webroot -w ~/www/ -d 2718.ml -d www.2718.ml
Bootstrapping dependencies for Amazon... (you can skip this with --no-bootstrap)
yum is /bin/yum                  
yum is hashed (/bin/yum)         
Loaded plugins: langpacks, priorities, update-motd
amzn2-core                                                                                                                                                                                                                                                           | 2.0 kB  00:00:00
Package gcc-7.2.1-2.amzn2.0.2.x86_64 already installed and latest version
Package augeas-libs-1.4.0-2.amzn2.2.x86_64 already installed and latest version
Package 1:openssl-1.0.2k-8.amzn2.0.1.x86_64 already installed and latest version
Package 1:openssl-devel-1.0.2k-8.amzn2.0.1.x86_64 already installed and latest version
Package libffi-devel-3.0.13-18.amzn2.x86_64 already installed and latest version
Package system-rpm-config-9.1.0-76.amzn2.0.1.noarch already installed and latest version
Package ca-certificates-2017.2.14-71.amzn2.noarch already installed and latest version
Package python-devel-2.7.5-58.amzn2.x86_64 already installed and latest version
Package python-virtualenv-1.10.1-4.amzn2.noarch already installed and latest version
Package python-tools-2.7.5-58.amzn2.x86_64 already installed and latest version
Package python2-pip-9.0.1-9.amzn2.0.2.noarch already installed and latest version
Nothing to do                    
Creating virtual environment...  
Installing Python packages...    
Installation succeeded.          
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Cert not yet due for renewal     
                                 
You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
(ref: /etc/letsencrypt/renewal/2718.ml.conf)
                                 
What would you like to do?       
-------------------------------------------------------------------------------
1: Keep the existing certificate for now
2: Renew & replace the cert (limit ~5 per 7 days)
-------------------------------------------------------------------------------
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Renewing an existing certificate 
Performing the following challenges:
http-01 challenge for 2718.ml    
http-01 challenge for www.2718.ml
Using the webroot path /home/ec2-user/www for all unmatched domains.
Waiting for verification...      
Cleaning up challenges           
                                 
IMPORTANT NOTES:                 
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/2718.ml/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/2718.ml/privkey.pem
   Your cert will expire on 2018-05-07. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot-auto
   again. To non-interactively renew *all* of your certificates, run
   "certbot-auto renew"          
 - If you like Certbot, please consider supporting our work by:
                                 
   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

YMMV because it was actually the second time for me to run certbot for 2718.ml. But the process should complete within a few seconds.

7. Let’s try it out

Thanks to https://gist.github.com/dergachev/7028596 and How do I map the pem files to python's SSLSocket parameters

[ec2-user@ip-172-31-11-240 www]$ cat ~/https.py
#!/usr/bin/env python2

import BaseHTTPServer, SimpleHTTPServer
import ssl

httpd = BaseHTTPServer.HTTPServer(('0', 443), SimpleHTTPServer.SimpleHTTPRequestHandler)
httpd.socket = ssl.wrap_socket(httpd.socket, certfile='/etc/letsencrypt/live/2718.ml/fullchain.pem',
                               keyfile='/etc/letsencrypt/live/2718.ml/privkey.pem',
                               server_side=True)
httpd.serve_forever()
[ec2-user@ip-172-31-11-240 www]$ sudo ../https.py

On my PC (Debian testing/buster x86_64):

$ curl -v --tlsv1.2 https://www.2718.ml/
*   Trying 54.169.9.49...
* TCP_NODELAY set
* Connected to www.2718.ml (54.169.9.49) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=2718.ml
*  start date: Feb  6 14:46:39 2018 GMT
*  expire date: May  7 14:46:39 2018 GMT
*  subjectAltName: host "www.2718.ml" matched cert's "www.2718.ml"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
> GET / HTTP/1.1
> Host: www.2718.ml
> User-Agent: curl/7.58.0
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Server: SimpleHTTP/0.6 Python/2.7.5
< Date: Tue, 06 Feb 2018 16:02:27 GMT
< Content-type: text/html
< Content-Length: 136
< Last-Modified: Tue, 06 Feb 2018 15:30:36 GMT
<
<!DOCTYPE html>
<html>
<head>
<title>Testing Let's Encrypt</title>
</head>
<body>
<h1>We are testing Let's Encrypt</h1>
</body>
</html>
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, Client hello (1):

:yum::+1::tada:


#8

@nodakai - Thank you very much.


#9

Amazing, Thank you very much. I was looking for this for hours!
This no-bootstrap flag is a lifesaver :slight_smile:


#10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.