Hostname(s) certbot-auto tries to connect to

[alexs77]

Hi

To access the Internet from our servers, we need to use a proxy. So that we don’t have to specify username and password, we have to predefine the hostnames, that are to be allowed.

I had our proxy team add acme-v01.api.letsencrypt.org to the list of allowed target sites. And when I use wget/curl, I can connect just fine from this Ubuntu 14.04 (yep… still…) system:

$ env https_proxy=http://192.168.177.23:8080 wget https://acme-v01.api.letsencrypt.org
--2016-09-12 15:19:54--  https://acme-v01.api.letsencrypt.org/
Connecting to 192.168.177.23:8080... connected.
Proxy request sent, awaiting response... 200 OK
Length: 2175 (2.1K) [text/html]
Saving to: ‘index.html.6’

100%[========================>] 2'175       --.-K/s   in 0s      

2016-09-12 15:19:55 (35.6 MB/s) - ‘index.html.6’ saved [2175/2175]

But when I try to use certbot-auto, it fails like this:

# env {http{s,}_proxy,HTTP{S,}_PROXY}=http://192.168.177.23:8080 certbot-auto --non-interactive certonly --webroot --expand -w "/var/www/well-known/$domain" -d "$domain" --no-self-upgrade
…
libssl-dev is already the newest version.
python-virtualenv is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Creating virtual environment...
Installing Python packages...
Traceback (most recent call last):
  File "/tmp/tmp.xtexkTalQu/pipstrap.py", line 146, in <module>
    exit(main())
  File "/tmp/tmp.xtexkTalQu/pipstrap.py", line 130, in main
    for url, digest in PACKAGES]
  File "/tmp/tmp.xtexkTalQu/pipstrap.py", line 112, in hashed_download
    response = opener().open(url)
  File "/usr/lib/python2.7/urllib2.py", line 404, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 422, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1222, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1184, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error Tunnel connection failed: 407 Proxy Authorization Required>

What’s happening there? What URL is certbot-auto trying to connect to?
And it was also visible, that it did do an “apt update”. What for? How to make it NOT do this?

Thanks,
Alexander

[cool110]

certbot-auto updates itself unless run with the --no-self-upgrade flag. apt update is used for the dependences and certbot updates itself from the Python Package Index and GitHub.

[alexs77]

I invoked certbot-auto with the --no-self-upgrade flag. It was the last parameter.

The position of the flag makes no difference at all. I tried:

# env {http{s,}_proxy,HTTP{S,}_PROXY}=http://192.168.177.23:8080 certbot-auto --no-self-upgrade --verbose --non-interactive certonly --webroot --expand -w "/var/www/well-known/$domain" -d "$domain"

As you can see, it’s now the 1st parameter.

And finally, there’s nothing to self update. 1 minute ago, I went to https://dl.eff.org/certbot-auto and got the most up to date version.

There’s something else going on.

[pfg]

certbot-auto is a bash script that installs dependencies that are required in order to run certbot. When you first run certbot-auto, or when there’s a new version of certbot available (unless you use --no-self-upgrade), it adds a number of OS packages via apt-get and installs some python packages certbot depends on.

apt-get requires access to the hostnames in /etc/apt/sources.list (I’m guessing you’ve already got a working proxy setup for that bit). The python packages are downloaded from https://pypi.python.org/. You probably also need access to https://dl.eff.org for the self-upgrade functionality. There’s also a reference to https://raw.githubusercontent.com in the code, but I’m not sure when or if that’s used, so I’d try without and see if it works.

Generally speaking, obtaining certbot via your native package manager is preferable and avoids these issues. certbot-auto is a bit of a stop-gap until native packages are available for all mainstream distros.

Unfortunately, there’s no native package for Ubuntu 14.04 at the moment, though there’s an older version of certbot - under the previous name letsencrypt - available for 16.04, as well as a certbot package in Debian backports.

[alexs77]

Hi

Okay, thanks a lot for the explanation. It sounds very complex. I am not
really sure, whether we, as a hoster, are comfortable with such a script,
which downloads things in such a way from the internet.

Isn’t there another official tool, which I could compile, maybe, and use on
Ubuntu Xenial, Trusty and which is also available for FreeBSD 10.x?

Or a much more simple shell script, which doesn’t download things in such a
wild way?

Viele Grüße

Alexander Skwar

[serverco]

Have a look at the list of alternate scripts, the Bash ones may fit your requirements. I’m slightly biased having written one of the bash scripts (getssl, which will certainly run on all the platforms you mention, normally without downloading anything additional to a standard server) - I suspect all of them will run on those platforms though, so have a look which best meets your requirements.