AH00072: make_sock: could not bind to address [::]:80 - Varnish / Magento

Hello everybody,

I’v been trying to solve my problem since last week, without success. I installed Magento with SSL and everything worked fine. Then, I installed Varnish cache, and then my certbot didn’t work anymore. My cron job for certbot renewal (every 12 hours) made my website down : it works again with a simple apache restart, but my SSL certificate isn’t renew.

Here my configuration :

My domain is: [redacted]
I ran this command: sudo certbot renew

It produced this output:
Attempting to renew cert ([redacted]) from /etc/letsencrypt/renewal/[redacted].conf produced an unexpected error: Error while running apache2ctl graceful.
httpd not running, trying to start
Action ‘graceful’ failed.
The Apache error log may have more information.

AH00548: NameVirtualHost has no effect and will be removed in the next release /etc/apache2/ports.conf:9
(98)Address already in use: AH00072: make_sock: could not bind to address [::]:80
(98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
AH00015: Unable to open logs
. Skipping.

My web server is (include version): Apache2

The operating system my web server runs on is (include version): Ubuntu 16.02

I can login to a root shell on my machine: YES

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 0.31.0

Here my configuration files :

default.conf

Summary

`<VirtualHost 127.0.0.1:8080>
# The ServerName directive sets the request scheme, hostname and port that
# the server uses to identify itself. This is used when creating
# redirection URLs. In the context of virtual hosts, the ServerName
# specifies what hostname must appear in the request’s Host: header to
# match this virtual host. For the default virtual host (this file) this
# value is not decisive as it is used as a last resort host regardless.
# However, you must set it for any further virtual host explicitly.
ServerName www.[redacted]
ServerAlias www.[redacted]
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html

# Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
# error, crit, alert, emerg.
# It is also possible to configure the loglevel for particular
# modules, e.g.
#LogLevel info ssl:warn

ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined

# For most configuration files from conf-available/, which are
# enabled or disabled at a global level, it is possible to
# include a line for only one particular virtual host. For example the
# following line enables the CGI configuration for this host only
# after it has been globally disabled with "a2disconf".
#Include conf-available/serve-cgi-bin.conf

vim: syntax=apache ts=4 sw=4 sts=4 sr noet

<Directory “/var/www/html”>
Options Indexes FollowSymLinks MultiViews
AllowOverride All
Order allow,deny
allow from all

`

default-ssl.conf

Summary

`
<VirtualHost *:443>
# The ServerName directive sets the request scheme, hostname and port that
# the server uses to identify itself. This is used when creating
# redirection URLs. In the context of virtual hosts, the ServerName
# specifies what hostname must appear in the request’s Host: header to
# match this virtual host. For the default virtual host (this file) this
# value is not decisive as it is used as a last resort host regardless.
# However, you must set it for any further virtual host explicitly.
#ServerName www.example.com

ServerAdmin webmaster@localhost
DocumentRoot /var/www/html

# Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
# error, crit, alert, emerg.
# It is also possible to configure the loglevel for particular
# modules, e.g.
#LogLevel info ssl:warn

ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined

# For most configuration files from conf-available/, which are
# enabled or disabled at a global level, it is possible to
# include a line for only one particular virtual host. For example the
# following line enables the CGI configuration for this host only
# after it has been globally disabled with "a2disconf".
#Include conf-available/serve-cgi-bin.conf

ServerName [redacted]

ProxyPreserveHost On
ProxyPass / http://127.0.0.1:80/
ProxyPassReverse / http://127.0.0.1:80/
RequestHeader set X-Forwarded-Port “443”
RequestHeader set X-Forwarded-Proto “https”

SSLEngine On
Include /etc/letsencrypt/options-ssl-apache.conf
ServerAlias www.[redacted]
SSLCertificateFile /etc/letsencrypt/live/[redacted]/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/[redacted]/privkey.pem

Options Indexes FollowSymLinks MultiViews AllowOverride All Order allow,deny allow from all `

ports.conf

Summary

`# If you just change the port or add more ports here, you will likely also

have to change the VirtualHost statement in

/etc/apache2/sites-enabled/000-default.conf

Listen *:8080

Listen 443 NameVirtualHost *:443 Listen 443 NameVirtualHost *:443

vim: syntax=apache ts=4 sw=4 sts=4 sr noet

`

default.vcl

Summary

`vcl 4.0;

import std;

The minimal Varnish version is 4.0

For SSL offloading, pass the following header in your proxy server or load balancer: ‘X-Forwarded-Proto: https’

backend default {
.host = “51.83.33.32”;
.port = “8080”;
.first_byte_timeout = 600s;
}

acl purge {
“51.83.33.32”;
}

sub vcl_recv {

if (req.method == "PURGE") {
    if (client.ip !~ purge) {
        return (synth(405, "Method not allowed"));
    }
    # To use the X-Pool header for purging varnish during automated deployments, make sure the X-Pool header
    # has been added to the response in your backend server config. This is used, for example, by the
    # capistrano-magento2 gem for purging old content from varnish during it's deploy routine.
    if (!req.http.X-Magento-Tags-Pattern && !req.http.X-Pool) {
        return (synth(400, "X-Magento-Tags-Pattern or X-Pool header required"));
    }
    if (req.http.X-Magento-Tags-Pattern) {
      ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern);
    }
    if (req.http.X-Pool) {
      ban("obj.http.X-Pool ~ " + req.http.X-Pool);
    }
    return (synth(200, "Purged"));
}

if (req.method != "GET" &&
    req.method != "HEAD" &&
    req.method != "PUT" &&
    req.method != "POST" &&
    req.method != "TRACE" &&
    req.method != "OPTIONS" &&
    req.method != "DELETE") {
      /* Non-RFC2616 or CONNECT which is weird. */
      return (pipe);
}

# We only deal with GET and HEAD by default
if (req.method != "GET" && req.method != "HEAD") {
    return (pass);
}

# Bypass shopping cart, checkout and search requests
if (req.url ~ "/checkout" || req.url ~ "/catalogsearch") {
    return (pass);
}

# Bypass Phpmyadmin
if (req.url ~ "^/phpmyadmin") {
   return(pass);
}

# Bypass health check requests
if (req.url ~ "/pub/health_check.php") {
    return (pass);
}

# Set initial grace period usage status
set req.http.grace = "none";

# normalize url in case of leading HTTP scheme and domain
set req.url = regsub(req.url, "^http[s]?://", "");

# Compression filter. See https://www.varnish-cache.org/trac/wiki/FAQ/Compression
if (req.http.Accept-Encoding) {
    if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|flv)$") {
        # No point in compressing these
        unset req.http.Accept-Encoding;
    } elsif (req.http.Accept-Encoding ~ "gzip") {
        set req.http.Accept-Encoding = "gzip";
    } elsif (req.http.Accept-Encoding ~ "deflate" && req.http.user-agent !~ "MSIE") {
        set req.http.Accept-Encoding = "deflate";
    } else {
        # unkown algorithm
        unset req.http.Accept-Encoding;
    }
}

# Remove Google gclid parameters to minimize the cache objects
set req.url = regsuball(req.url,"\?gclid=[^&]+$",""); # strips when QS = "?gclid=AAA"
set req.url = regsuball(req.url,"\?gclid=[^&]+&","?"); # strips when QS = "?gclid=AAA&foo=bar"
set req.url = regsuball(req.url,"&gclid=[^&]+",""); # strips when QS = "?foo=bar&gclid=AAA" or QS = "?foo=bar&gclid=AAA&bar=baz"

# Static files caching
if (req.url ~ "^/(pub/)?(media|static)/") {
    # Static files should not be cached by default
    # return (pass);

    # But if you use a few locales and don't use CDN you can enable caching static files by commenting previous line (#return (pass);) and uncommenting next 3 lines
    unset req.http.Https;
    unset req.http.X-Forwarded-Proto;
    unset req.http.Cookie;
}

return (hash);

}

sub vcl_hash {
if (req.http.cookie ~ “X-Magento-Vary=”) {
hash_data(regsub(req.http.cookie, “^.?X-Magento-Vary=([^;]+);.*$”, “\1”));
}

# For multi site configurations to not cache each other's content
if (req.http.host) {
    hash_data(req.http.host);
} else {
    hash_data(server.ip);
}

# To make sure http users don't see ssl warning
if (req.http.X-Forwarded-Proto) {
    hash_data(req.http.X-Forwarded-Proto);
}

}

sub vcl_backend_response {

set beresp.grace = 3d;

if (beresp.http.content-type ~ "text") {
    set beresp.do_esi = true;
}

if (bereq.url ~ "\.js$" || beresp.http.content-type ~ "text") {
    set beresp.do_gzip = true;
}

if (beresp.http.X-Magento-Debug) {
    set beresp.http.X-Magento-Cache-Control = beresp.http.Cache-Control;
}

# cache only successfully responses and 404s
if (beresp.status != 200 && beresp.status != 404) {
    set beresp.ttl = 0s;
    set beresp.uncacheable = true;
    return (deliver);
} elsif (beresp.http.Cache-Control ~ "private") {
    set beresp.uncacheable = true;
    set beresp.ttl = 86400s;
    return (deliver);
}

# validate if we need to cache it and prevent from setting cookie
# images, css and js are cacheable by default so we have to remove cookie also
if (beresp.ttl > 0s && (bereq.method == "GET" || bereq.method == "HEAD")) {
    unset beresp.http.set-cookie;
}

If page is not cacheable then bypass varnish for 2 minutes as Hit-For-Pass

if (beresp.ttl <= 0s ||
beresp.http.Surrogate-control ~ “no-store” ||
(!beresp.http.Surrogate-Control &&
beresp.http.Cache-Control ~ “no-cache|no-store”) ||
beresp.http.Vary == “*”) {
# Mark as Hit-For-Pass for the next 2 minutes
set beresp.ttl = 120s;
set beresp.uncacheable = true;
}

return (deliver);

}

sub vcl_deliver {
if (resp.http.X-Magento-Debug) {
if (resp.http.x-varnish ~ " ") {
set resp.http.X-Magento-Cache-Debug = “HIT”;
set resp.http.Grace = req.http.grace;
} else {
set resp.http.X-Magento-Cache-Debug = “MISS”;
}
} else {
unset resp.http.Age;
}
}
`

If you need more information, do not hesitate !

Hi @deude

what's the content of your renewal file?

Check

/etc/letsencrypt/renewal

Hi @JuergenAuer !

Thanks for your response !

Here my files :

www.[redacted].conf

Summary

`# renew_before_expiry = 30 days
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/www.[redacted]
cert = /etc/letsencrypt/live/www.[redacted]/cert.pem
privkey = /etc/letsencrypt/live/www.[redacted]/privkey.pem
chain = /etc/letsencrypt/live/www.[redacted]/chain.pem
fullchain = /etc/letsencrypt/live/www.[redacted]/fullchain.pem

Options used in the renewal process

[renewalparams]
account = 3c06419[…]18 #I’m not sure I can display the account ID
authenticator = apache
server = https://acme-v02.api.letsencrypt.org/directory
installer = apache
`

[redacted].conf

Summary

`# renew_before_expiry = 30 days
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/[redacted]
cert = /etc/letsencrypt/live/[redacted]/cert.pem
privkey = /etc/letsencrypt/live/[redacted]/privkey.pem
chain = /etc/letsencrypt/live/[redacted]/chain.pem
fullchain = /etc/letsencrypt/live/[redacted]/fullchain.pem

Options used in the renewal process

[renewalparams]
installer = apache
account = 3c064199[…]018 #I’m not sure I can display the account ID
authenticator = apache
server = https://acme-v02.api.letsencrypt.org/directory`

If you are using Varnish (on port 80), it’s presumably in front of/proxying for Apache, right?

In that case, try:

certbot renew --cert-name www.[redacted] --http-01-port 8080 --dry-run

The reason we use 8080 is because that’s where your Apache server is running according to your default.vcl.

When Certbot modifies your Apache config during authentication, it searches for virtual hosts that match that port number, and adds one if it doesn’t exist. When the port number is the default (80), this obviously causes a conflict, since Apache isn’t meant to listen on port 80 in your setup.

You use the apache authenticator. But that can't work, if port 80 is blocked and Apache uses another port.

So

  • use the solution of @_az (or)
  • switch to webroot, so Certbot can find the root of your Apache and doesn't touch / check your Apache port configuration

Thanks guys !

I read that the --http-01-port 8080 was obsolet so I didn’t even try …

When I try the :
certbot renew --cert-name www.[redacted] --http-01-port 8080 --dry-run

I have the following error :

    Attempting to renew cert ([redacted]) from /etc/letsencrypt/renewal/[redacted].conf produced an unexpected error: Failed authorization procedure. [redacted] (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://[redacted]/coming/soon.html [51.83.33.32]: "<!DOCTYPE html>\r\n<html lang=\"fr-fr\" class=\"no-js\">\r\n\r\n\t<head>\r\n\t\t<meta charset=\"utf-8\">\r\n        <title>[redacted] - Ventes pri", fan.[redacted] (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://[redacted]/coming/soon.html [51.83.33.32]: "<!DOCTYPE html>\r\n<html lang=\"fr-fr\" class=\"no-js\">\r\n\r\n\t<head>\r\n\t\t<meta charset=\"utf-8\">\r\n        <title>[redacted] - Ventes pri". Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/[redacted]/fullchain.pem (failure)

And if I change Apache for webroot, I have the following error :

Attempting to renew cert ([redacted]) from /etc/letsencrypt/renewal/[redacted].conf produced an unexpected error: An object does not implement interface <InterfaceClass certbot.interfaces.IInstaller>

    . Skipping.

All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/[redacted]/fullchain.pem (failure)

There is a redirect to https. Looks like that doesn't work.

Share your complete command. Looks more like a typo. Certbot 0.31 should have the webroot.

I just change apache for webroot with a :
nano /etc/letsencrypt/renewal/[redacted].conf

And then I tried the certbot renew :
certbot renew --cert-name [redacted] --http-01-port 8080

I’m sorry, I’m a bit newbie with ssh and certbot …

If you use webroot and the port, that can't work. These are different things. And you need a -w parameter.

Please read

https://certbot.eff.org/docs/using.html

to learn the commands and combinations you can use

Thanks !

So I tried this :
certbot certonly --webroot -w /var/www/html -d [redacted]

And I have the following errors :

Summary
Failed authorization procedure. [redacted] (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://[redacted]/coming/soon.html [51.83.33.32]: "<!DOCTYPE html>\r\n<html lang=\"fr-fr\" class=\"no-js\">\r\n\r\n\t<head>\r\n\t\t<meta charset=\"utf-8\">\r\n        <title>[redacted] - Ventes pri"

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: [redacted]
   Type:   unauthorized
   Detail: Invalid response from
   https://[redacted]/coming/soon.html [51.83.33.32]: "<!DOCTYPE
   html>\r\n<html lang=\"fr-fr\"
   class=\"no-js\">\r\n\r\n\t<head>\r\n\t\t<meta
   charset=\"utf-8\">\r\n        <title>[redacted] - Ventes pri"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

I can’t understand why this is so hard, I just added the varnish cache …

Read your output or check the domain with online tools - https://check-your-website.server-daten.de/?q=[redacted]

There is a redirect to /coming/soon.html - that's not the content of the validation file Letsencrypt expects.

Domainname Http-Status redirect Sec. G
• http://[redacted]/
51.83.33.32 302 https://[redacted]/coming/soon.html 0.053 A
http://www.[redacted]/
51.83.33.32 302 https://[redacted]/coming/soon.html 0.043 E
http://www.[redacted]/
2001:41d0:404:200::1fab -14 10.023 T
Timeout - The operation has timed out
• https://[redacted]/
51.83.33.32 302 https://[redacted]/coming/soon.html 3.250 B
https://www.[redacted]/
51.83.33.32 302 https://[redacted]/coming/soon.html 3.214 B
https://www.[redacted]/
2001:41d0:404:200::1fab -14 10.034 T
Timeout - The operation has timed out
• https://[redacted]/coming/soon.html 200 3.144 I
• http://[redacted]/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
51.83.33.32 302 https://[redacted]/coming/soon.html 0.047 A
Visible Content: Found The document has moved here . Apache/2.4.18 (Ubuntu) Server at [redacted] Port 80
http://www.[redacted]/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
51.83.33.32 302 https://[redacted]/coming/soon.html 0.057 E
Visible Content: Found The document has moved here . Apache/2.4.18 (Ubuntu) Server at www.[redacted] Port 80
http://www.[redacted]/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
2001:41d0:404:200::1fab -14 10.027 T
Timeout - The operation has timed out
Visible Content:

/.well-known/acme-challenge is redirected to /coming/soon.html.

And your configuration isn't good. The non-www version has only ipv4, the www version has ipv4 and ipv6. Add ipv6 to your non-www version, configure your ipv6 and create one certificate with both domain names.

2 Likes

Ok, I didn’t think it was a problem with the same URL : so I excluded the Localhost IP from redirection in my .htaccess and no more error !

I create a new certificate without error, and I deleted the old one. I will do some tests in the next few days but everything seems good now.

I didn’t change anything with the IPV6, I don’t know what to do and how to do, so if you have a link or an explanation, I’m in !

Thank you very much @JuergenAuer for your help, problem solved ! :slight_smile:

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.