Problems with the IP address local versus global : how to make cerbot working?

marcoippolito · April 11, 2019, 8:43am

Actually in the PC hosting the Web Server I have Ubuntu 18.04.01 Server Edition.
For browsing in internet and for connecting via ssh to the PC I use a Windows 10 laptop. But does it affect the PC hosting the web page? It shouldn’t.

These are the DNS settings of my Domain : DomainDNSSettings

Now I disabled :

the Karspersky Internet Security Firewall of my laptop:
KasperksyInternetSecurityFireWall
the overall Windows Defender Firewall of my laptop:
WindowsDefenderFirewall

But unfortunately again, refreshing or connecting with https://check-your-website.server-daten.de/?q=ggc.world it says:
“ConnectFailure - Unable to connect to the remote server No connection could be made because the target machine actively refused it 37.116.214.189:80”

and doing :
sudo certbot certonly --webroot -w /home/marco/go/src/MyPage/ -d ggc.world -d www.ggc.world
I got the same error message as above
even if I do not understand what it could interfere with the Internet Service Provider’s Security Configurations

marcoippolito · April 11, 2019, 8:48am

The only web server is running on:

httpaddr = 192.168.1.7
httpport = 8080

mnordhoff · April 11, 2019, 8:58am

If there's no server listening on the port, "connection refused" is what's supposed to happen.

marcoippolito · April 11, 2019, 9:07am

Changing in the configuration file of the web page the port into httpport = 80
when running the web server it says:

marco@pc01:~/go/src/MyPage$ bee run

| ___
| |/ / ___ ___
| ___ \ / _ \ / _
| |/ /| /| /
_/ _| __| v1.10.0
2019/04/11 09:00:56 INFO 0001 Using ‘MyPage’ as ‘appname’
2019/04/11 09:00:56 INFO 0002 Initializing watcher…
2019/04/11 09:00:57 SUCCESS 0003 Built Successfully!
2019/04/11 09:00:57 INFO 0004 Restarting ‘MyPage’…
2019/04/11 09:00:57 SUCCESS 0005 ‘./MyPage’ is running…
2019/04/11 09:00:57.225 [I] [asm_amd64.s:2361] http server Running on http://192.168.1.7:80
2019/04/11 09:00:57.225 [C] [asm_amd64.s:2361] ListenAndServe: listen tcp 192.168.1.7:80: bind: permission denied

I thought I could use any, almost any, ports for my web server, even if for Beego the listening port is set to 8080 by default.
So…should I use the Certbot’s standalone plugin, which temporarily runs an additional webserver, or should be better to run only Beego web server and find the way to modify the port to 80 instead of the 8080 ?

JuergenAuer · April 11, 2019, 9:13am

There is no new check. This is my old / first check.

Click "Check" to recheck your domain.

mnordhoff · April 11, 2019, 9:15am

Certbot’s standalone server would be easy to use.

I don’t know if Beego is designed to run on ports below 1024. Doing that securely is a bit complicated. I glanced at their documentation and didn’t see anything.

Beego’s intention might be that you run it on a high port, and then configure another server to reverse proxy to it. I don’t know.

marcoippolito · April 11, 2019, 9:24am

@JuergenAuer After clicking the check botton while keeping port 8080 for the webserver, even if Matt @mnordhoff explained me that Certbot requires a webserver on port 80, the same result happens: “ConnectFailure - Unable to connect to the remote server No connection could be made because the target machine actively refused it 37.116.214.189:80”.
So I guess I need to have a webserver in port 80.
Since I didn’t succeed in changing the Beego webserver http port from 8080 to 80, can I run two webserver at the same time, the Beego webserver in port 8080 and the Certbot’s webserver in port 80? or two webservers in two different ports in the same PC might generate undesirable side effects?

JuergenAuer · April 11, 2019, 9:31am

Yes, you need a running webserver port 80.

That should always work.

Install a simple Apache or nginx with a minimal configuration, then use that with --webroot.

Or use --standalone, but then it's difficult to debug.

marcoippolito · April 11, 2019, 9:49am

Between Apache ngnix which one is the simpler and lighter?
Thanks a lot both @mnordhoff for your kind tireless help

marcoippolito · April 11, 2019, 6:10pm

I installed the latest NGINX 1.155 version. Following the indications found in the Go Web Development Cookbook regarding the deployment of Beego with Nginx : DeployingBeegoAppWithNginx and the indications in here: https://beego.me/docs/deploy/nginx.md, I modified the /etc/nginx/conf.d/default.conf in this way :

server {
listen 80;
server_name ggc.world;

location / {
    proxy_pass http://192.168.1.7:80/;
}

error_page   500 502 503 504  /50x.html;
location = /50x.html {
    root   /usr/share/nginx/html;
}

}

server {
listen 80;
server_name www.ggc.world;

location / {
    proxy_pass http://192.168.1.7:80/;
}

error_page   500 502 503 504  /50x.html;
location = /50x.html {
    root   /usr/share/nginx/html;
}

}

sudo service nginx restart

marco@pc01:~/go/src/MyPage$ bee run

| ___
| |/ / ___ ___
| ___ \ / _ \ / _
| |/ /| /| /
_/ _| __| v1.10.0
2019/04/11 17:46:03 INFO 0001 Using ‘MyPage’ as ‘appname’
2019/04/11 17:46:03 INFO 0002 Initializing watcher…
2019/04/11 17:46:03 SUCCESS 0003 Built Successfully!
2019/04/11 17:46:03 INFO 0004 Restarting ‘MyPage’…
2019/04/11 17:46:03 SUCCESS 0005 ‘./MyPage’ is running…
2019/04/11 17:46:03.943 [I] [asm_amd64.s:2361] http server Running on http://192.168.1.7:8080

Then I ran --webroot :

marco@pc01:~/go/src/MyPage$ sudo certbot certonly --webroot -w /home/marco/go/src/MyPage -d ggc.world -d www.ggc.world
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for ggc.world
http-01 challenge for www.ggc.world
Using the webroot path /home/marco/go/src/MyPage for all unmatched domains.
Waiting for verification…
Cleaning up challenges
Failed authorization procedure. ggc.world (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://ggc.world/.well-known/acme-challenge/yTAdFkrEf94feHX4p2Shau_pJxglRxZq6P98al7ZVCc [37.116.214.189]: “\n\n\nError\n\n body {\n width: 35em;\n margin: 0 auto;\n font-”, www.ggc.world (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.ggc.world/.well-known/acme-challenge/SHWsx7hV4zmVgw-FTOhjIIrTvWQF9aDFIItbJmxqwvE [37.116.214.189]: “\n\n\nError\n\n body {\n width: 35em;\n margin: 0 auto;\n font-”

IMPORTANT NOTES:

The following errors were reported by the server:

Domain: ggc.world
Type: unauthorized
Detail: Invalid response from
http://ggc.world/.well-known/acme-challenge/yTAdFkrEf94feHX4p2Shau_pJxglRxZq6P98al7ZVCc
[37.116.214.189]: “\n\n\nError\n\n body {\n
width: 35em;\n margin: 0 auto;\n font-”

Domain: www.ggc.world
Type: unauthorized
Detail: Invalid response from
http://www.ggc.world/.well-known/acme-challenge/SHWsx7hV4zmVgw-FTOhjIIrTvWQF9aDFIItbJmxqwvE
[37.116.214.189]: “\n\n\nError\n\n body {\n
width: 35em;\n margin: 0 auto;\n font-”

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.

JuergenAuer · April 11, 2019, 6:26pm

Then check your webroot.

Create the two subdirectories

/home/marco/go/src/MyPage/.well-known/acme-challenge

there a file (file name 1234) and try to load that file via

http://ggc.world/.well-known/acme-challenge/1234

Or use an online tool to check the url with the subfolders.

mnordhoff · April 11, 2019, 9:32pm

Unless 192.167.1.7:80 is different, Nginx is configured to proxy to itself, not to proxy to Beego.

marcoippolito · April 12, 2019, 9:03am

marco@pc01:~/go/src/MyPage/.well-known$ mkdir acme-challenge
marco@pc01:~/go/src/MyPage/.well-known$ cd acme-challenge
marco@pc01:~/go/src/MyPage/.well-known/acme-challenge$ nano 1234

being the only content of the file 1234 the word “test”

Both with

location / {
    proxy_pass http://192.168.1.7:80;
}

and

location / {
    proxy_pass http://192.168.1.7:8080;
}

Using this online testing tool: https://pentest-tools.com/website-vulnerability-scanning/discover-hidden-directories-and-files#
the result of this test is negative: OnlineTestingToolResult
and trying to access the file from my mobile phone gives the same result: FileAccessTest
while the main page is actually visible online: FrontPageOnline

Putting

location / {
    proxy_pass ggc.world:8080;
}

location / {
    proxy_pass www.ggc.world:8080;
}

gives error when restarting nginx:

See “systemctl status nginx.service” and “journalctl -xe” for details.
failed!

@mnordhoff I’m trying to understand the rationale behing proxy_pass:
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass
“Sets the protocol and address of a proxied server and an optional URI to which a location should be mapped. As a protocol, “http” or “https” can be specified. The address can be specified as a domain name or IP address, and an optional port”
Is it correct to say that the proxy_pass, if properly set, establishes a connection between ngnix and beego?
I would like to use Beego as MVC Golang framework, but if there are potential problems and pitfalls in using Beego in conjunction with Nginx, I’m open to check other viable options.

marcoippolito · April 12, 2019, 3:16pm

@JuergenAuer I tried also with this service: https://www.sslforfree.com/create?domains=ggc.world+www.ggc.world

I downloaded both of files :

https://www.sslforfree.com/create?verification=O6c2rGTDjC3YoOHOzAAdI74ghWth0lVnlLA7mf38F-8
https://www.sslforfree.com/create?verification=NoAPrMQXxgYemO56I_VyRIrtX_9qtceTGxgmIdTf1CM
in the /home/marco/go/src/MyPage/.well-known/acme-challenge subfolder

To verify successful upload I clicked on these two links:
A) http://ggc.world/.well-known/acme-challenge/O6c2rGTDjC3YoOHOzAAdI74ghWth0lVnlLA7mf38F-8
B) http://www.ggc.world/.well-known/acme-challenge/NoAPrMQXxgYemO56I_VyRIrtX_9qtceTGxgmIdTf1CM
But for both of them I got : " 404 Not Found" message …
What could it be the causes of this problem?

JuergenAuer · April 12, 2019, 3:29pm

If these tests don't work, there are some options:

your webroot is wrong
you have additional definitions

But I don't understand your configuration with a nginx and that other webserver.

marcoippolito · April 12, 2019, 3:41pm

How can I check if my webroot is wrong?
Sorry for my knowledge-gap, which I’m trying to quickly fill up… what do you mean as “additional definitions”?

I checked also in this explanation: https://kenyaappexperts.com/blog/how-to-deploy-golang-to-production-step-by-step/ and the configuration in /etc/nginx/conf.d/default.conf :

    location / {
    #    root   /usr/share/nginx/html;
    #    index  index.html index.htm;
        proxy_pass http://ggc.world:8080;
     }

seems correct.

marcoippolito · April 13, 2019, 7:15am

@JuergenAuer this is the domain name’s configurations set in NameCheap.com :
DomainNameConfiguration
DomainNameConfiguration2

And in /etc/nginx/conf.d/default.conf

server {
    listen       80;
    server_name ggc.world;
    location / {
    #    root   /usr/share/nginx/html;
    #    index  index.html index.htm;
        #proxy_pass http://192.168.1.7:8080/;
        proxy_pass http://ggc.world:8080;
    }
}

server {
    listen       80;
    server_name www.ggc.world;
    location / {
        #proxy_pass http://192.168.1.7:8080/;
        proxy_pass http://www.ggc.world:8080;
    }
}

Where I put a wrong webroot?

JuergenAuer · April 13, 2019, 7:35am

Looks like you have no webroot. Where is

/home/marco/go/src/MyPage/

defined?

I'm not so firm with such nginx proxy configurations.

But checking this

you can have a lot of location definitions with own proxy definitions.

So create an own location with /.well-known/acme-challenge and use that.

marcoippolito · April 15, 2019, 8:36am

@JuergenAuer

marco@pc01:/etc/nginx/conf.d$ sudo ufw status
Status: inactive

After disabling the Kasperky Firewall of the Windows laptop I’m using to connect via ssh to the Ubuntu PC hosting the web server and the web page, I modified the /etc/nginx/conf.d/default.conf as follows:

server {
    listen       80;
    server_name ggc.world www.ggc.world;

    location / {
        proxy_pass http://ggc.world:8080/home/marco/go/src/MyPage/.well-known/acme-
        challenge;
    }
}
server {
    listen       80;
    server_name www.ggc.world;

    location / {
        proxy_pass http://www.ggc.world:8080/home/marco/go/src/MyPage/.well-
       known/acme-challenge;
    }
}


marco@pc01:~/go/src/MyPage$ sudo certbot certonly --webroot -w /home/marco/go/src
/MyPage -d ggc.world -d www.ggc.world
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for ggc.world
http-01 challenge for www.ggc.world
Using the webroot path /home/marco/go/src/MyPage for all unmatched domains.

Waiting for verification...
Cleaning up challenges
Failed authorization procedure. www.ggc.world (http-01): urn:ietf:params:acme:error:connection :: The   
server could not connect to the client to verify the domain :: Fetching http://www.ggc.world/.well-  
known/acme-challenge/WYmLKi8OczjZFsWPklkxR52FUFKsn830-6B0W54-IVo: Timeout during 
connect (likely firewall problem)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: www.ggc.world
   Type:   connection
   Detail: Fetching
   http://www.ggc.world/.well-known/acme-challenge
  /WYmLKi8OczjZFsWPklkxR52FUFKsn830-6B0W54-IVo:
  Timeout during connect (likely firewall problem)

  To fix these errors, please make sure that your domain name was
  entered correctly and the DNS A/AAAA record(s) for that domain
  contain(s) the right IP address. Additionally, please check that
  your computer has a publicly routable IP address and that no
  firewalls are preventing the server from communicating with the
  client. If you're using the webroot plugin, you should also verify
 that you are serving files from the webroot path you provided.

marcoippolito · April 27, 2019, 3:09pm

@JuergenAuer in order to eliminate the windows firewall, I tried also using an ubuntu desktop laptop as client, connecting via ssh to the Ubuntu Server :

marco@pc01:~/go/src/MyPage$ sudo certbot certonly --webroot -w /home/marco/go/src
/MyPage/ -d ggc.world -d www.ggc.world
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for ggc.world
http-01 challenge for www.ggc.world
Using the webroot path /home/marco/go/src/MyPage for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. ggc.world (http-01): 
urn:ietf:params:acme:error:connection :: The server could not connect to the client to
 verify the domain :: Fetching http://ggc.world/.well-known/acme-challenge 
/fgJOh0VJIfvF8WCXX2nJiB2kOPL7zitWh-gxU_wYeMc: Timeout during connect (likely 
firewall problem), www.ggc.world (http-01): urn:ietf:params:acme:error:connection :: The 
server could not connect to the client to verify the domain :: Fetching   
http://www.ggc.world/.well-known/acme-challenge/   
PquYA5Ijly92gc1SrpYZFQFsYUjP7XxvCDD2BEWr-BY: Timeout during connect (likely 
firewall problem)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: ggc.world
   Type:   connection
   Detail: Fetching
   http://ggc.world/.well-known/acme-challenge
/fgJOh0VJIfvF8WCXX2nJiB2kOPL7zitWh-gxU_wYeMc:
   Timeout during connect (likely firewall problem)

   Domain: www.ggc.world
   Type:   connection
   Detail: Fetching
   http://www.ggc.world/.well-known/acme-challenge
/PquYA5Ijly92gc1SrpYZFQFsYUjP7XxvCDD2BEWr-BY:
   Timeout during connect (likely firewall problem)

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.

marco@pc01:~/go/src/MyPage$ sudo ufw status
Status: inactive

Looking forward to your kind help.
Marco