Understanding server name resolving vs Host headers in HTTPS

Help
1

I'm trying to understand why does curl resolving vs Host header works for HTTPS websites.

My problem is the following:

For HTTP, these two are exactly the same:

curl -I http://direct.openfreemap.org/styles/liberty

and

curl -H "Host: direct.openfreemap.org" -I http://144.76.168.195/styles/liberty

however, for HTTPS, there is no way to make it work.

curl -H "Host: direct.openfreemap.org" -I https://144.76.168.195/styles/liberty

curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

The only way to make it work with HTTPS is to use the special, low-level "resolve" option with curl:

curl --resolve direct.openfreemap.org:443:144.76.168.195 -I https://direct.openfreemap.org/styles/liberty

My problem is that while it works with curl, I cannot do the same in a JS environment, for example in a Cloudflare worker. I'm trying to use Cloudflare workers to health-check individual servers behind a Round-Robin DNS record, and this would be the only way.

Can you tell me how does HTTPS work, or how can I possibly fix this? The "Host" header is definitely not enough.

2

Not for me. Curl works perfectly with HTTPS:

osiris@erazer ~ $ curl -I https://direct.openfreemap.org/styles/liberty
HTTP/2 200 
server: nginx
date: Sun, 09 Jun 2024 10:41:37 GMT
content-type: application/json
content-length: 42425
last-modified: Fri, 31 Dec 1999 23:00:00 GMT
etag: "386d3570-a5b9"
expires: Mon, 10 Jun 2024 10:41:37 GMT
cache-control: max-age=86400
access-control-allow-origin: *
cache-control: public

osiris@erazer ~ $

I also don't get any "self signed certificate" error, there seems to be a Let's Encrypt certificate configured.

Can you perhaps say more about what you're trying to do and why you need all those "Host" headers to begin with?

1 Like
3

This has nothing to do with this forum.
It is expected:

image
image891×101 6.61 KB

The -H header is NOT for HTTPS.

Why are you using an IP in the URL instead of the name on the cert?
If the name doesn't resolve to that IP, then you can override it locally in the /etc/hosts file.

3 Likes
4

headers, host or whatever it will be is http layer thing, and TLS layer ignores it: if you call curl by IP address it will send request with any server name indication, so server have to return default certificate that's likely not valid for your expected name

3 Likes
5

he said he wants to connect to specific backend server from few round robin servers in DNS

4 Likes
7

Then instead of -H, he will have to do something else.
I proposed overriding DNS.
[there are other solutions]

A simpler solution is to provide [additional] unique names and certs to each backend.

None-the-less, the topic text and first post make the request clear:
"I need help understanding cURL with HTTPS."
This is not a forum for that.

4 Likes
8

Maybe it's as simple as a SNI issue. As far as I can tell from man curl, the -H Host ... option does not change anything with relation to the SNI value.

Thus a curl command with the https://hostname/ is required to set hostname as the SNI value and if OP then needs to do some fancy resolving using --resolve due to their internal shenanigans, well, that's just that :man_shrugging:t2:

Maybe OP needs to Google/DuckDuckGo/ChatGPT with keywords sni and javascript to fix their JS environment, I dunno..

Maybe it's as simple as using and setting tlsSocket.servername in their Node.js (assuming they're using Node.js) script :man_shrugging:t2: Or maybe set the hostname property/context with server.addContext(), I dunno, I don't "speak" Node.js.. There are 17 mentions of sni on TLS (SSL) | Node.js v22.2.0 Documentation, so probably enough to read about.

2 Likes
9

I don't have any new insights into how to setup SNI with js.

But, do you have to use HTTPS for the health check? Wouldn't an HTTP endpoint be sufficient to know if the backend server is responding?

FWIW, openssl can set SNI separately as well:

openssl s_client -connect 144.76.168.195:443 --servername direct.openfreemap.org
3 Likes
10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.