We don't mind you downloading the PEM file from us in an automated fashion, but please don't do it more often than once per day. It is only updated once every few months anyway.
You might have issues downloading it without an existing root store, since it is hosted from an HTTPS site. Probably just better to bundle it with each new release.
I think this whole thread is great demonstration on why its probably better to just use libcurl (which uses the operating system's native SSL capabilities) than manually wrangling with low level technicalities.
Turns out I do already have a root certificate issued by Let’s Encrypt. The ca.cer file. I should’ve looked at it earlier. I can just use that and make sure it’s renewed as necessary, right? Should be fine.
@_az If I want to load all of the root certificates from the store into a Boost SSL context as a std::string, would this be okay to do that?
#ifndef ROOT_CERTIFICATE_H
#define ROOT_CERTIFICATE_H
#include <boost/asio/ssl.hpp>
#include "wincrypt.h"
#include <string>
namespace ssl = boost::asio::ssl; // from <boost/asio/ssl.hpp>
namespace detail
{
// The template argument is gratuituous, to
// allow the implementation to be header-only.
//
template<class = void>
void load_root_certificates(ssl::context &ctx, boost::system::error_code &ec)
{
PCCERT_CONTEXT pcert_context = NULL;
const char *pzstore_name = "ROOT";
std::string cert;
// Try to open root certificate store
// If it succeeds, it'll return a handle to the certificate store
// If it fails, it'll return NULL
HANDLE hstore_handle = CertOpenSystemStoreA(NULL, pzstore_name);
if (hstore_handle != NULL)
{
// Extract the certificates from the store in a loop
while (pcert_context = CertEnumCertificatesInStore(hstore_handle, pcert_context))
{
// Concatenate certs to std::string cert variable
cert = std::string(std::strcat(cert.c_str(), pcert_context));
}
}
ctx.add_certificate_authority(boost::asio::buffer(cert.data(), cert.size()), ec);
if (ec)
{
return;
}
}
}
// Load the certificate into an SSL context
//
// This function is inline so that its easy to take
// the address and there's nothing weird like a
// gratuituous template argument; thus it appears
// like a "normal" function.
//
inline void load_root_certificates(ssl::context &ctx, boost::system::error_code &ec)
{
detail::load_root_certificates(ctx, ec);
}
inline void load_root_certificates(ssl::context &ctx)
{
boost::system::error_code ec;
detail::load_root_certificates(ctx, ec);
if (ec)
{
throw boost::system::system_error{ ec };
}
}
#endif
Will this load all of certs into the Boost SSL context, or did I do something wrong (aside from maybe messing up by trying to concatenate to cert.c_str() which might be a nullptr)?
Edit: It doesn’t even compile. How do I load the cert contexts from the Windows root certificate store into the Boost.Asio SSL context?
@_az I did what I thought would be good for loading the root certs, but now I can’t get to my app in my browser when the web server console app is running. If I give a link to the GitHub repository here, will you please look at the C++ code and tell me what I might’ve done wrong? I’ll try asking on the Boost mailing list too. Anyway, here’s the link.
Well, right now it’s also possible that it’s just because my dynamic DNS hasn’t updated yet because I just checked my server’s IP address and it’s still an older one. I have the client software installed on my computer and its background service is running. So I don’t know what’s going on. Maybe I should complain to them too.
Though, just to say: my public IP address actually isn’t dynamic; it only changed because the router reset. But this happens a lot due to power being disconnected. This is Pakistan and its load-shedding. Even if this weren’t the case, my IP address would also change if I got a new router for whatever reason (like if I moved). So I got a dynamic DNS name just to safe in these kinds of cases.
@_az I was right about the DNS. But now there’s another problem. When I try to send a request to the currency API I’m using from my server-side code, my exception-catching code catches an std::exception whose what string just says, “uninitialized”. I have no idea where the exception came from. I printed the exception type using typeid, but it just says that it’s a struct boost::wrapexcept<class boost::system::system_error>. I tried putting this line of code std::cerr << "Exception thrown from here\n"; at the places where I know a boost::system::error_code object is used with an exception of that type being thrown, but none of those got printed. I’m confused.
I also printed out the certs I got from the store to my server app console window. I noticed there are some weird characters and even some sensible words (telling you what authority signed the following certificate (which certificate it’s referring to is a guess on my part)) in between some certificates. I wonder if this could cause problems. I also installed ca.cer more than once IIRC and I wonder if that could cause duplicate certs to appear in the certificate store and cause problems. I put the server console app’s output in this Gist. Notice the words and weird characters?
This is the function the exception is caught in:
// This function queries the currency API for a list of currencies
const json &cache_storage::query_list(const char *currencykey, const json &sentry)
{
boost::beast::error_code ec;
try
{
std::string host{ "bankersalgo.com" }, api_endpoint{ "/currencies_list" }, key{ currencykey };
std::string target = api_endpoint + '/' + key;
std::string port{ "443" };
int version = 11;
// The io_context is required for all IO
boost::asio::io_context ioc;
// The SSL context is required, and holds certificates
ssl::context ctx{ ssl::context::tlsv12_client };
// This holds the root certificate used for verification
load_root_certificates(ctx);
// Verify the remote server's certificate
ctx.set_verify_mode(ssl::verify_peer);
// These objects perform our IO
tcp::resolver resolver{ ioc };
ssl::stream<tcp::socket> stream{ ioc, ctx };
// Set SNI Hostname (many hosts need this to handshake successfully)
if (!SSL_set_tlsext_host_name(stream.native_handle(), host.c_str()))
{
boost::system::error_code ec{ static_cast<int>(::ERR_get_error()), boost::asio::error::get_ssl_category() };
throw boost::system::system_error{ ec };
std::cerr << "Exception thrown from here\n";
}
// Look up the domain name
const auto results = resolver.resolve(host, port);
// Make the connection on the IP address we get from a lookup
boost::asio::connect(stream.next_layer(), results.begin(), results.end());
// Set up an HTTP GET request message
http::request<http::string_body> req{ http::verb::get, target, version };
req.set(http::field::host, host);
req.set(http::field::content_type, "application/json");
req.set(http::field::accept, "application/json");
req.set(http::field::user_agent, BOOST_BEAST_VERSION_STRING);
req.set(http::field::content_encoding, "gzip");
// Send the HTTP request to the remote host
http::write(stream, req);
// This buffer is used for reading and must be persisted
boost::beast::flat_buffer buffer;
// Declare a container to hold the response
http::response<http::string_body> res;
// Receive the HTTP response
http::read(stream, buffer, res);
std::cout << res.body() << "\n";
// Gracefully close the stream
boost::system::error_code ec;
stream.shutdown(ec);
if (ec == boost::asio::error::eof)
{
// Rationale:
// http://stackoverflow.com/questions/25587403/boost-asio-ssl-async-shutdown-always-finishes-with-an-error
ec.assign(0, ec.category());
}
if (ec)
{
throw boost::system::system_error{ ec };
std::cerr << "Exception thrown from here\n";
}
// If we get here then the connection is closed gracefully
return json::parse(res.body());
}
catch (const boost::beast::error_code &ec)
{
std::cerr << "Line 862: Error: " << ec.message() << '\n';
}
catch (const json::exception &e)
{
std::cerr << "Line 866: Error: " << e.what() << '\n';
}
catch (const std::exception &e)
{
std::cerr << "Line 870: Error: " << e.what() << '\n';
std::cerr << "The type of the exception is " << typeid(e).name() << '\n';
}
return sentry;
}
Any replies or help is appreciated. Thanks in advance.
@_az I got the garbage stuff out of the certificates by specifying the size of the char* I needed to create to hold the certificates. But there’s still some confusion and I’m afraid I might still be doing something wrong. Here’s the updated link to the GitHub repository with the entire source code.
As you can see if you look at root_certificate.hpp, I called ssl::context::use_certificate_chain and then called ssl::context::add_certificate_authority; I’m not sure if this is the order or if it’s even okay to use them together. Hopefully you know more about Asio than me and can help me out because it doesn’t seem like I can get any help me from the Boost mailing lists at all now (I guess I asked too many questions, some of them being stupid and/or premature).
After that, there’s still the exception I caught in the client code where I’m trying to send the request to the currency API at bankersalgo.com. When I navigate to the app in my browser while the server app is running, it crashes after printing “Line 870: Error: uninitialized”, with “uninitialized” being the what string of an exception object, and exiting with some negative number. I checked its type and it’s a boost::beast::system::system_error wrapped into a std::exception. It’s a struct holding that wrapping (I could be remembering wrong, though). But for the life of me I really can’t tell where it came from. I tried putting std::cout calls in root_certificate.hpp where a boost::system::error_code object is used and also in the function where I’m trying to make the encrypted GET request to the currency API, but none of those were hit. Or it could be that I did wrong by putting the call to std::cout after exception was thrown when I should’ve put it before, although that still doesn’t take into account the ones in the header file because no exception is being explicitly thrown there.
Anyway, any ideas on what I could do to fix the problem?
This is getting too far into homework help for me, sorry. These problems are way past the scope of Let’s Encrypt.
To improve your chances in general, you might include some build instructions use cmake or smething, because I got a bunch of template errors trying to build with gcc -std=c++17.
I’m using Visual Studio 2017 with C++17 enabled. You might need some preprocessor definitions set in Project Properties (there are some required by Boost and one required by Jinja2Cpp). On my system, it also compiles under the LLVM toolchain on Visual Studio 2017.
Even if you can’t help with the error, the root certificate stuff I asked about should still be within the scope of this thread at least, right?
@_az I was able to pinpoint that the exception is thrown from this function call:
// Send the HTTP request to the remote host
http::write(stream, req);
I tried to look at the documentation for the function here, but I still don’t get why it fails.
But I can’t shake the suspicion that it may have something to do with something being wrong in root_certificate.hpp, so I wonder if you could help me verify that.
Here’s the code in the header:
#ifndef ROOT_CERTIFICATE_H
#define ROOT_CERTIFICATE_H
#include <boost/asio/ssl.hpp>
#include <wincrypt.h>
#include <iostream>
#include <string>
namespace ssl = boost::asio::ssl; // from <boost/asio/ssl.hpp>
namespace detail
{
// The template argument is gratuituous, to
// allow the implementation to be header-only.
//
template<class = void>
void load_root_certificates(ssl::context &ctx, boost::system::error_code &ec)
{
PCCERT_CONTEXT pcert_context = nullptr;
const char *pzstore_name = "ROOT";
// Try to open root certificate store
// If it succeeds, it'll return a handle to the certificate store
// If it fails, it'll return NULL
auto hstore_handle = CertOpenSystemStoreA(NULL, pzstore_name);
char *data = nullptr;
std::string certificates;
X509 *x509 = nullptr;
BIO *bio = nullptr;
if (hstore_handle != nullptr)
{
// Extract the certificates from the store in a loop
while ((pcert_context = CertEnumCertificatesInStore(hstore_handle, pcert_context)) != NULL)
{
x509 = d2i_X509(nullptr, const_cast<const BYTE**>(&pcert_context->pbCertEncoded), pcert_context->cbCertEncoded);
bio = BIO_new(BIO_s_mem());
if (PEM_write_bio_X509(bio, x509))
{
auto len = BIO_get_mem_data(bio, &data);
if (certificates.size() == 0)
{
certificates = { data, static_cast<std::size_t>(len) };
ctx.add_certificate_authority(boost::asio::buffer(certificates.data(), certificates.size()), ec);
if (ec)
{
BIO_free(bio);
X509_free(x509);
CertCloseStore(hstore_handle, 0);
return;
}
}
else
{
certificates.append(data, static_cast<std::size_t>(len));
ctx.add_certificate_authority(boost::asio::buffer(certificates.data(), certificates.size()), ec);
if (ec)
{
BIO_free(bio);
X509_free(x509);
CertCloseStore(hstore_handle, 0);
return;
}
}
}
BIO_free(bio);
X509_free(x509);
}
CertCloseStore(hstore_handle, 0);
}
const std::string certs{ certificates };
}
}
// Load the certificate into an SSL context
//
// This function is inline so that its easy to take
// the address and there's nothing weird like a
// gratuituous template argument; thus it appears
// like a "normal" function.
//
inline void load_root_certificates(ssl::context &ctx, boost::system::error_code &ec)
{
detail::load_root_certificates(ctx, ec);
}
inline void load_root_certificates(ssl::context &ctx)
{
boost::system::error_code ec;
detail::load_root_certificates(ctx, ec);
if (ec)
{
throw boost::system::system_error{ ec };
}
}
#endif
I said earlier, I don't have Windows. These aren't helpful instructions, you're putting the burden of figuring out your build environment on me.
I extracted the parts that relate to Let's Encrypt into a standalone client/server example here, using your certificate and private key and mostly your code.
And do I at least have what I should do in my root_certificate.hpp file correct?
I guess you can’t help me with finding out why the call to boost::beast::http::write fails, then? Because even though I found out that that’s what’s throwing the exception with the what string “uninitialized”, I still can’t figure out why it’s happening. All I know is that that method throws a system_error exception when it fails.
I appreciate that you need some help here, but I agree with @_az that this is beyond the scope of problems we solve on this forum. I suggest you find a general-purpose C++ help group to answer the remainder of your questions.
Please do start a new thread once you get these issues solved if you find that you still have Let’s Encrypt-specific questions.