The request message was malformed :: Unable to update challenge :: Challenge data was corrupted

Server
1

Hello,

I am trying to serve the acme-challenge file on one server across multiple servers for SAN certificate.

I am running the letsencrypt client on my os x named computer1.
I have two unix servers, nginx is installed on them, named server1 and server2. I configured nginx to pass /.well-known requests to my client1.

location /.well-known {
    proxy_pass http://client1.dnsname;
    proxy_set_header Host "client1.dnsname";
}

I have one web application (ASP.NET 5) on windows server, I configured my application to proxy /.well-known request to client1 as well. It is offtopic but I am providing the code here.

            .Use(async(a, b) =>
            {
                if(a.Request.Path.ToUriComponent()?.StartsWith("/.well-known") == true) {
                    using(var client = new HttpClient()) {
                        var state = await client.GetAsync("http://client1.dnsname" + a.Request.Path);

                        a.Response.ContentType = state.Content.Headers.ContentType.ToString();

                        var str = await state.Content.ReadAsStringAsync();
                        await a.Response.WriteAsync(str);
                        
                        //await state.Content.CopyToAsync(a.Response.Body);
                        //await a.Response.Body.FlushAsync();
                    }
                } else { 
                    await b();
                }
            })

I verified the proxying the acme-challenge works great.

 var a = ["domain1", "domain2", "domain3"];
 a.forEach(function(host) {

  request('http://' + host + '/.well-known/acme-challenge/CODE1', function (error, response, body) {
    if(error) {
      console.log(error, host);
    } else {
      if (response.statusCode == 200) {
        if(body == "CODE2") {
          console.log(host + " OK");
        } else {
          console.log(host);
        }
      } else {
        console.log(host);
      }
    }
  })

});

Everything works well for simple case. However I get “The request message was malformed :: Unable to update challenge :: Challenge data was corrupted” error.

Log file shows I have an Incoming Request to serve acme-challenge, many of them works great.

2015-11-16 22:14:05,763:DEBUG:acme.standalone:66.133.109.36 - - Incoming request
...lots of success messages, still I don't know which are them are succeded...
2015-11-16 22:14:11,104:DEBUG:acme.standalone:66.133.109.36 - - Incoming request
2015-11-16 22:14:11,104:DEBUG:acme.standalone:66.133.109.36 - - Serving HTTP01 with token u'CODE1'
2015-11-16 22:14:11,104:DEBUG:acme.standalone:66.133.109.36 - - "GET /.well-known/acme-challenge/CODE1 HTTP/1.1" 200 -
2015-11-16 22:14:11,555:DEBUG:requests.packages.urllib3.connectionpool:"POST /acme/challenge/CODE1/323827 HTTP/1.1" 400 105
2015-11-16 22:14:11,562:DEBUG:root:Received <Response [400]>. Headers: {'Content-Length': '105', 'Expires': 'Mon, 16 Nov 2015 22:14:11 GMT', 'Server': 'nginx', 'Connection': 'close', 'Pragma': 'no-cache', 'Cache-Control': 'max-age=0, no-cache, no-store', 'Date': 'Mon, 16 Nov 2015 22:14:11 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'CODE'}. Content: '{"type":"urn:acme:error:malformed","detail":"Unable to update challenge :: Challenge data was corrupted"}'
2015-11-16 22:14:11,562:DEBUG:acme.client:Storing nonce: 'some bytes'
2015-11-16 22:14:11,562:DEBUG:acme.client:Received response <Response [400]> (headers: {'Content-Length': '105', 'Expires': 'Mon, 16 Nov 2015 22:14:11 GMT', 'Server': 'nginx', 'Connection': 'close', 'Pragma': 'no-cache', 'Cache-Control': 'max-age=0, no-cache, no-store', 'Date': 'Mon, 16 Nov 2015 22:14:11 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'oZbSLm6UwzzQkaIIzdEsziTiDd4JIAqQEkQQWvEtvrw'}): '{"type":"urn:acme:error:malformed","detail":"Unable to update challenge :: Challenge data was corrupted"}'

Now, I got Unable to update challenge :: Challenge data was corrupted. My verifier script works fine and I don’t know what causes the problem. The client is not informative about which domain has malformed challenge data.

Again, I want to print out that:

from win-server
http://some.domain.running.on.win/.well-known/acme-challenge/CODE1

from client1
http://localhost/.well-known/acme-challenge/CODE1

has both “Content-Type” header (text/plain) and has same content (CODE2). The only difference is some headers “Server: Microsoft-IIS/8.0”, and “Server: BaseHTTP/0.3 Python/2.7.10”. First one has Content-Length too second one does not. I tested them manually with curl from client1.

Also the tester script has tested all domains, /.well-known/acme-challenge/CODE1 has CODE2 as content.

Is there any point that I am missing here? Is there any way to direct point which of my domains are failed the challenge?

Thanks,

2
  1. One of the nginx servers were giving “Content-Type: text/plain; charset=utf-8” instead of “Conten-Type: text/plain”, I solved it.
  2. Both nginx were giving chunked responses. I solved it too.

But my problem still continues. No failed hostname on letsencrypt client log, which making the whole debugging procudure harder (is there any way to show it?).