Certbot Auto-Renew Crashing Ubunutu Server?

blancast · July 17, 2018, 2:58pm

I’ve been seeing consistent outages every single day, twice a day that coincide with Certbot’s autorenewal process. I’m having trouble finding anything wrong in the letsencrypt log, but I have an example below of an instance where the bot stopped apache, and failed to restart it:

b'{\n  "PJS_q1jKMxk": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",\n  "key-change": "https://acme-v01.api.letsencrypt.org/acme/key-change",\n  "meta": {\n    "caaIdentities": [\n      "letsencrypt.org"\n    ],\n    "terms-of-service": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",\n    "website": "https://letsencrypt.org"\n  },\n  "new-authz": "https://acme-v01.api.letsencrypt.org/acme/new-authz",\n  "new-cert": "https://acme-v01.api.letsencrypt.org/acme/new-cert",\n  "new-reg": "https://acme-v01.api.letsencrypt.org/acme/new-reg",\n  "revoke-cert": "https://acme-v01.api.letsencrypt.org/acme/revoke-cert"\n}'
2018-07-16 00:34:53,598:INFO:certbot.hooks:Running pre-hook command: systemctl stop apache2
2018-07-16 00:34:54,801:INFO:certbot.main:Renewing an existing certificate
2018-07-16 00:34:54,803:DEBUG:acme.client:Requesting fresh nonce
2018-07-16 00:34:54,803:DEBUG:acme.client:Sending HEAD request to https://acme-v01.api.letsencrypt.org/acme/new-authz.
2018-07-16 00:34:54,900:DEBUG:requests.packages.urllib3.connectionpool:"HEAD /acme/new-authz HTTP/1.1" 405 0
2018-07-16 00:34:54,901:DEBUG:acme.client:Received response:
HTTP 405
Connection: keep-alive
Allow: POST
Replay-Nonce: t3L_ObBsOsyFun8gUCs-jgg8aX68B3nm9aas0bGM2OU
Content-Type: application/problem+json
Content-Length: 91
Pragma: no-cache
Server: nginx
Cache-Control: max-age=0, no-cache, no-store
Date: Mon, 16 Jul 2018 00:34:54 GMT
Expires: Mon, 16 Jul 2018 00:34:54 GMT

b''
2018-07-16 00:34:54,901:DEBUG:acme.client:Storing nonce: t3L_ObBsOsyFun8gUCs-jgg8aX68B3nm9aas0bGM2OU
2018-07-16 00:34:54,901:DEBUG:acme.client:JWS payload:
b'{\n  "resource": "new-authz",\n  "identifier": {\n    "type": "dns",\n    "value": "www.artisanbakeryexpo.com"\n  }\n}'
2018-07-16 00:34:54,906:DEBUG:acme.client:Sending POST request to https://acme-v01.api.letsencrypt.org/acme/new-authz:
{
  "protected": "eyJqd2siOiB7Im4iOiAic1g0dFpxcWtmTE1ibnp6bzQwNy1nTlJIQTlraXpkXzhmd2ZVcXdjWkdocGNNRVQ3VnNKTHBKXzN3UV9QbVBWUE56VExCTk53REpMYjRpcVhxcTVCMDlzc1JuN3VtYlRYRkpuZjJJWmRXV1BRblNlM24tbWl6SXZwTXVRbzRETTM1ZjZtQXpWa0UxbjZKLVpLdUhJTGFLbGNIclo3Yzd3YWhyUWlNZEVsemJOM2EtMXk5cmo5Mkl3OHFFS0Znek40N3JscWZRQ0Vwa1ZNZUNseHFWRnpwbFEySGVlemhMbFJNR3V4ZUlwVHFPbnFJQXdfYU94NHltZVFEZFRUbVo0aFA4Vk1jWjNMQWlqa0FUT2s5TnQ2UDVHbnNqdHZlNXFxMkg4UkNNdVBDZXk2MDZyYjRYWTdaN3N3UFkxY2o5VTB3SEcwWXBjMUpjOXZROUFHNWdmRDl3IiwgImt0eSI6ICJSU0EiLCAiZSI6ICJBUUFCIn0sICJhbGciOiAiUlMyNTYiLCAibm9uY2UiOiAidDNMX09iQnNPc3lGdW44Z1VDcy1qZ2c4YVg2OEIzbm05YWFzMGJHTTJPVSJ9",
  "payload": "ewogICJyZXNvdXJjZSI6ICJuZXctYXV0aHoiLAogICJpZGVudGlmaWVyIjogewogICAgInR5cGUiOiAiZG5zIiwKICAgICJ2YWx1ZSI6ICJ3d3cuYXJ0aXNhbmJha2VyeWV4cG8uY29tIgogIH0KfQ",
  "signature": "iwoS7Y1ZYE4eoY9NJpHspqW8q41alAkBQd3akPpABSzf7xWlIzIGyAnmN6nNrgaO0rZ4N1bmMWoHejrC-G0vEjoWG-stAsb9xEQqVivyiZd9r_trQ0cz1J3HrgZStfrzMin7vJkTP1dIp0zVqVZ0DsicXcSHRAAo84NOKMrl5cl2tHcphOIb680q4LVzP3H5TspmzgxQLOF4Ed-mHwZxiuM6fACWrkZt3FUfWZDP2ojvemGet20DqwHxkiWGZdHRshSNkfJfwFNmk-T1AuG4kZ9fBDmutfH-YqAB7GIxCMdGKthhL1VZ3OFS1QPE87GAK5bHsp5-k6-WFVRvnEfvCA"
}
2018-07-16 00:34:55,020:DEBUG:requests.packages.urllib3.connectionpool:"POST /acme/new-authz HTTP/1.1" 201 1277
2018-07-16 00:34:55,021:DEBUG:acme.client:Received response:
HTTP 201
Strict-Transport-Security: max-age=604800
Content-Length: 1277
Server: nginx
Location: https://acme-v01.api.letsencrypt.org/acme/authz/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM
X-Frame-Options: DENY
Link: <https://acme-v01.api.letsencrypt.org/acme/new-cert>;rel="next"
Connection: keep-alive
Replay-Nonce: GuxpKgiC-WPr90ZhIAQFgd2osf31cBz3itFHMfMoRtI
Content-Type: application/json
Pragma: no-cache
Boulder-Requester: 31009878
Cache-Control: max-age=0, no-cache, no-store
Date: Mon, 16 Jul 2018 00:34:55 GMT
Expires: Mon, 16 Jul 2018 00:34:55 GMT

b'{\n  "identifier": {\n    "type": "dns",\n    "value": "www.artisanbakeryexpo.com"\n  },\n  "status": "pending",\n  "expires": "2018-07-23T00:34:54Z",\n  "challenges": [\n    {\n      "type": "tls-alpn-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719372",\n      "token": "wG_r4SB4r6-MBPGk1Jj_t5_P_rYx5rQDpn7IgYGcdVs"\n    },\n    {\n      "type": "tls-sni-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719373",\n      "token": "N8afy768VV8eWGADSA0sMHFOrC1HPv3_9eQCX56KUSw"\n    },\n    {\n      "type": "dns-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719374",\n      "token": "BG4GgKa4LEdaK6PalhdcgyEFhRWgNE3-yLTJ4nDrB_4"\n    },\n    {\n      "type": "http-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719375",\n      "token": "P9YF1Smd8p-Em8giONqQzCo_o1duBBS3lauaxjX_g0w"\n    }\n  ],\n  "combinations": [\n    [\n      1\n    ],\n    [\n      2\n    ],\n    [\n      3\n    ],\n    [\n      0\n    ]\n  ]\n}'
2018-07-16 00:34:55,021:DEBUG:acme.client:Storing nonce: GuxpKgiC-WPr90ZhIAQFgd2osf31cBz3itFHMfMoRtI
2018-07-16 00:34:55,021:DEBUG:acme.challenges:tls-alpn-01 was not recognized, full message: {'token': 'wG_r4SB4r6-MBPGk1Jj_t5_P_rYx5rQDpn7IgYGcdVs', 'type': 'tls-alpn-01', 'uri': 'https://acme-v01.api.letsencrypt.org/acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719372', 'status': 'pending'}
2018-07-16 00:34:55,022:DEBUG:acme.client:JWS payload:
b'{\n  "resource": "new-authz",\n  "identifier": {\n    "type": "dns",\n    "value": "artisanbakeryexpo.com"\n  }\n}'
2018-07-16 00:34:55,025:DEBUG:acme.client:Sending POST request to https://acme-v01.api.letsencrypt.org/acme/new-authz:
{
  "protected": "eyJqd2siOiB7Im4iOiAic1g0dFpxcWtmTE1ibnp6bzQwNy1nTlJIQTlraXpkXzhmd2ZVcXdjWkdocGNNRVQ3VnNKTHBKXzN3UV9QbVBWUE56VExCTk53REpMYjRpcVhxcTVCMDlzc1JuN3VtYlRYRkpuZjJJWmRXV1BRblNlM24tbWl6SXZwTXVRbzRETTM1ZjZtQXpWa0UxbjZKLVpLdUhJTGFLbGNIclo3Yzd3YWhyUWlNZEVsemJOM2EtMXk5cmo5Mkl3OHFFS0Znek40N3JscWZRQ0Vwa1ZNZUNseHFWRnpwbFEySGVlemhMbFJNR3V4ZUlwVHFPbnFJQXdfYU94NHltZVFEZFRUbVo0aFA4Vk1jWjNMQWlqa0FUT2s5TnQ2UDVHbnNqdHZlNXFxMkg4UkNNdVBDZXk2MDZyYjRYWTdaN3N3UFkxY2o5VTB3SEcwWXBjMUpjOXZROUFHNWdmRDl3IiwgImt0eSI6ICJSU0EiLCAiZSI6ICJBUUFCIn0sICJhbGciOiAiUlMyNTYiLCAibm9uY2UiOiAiR3V4cEtnaUMtV1ByOTBaaElBUUZnZDJvc2YzMWNCejNpdEZITWZNb1J0SSJ9",
  "payload": "ewogICJyZXNvdXJjZSI6ICJuZXctYXV0aHoiLAogICJpZGVudGlmaWVyIjogewogICAgInR5cGUiOiAiZG5zIiwKICAgICJ2YWx1ZSI6ICJhcnRpc2FuYmFrZXJ5ZXhwby5jb20iCiAgfQp9",
  "signature": "Lo0Nsr3CmqNnUglvNPcEzHeVUyg_achXmscNsikLxS6aBBsu2lLZBHBhoc_ETeescArEHB-igK2KkVbIdo-iLgyTtZfPyqrE52kcL4W3yTR4cAVkAUbD1A9qU3efEoTEHnbsKSFFceVovScwmQX-6cPY3fn0fmaXX0vsjUO5fpHf0d-CR-1qkGHlslLHRYAxVVYXKNJD0TQNRZfvZ_9AB6fEm-I_76v1qFPoGTBvyEDmuNEO8KInV1x7VRYdI8v0nnuFw1fJJwa5xs1oPBX_RNLoj_nZTnt6-DDYmTN-Qh1Ac8E8Uc4Br-easWnqb_lcfYzvclt4lFKQjgmYsylXuA"
}
2018-07-16 00:34:55,139:DEBUG:requests.packages.urllib3.connectionpool:"POST /acme/new-authz HTTP/1.1" 201 1273
2018-07-16 00:34:55,140:DEBUG:acme.client:Received response:
HTTP 201
Strict-Transport-Security: max-age=604800
Content-Length: 1273
Server: nginx
Location: https://acme-v01.api.letsencrypt.org/acme/authz/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA
X-Frame-Options: DENY
Link: <https://acme-v01.api.letsencrypt.org/acme/new-cert>;rel="next"
Connection: keep-alive
Replay-Nonce: f7e0Yl3Fs8OX232D63sXoR6LYbynUiAlyiWZx0AAlV8
Content-Type: application/json
Pragma: no-cache
Boulder-Requester: 31009878
Cache-Control: max-age=0, no-cache, no-store
Date: Mon, 16 Jul 2018 00:34:55 GMT
Expires: Mon, 16 Jul 2018 00:34:55 GMT

b'{\n  "identifier": {\n    "type": "dns",\n    "value": "artisanbakeryexpo.com"\n  },\n  "status": "pending",\n  "expires": "2018-07-23T00:34:55Z",\n  "challenges": [\n    {\n      "type": "tls-alpn-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719402",\n      "token": "Sv7BlGqGlCfTpRQYMMIxtl_K4RCiJKWThF2mS2lfao8"\n    },\n    {\n      "type": "http-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719404",\n      "token": "JGV941ANPbXKR09YdJQtPitu-4bkN1PhyReXfiOVKRU"\n    },\n    {\n      "type": "dns-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719406",\n      "token": "rXIPrMrmxKrxBrGd9WF3-LiiCR3oVdBGwCOyIaL_z0Y"\n    },\n    {\n      "type": "tls-sni-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719408",\n      "token": "ZPokLe274xd0hGPbw1qf80q6lPNoTbdWsT_Qqy8ug0k"\n    }\n  ],\n  "combinations": [\n    [\n      2\n    ],\n    [\n      0\n    ],\n    [\n      1\n    ],\n    [\n      3\n    ]\n  ]\n}'
2018-07-16 00:34:55,141:DEBUG:acme.client:Storing nonce: f7e0Yl3Fs8OX232D63sXoR6LYbynUiAlyiWZx0AAlV8
2018-07-16 00:34:55,141:DEBUG:acme.challenges:tls-alpn-01 was not recognized, full message: {'token': 'Sv7BlGqGlCfTpRQYMMIxtl_K4RCiJKWThF2mS2lfao8', 'type': 'tls-alpn-01', 'uri': 'https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719402', 'status': 'pending'}
2018-07-16 00:34:55,142:INFO:certbot.auth_handler:Performing the following challenges:
2018-07-16 00:34:55,142:INFO:certbot.auth_handler:tls-sni-01 challenge for www.artisanbakeryexpo.com
2018-07-16 00:34:55,142:INFO:certbot.auth_handler:tls-sni-01 challenge for artisanbakeryexpo.com
2018-07-16 00:34:55,143:DEBUG:acme.standalone:Failed to bind to :443 using IPv4
2018-07-16 00:34:55,154:INFO:certbot.auth_handler:Waiting for verification...
2018-07-16 00:34:55,154:DEBUG:acme.client:JWS payload:
b'{\n  "resource": "challenge",\n  "type": "tls-sni-01",\n  "keyAuthorization": "N8afy768VV8eWGADSA0sMHFOrC1HPv3_9eQCX56KUSw.ncyRLUkqvLcjposr0QGtoax39E9Z7udPWxPBpQVHF2Y"\n}'
2018-07-16 00:34:55,157:DEBUG:acme.client:Sending POST request to https://acme-v01.api.letsencrypt.org/acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719373:
{
  "protected": "eyJqd2siOiB7Im4iOiAic1g0dFpxcWtmTE1ibnp6bzQwNy1nTlJIQTlraXpkXzhmd2ZVcXdjWkdocGNNRVQ3VnNKTHBKXzN3UV9QbVBWUE56VExCTk53REpMYjRpcVhxcTVCMDlzc1JuN3VtYlRYRkpuZjJJWmRXV1BRblNlM24tbWl6SXZwTXVRbzRETTM1ZjZtQXpWa0UxbjZKLVpLdUhJTGFLbGNIclo3Yzd3YWhyUWlNZEVsemJOM2EtMXk5cmo5Mkl3OHFFS0Znek40N3JscWZRQ0Vwa1ZNZUNseHFWRnpwbFEySGVlemhMbFJNR3V4ZUlwVHFPbnFJQXdfYU94NHltZVFEZFRUbVo0aFA4Vk1jWjNMQWlqa0FUT2s5TnQ2UDVHbnNqdHZlNXFxMkg4UkNNdVBDZXk2MDZyYjRYWTdaN3N3UFkxY2o5VTB3SEcwWXBjMUpjOXZROUFHNWdmRDl3IiwgImt0eSI6ICJSU0EiLCAiZSI6ICJBUUFCIn0sICJhbGciOiAiUlMyNTYiLCAibm9uY2UiOiAiZjdlMFlsM0ZzOE9YMjMyRDYzc1hvUjZMWWJ5blVpQWx5aVdaeDBBQWxWOCJ9",
  "payload": "ewogICJyZXNvdXJjZSI6ICJjaGFsbGVuZ2UiLAogICJ0eXBlIjogInRscy1zbmktMDEiLAogICJrZXlBdXRob3JpemF0aW9uIjogIk44YWZ5NzY4VlY4ZVdHQURTQTBzTUhGT3JDMUhQdjNfOWVRQ1g1NktVU3cubmN5UkxVa3F2TGNqcG9zcjBRR3RvYXgzOUU5Wjd1ZFBXeFBCcFFWSEYyWSIKfQ",
  "signature": "dHcNOgqMQefd-AgDHcCZASJV4GCyQ-pSf2r3mAKOffjwbJF5RalSQCJQcVfiTIkbtaSDbQgxnbER4LhvYV_tG9wQBq8uKdMvj5XGYqV7vYywznNY5VYYcuOJ1wRKhsgysyZ1nPA92DU2FXCjgXz_mDwMPr0dYTmJLa9C0TN5nxnTGzijv1hmLvCTSH7Z_wtf42yfkoNX9eTtCTDOWynk3DwD8ZNUQQJGErUOSo7pMsnkmLg7o6mV7BTrpRl646gs15xiu6ADkGtMHEZTf8Cu_sfVy5pQmyEjMvgcnXEfxVB517SOGcawSN_TMdiQzy5y8pnDiS1KFM74PhDevjnNXA"
}
2018-07-16 00:34:55,269:DEBUG:requests.packages.urllib3.connectionpool:"POST /acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719373 HTTP/1.1" 202 339
2018-07-16 00:34:55,270:DEBUG:acme.client:Received response:
HTTP 202
Content-Length: 339
Server: nginx
Location: https://acme-v01.api.letsencrypt.org/acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719373
Link: <https://acme-v01.api.letsencrypt.org/acme/authz/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM>;rel="up"
Connection: keep-alive
Replay-Nonce: tnk52xd6vmiNwpKQ700CZvLQg6-Lvmv1JImEZDbkmkU
Content-Type: application/json
Pragma: no-cache
Boulder-Requester: 31009878
Cache-Control: max-age=0, no-cache, no-store
Date: Mon, 16 Jul 2018 00:34:55 GMT
Expires: Mon, 16 Jul 2018 00:34:55 GMT

b'{\n  "type": "tls-sni-01",\n  "status": "pending",\n  "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719373",\n  "token": "N8afy768VV8eWGADSA0sMHFOrC1HPv3_9eQCX56KUSw",\n  "keyAuthorization": "N8afy768VV8eWGADSA0sMHFOrC1HPv3_9eQCX56KUSw.ncyRLUkqvLcjposr0QGtoax39E9Z7udPWxPBpQVHF2Y"\n}'
2018-07-16 00:34:55,270:DEBUG:acme.client:Storing nonce: tnk52xd6vmiNwpKQ700CZvLQg6-Lvmv1JImEZDbkmkU
2018-07-16 00:34:55,271:DEBUG:acme.client:JWS payload:
b'{\n  "resource": "challenge",\n  "type": "tls-sni-01",\n  "keyAuthorization": "ZPokLe274xd0hGPbw1qf80q6lPNoTbdWsT_Qqy8ug0k.ncyRLUkqvLcjposr0QGtoax39E9Z7udPWxPBpQVHF2Y"\n}'
2018-07-16 00:34:55,274:DEBUG:acme.client:Sending POST request to https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719408:
{
  "protected": "eyJqd2siOiB7Im4iOiAic1g0dFpxcWtmTE1ibnp6bzQwNy1nTlJIQTlraXpkXzhmd2ZVcXdjWkdocGNNRVQ3VnNKTHBKXzN3UV9QbVBWUE56VExCTk53REpMYjRpcVhxcTVCMDlzc1JuN3VtYlRYRkpuZjJJWmRXV1BRblNlM24tbWl6SXZwTXVRbzRETTM1ZjZtQXpWa0UxbjZKLVpLdUhJTGFLbGNIclo3Yzd3YWhyUWlNZEVsemJOM2EtMXk5cmo5Mkl3OHFFS0Znek40N3JscWZRQ0Vwa1ZNZUNseHFWRnpwbFEySGVlemhMbFJNR3V4ZUlwVHFPbnFJQXdfYU94NHltZVFEZFRUbVo0aFA4Vk1jWjNMQWlqa0FUT2s5TnQ2UDVHbnNqdHZlNXFxMkg4UkNNdVBDZXk2MDZyYjRYWTdaN3N3UFkxY2o5VTB3SEcwWXBjMUpjOXZROUFHNWdmRDl3IiwgImt0eSI6ICJSU0EiLCAiZSI6ICJBUUFCIn0sICJhbGciOiAiUlMyNTYiLCAibm9uY2UiOiAidG5rNTJ4ZDZ2bWlOd3BLUTcwMENadkxRZzYtTHZtdjFKSW1FWkRia21rVSJ9",
  "payload": "ewogICJyZXNvdXJjZSI6ICJjaGFsbGVuZ2UiLAogICJ0eXBlIjogInRscy1zbmktMDEiLAogICJrZXlBdXRob3JpemF0aW9uIjogIlpQb2tMZTI3NHhkMGhHUGJ3MXFmODBxNmxQTm9UYmRXc1RfUXF5OHVnMGsubmN5UkxVa3F2TGNqcG9zcjBRR3RvYXgzOUU5Wjd1ZFBXeFBCcFFWSEYyWSIKfQ",
  "signature": "J873yIhCQZ4TGlwB-vYHm-z5BEhTlPA3Fh1VbFCQZNGfaX64B-jypIvyVdH7BzX-1alrVP8WyFYbJTBzlS-gbE0UeajXXLVa-Qfa23wpKeEmdfocwoSlRLs11ebkLbHfaABXxWc2XbqAUIvWmnVALE739sQkve4dd2QuQa87czQoaPxAW7nrMcc3YSEUzAuWpa8SnnbhVLlOEdaUS3Nh6vvmUt7Ol1cBXDbNjXib1Jqz370GfxiXhEZAP2i5pX7a7De8T5ejClPh9BZ9cBHW0HJdtmPtJlIodIrS46v2pQalZmUaklEyx4C-MxGyLkvLlcMw0rzEYFqtFC5YJFyxBQ"
}
2018-07-16 00:34:55,385:DEBUG:requests.packages.urllib3.connectionpool:"POST /acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719408 HTTP/1.1" 202 339
2018-07-16 00:34:55,386:DEBUG:acme.client:Received response:
HTTP 202
Content-Length: 339
Server: nginx
Location: https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719408
Link: <https://acme-v01.api.letsencrypt.org/acme/authz/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA>;rel="up"
Connection: keep-alive
Replay-Nonce: qQf2eR2v9RueomvQXeJ3eRC_dHtJ8r9XmwgDuIyW-K4
Content-Type: application/json
Pragma: no-cache
Boulder-Requester: 31009878
Cache-Control: max-age=0, no-cache, no-store
Date: Mon, 16 Jul 2018 00:34:55 GMT
Expires: Mon, 16 Jul 2018 00:34:55 GMT

b'{\n  "type": "tls-sni-01",\n  "status": "pending",\n  "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719408",\n  "token": "ZPokLe274xd0hGPbw1qf80q6lPNoTbdWsT_Qqy8ug0k",\n  "keyAuthorization": "ZPokLe274xd0hGPbw1qf80q6lPNoTbdWsT_Qqy8ug0k.ncyRLUkqvLcjposr0QGtoax39E9Z7udPWxPBpQVHF2Y"\n}'
2018-07-16 00:34:55,386:DEBUG:acme.client:Storing nonce: qQf2eR2v9RueomvQXeJ3eRC_dHtJ8r9XmwgDuIyW-K4
2018-07-16 00:34:55,450:DEBUG:acme.crypto_util:Performing handshake with ('::ffff:66.133.109.36', 59702, 0, 0)
2018-07-16 00:34:55,556:DEBUG:acme.standalone:::ffff:66.133.109.36 - - Incoming request
2018-07-16 00:34:55,557:DEBUG:acme.crypto_util:Performing handshake with ('::ffff:106.38.241.172', 62809, 0, 0)
2018-07-16 00:34:58,389:DEBUG:acme.client:Sending GET request to https://acme-v01.api.letsencrypt.org/acme/authz/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA.
2018-07-16 00:34:58,489:DEBUG:requests.packages.urllib3.connectionpool:"GET /acme/authz/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA HTTP/1.1" 200 1390
2018-07-16 00:34:58,489:DEBUG:acme.client:Received response:
HTTP 200
Content-Length: 1390
Server: nginx
X-Frame-Options: DENY
Link: <https://acme-v01.api.letsencrypt.org/acme/new-cert>;rel="next"
Connection: keep-alive
Replay-Nonce: MU_Gtv61rZRSSZa3RUacTWiAuiWKy1MXyz2aaIurQNY
Content-Type: application/json
Date: Mon, 16 Jul 2018 00:34:58 GMT
Pragma: no-cache
Cache-Control: max-age=0, no-cache, no-store
Strict-Transport-Security: max-age=604800
Expires: Mon, 16 Jul 2018 00:34:58 GMT

b'{\n  "identifier": {\n    "type": "dns",\n    "value": "artisanbakeryexpo.com"\n  },\n  "status": "pending",\n  "expires": "2018-07-23T00:34:55Z",\n  "challenges": [\n    {\n      "type": "tls-alpn-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719402",\n      "token": "Sv7BlGqGlCfTpRQYMMIxtl_K4RCiJKWThF2mS2lfao8"\n    },\n    {\n      "type": "http-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719404",\n      "token": "JGV941ANPbXKR09YdJQtPitu-4bkN1PhyReXfiOVKRU"\n    },\n    {\n      "type": "dns-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719406",\n      "token": "rXIPrMrmxKrxBrGd9WF3-LiiCR3oVdBGwCOyIaL_z0Y"\n    },\n    {\n      "type": "tls-sni-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719408",\n      "token": "ZPokLe274xd0hGPbw1qf80q6lPNoTbdWsT_Qqy8ug0k",\n      "keyAuthorization": "ZPokLe274xd0hGPbw1qf80q6lPNoTbdWsT_Qqy8ug0k.ncyRLUkqvLcjposr0QGtoax39E9Z7udPWxPBpQVHF2Y"\n    }\n  ],\n  "combinations": [\n    [\n      2\n    ],\n    [\n      0\n    ],\n    [\n      1\n    ],\n    [\n      3\n    ]\n  ]\n}'
2018-07-16 00:34:58,490:DEBUG:acme.challenges:tls-alpn-01 was not recognized, full message: {'token': 'Sv7BlGqGlCfTpRQYMMIxtl_K4RCiJKWThF2mS2lfao8', 'type': 'tls-alpn-01', 'uri': 'https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719402', 'status': 'pending'}
2018-07-16 00:34:58,490:DEBUG:acme.client:Sending GET request to https://acme-v01.api.letsencrypt.org/acme/authz/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM.
2018-07-16 00:34:58,590:DEBUG:requests.packages.urllib3.connectionpool:"GET /acme/authz/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM HTTP/1.1" 200 1640
2018-07-16 00:34:58,591:DEBUG:acme.client:Received response:
HTTP 200
Content-Length: 1640
Server: nginx
X-Frame-Options: DENY
Link: <https://acme-v01.api.letsencrypt.org/acme/new-cert>;rel="next"
Connection: keep-alive
Replay-Nonce: AMNxbTCiOUVra16ssu2a13_Tpj0yETaDYu1zNsqy6-Q
Content-Type: application/json
Date: Mon, 16 Jul 2018 00:34:58 GMT
Pragma: no-cache
Cache-Control: max-age=0, no-cache, no-store
Strict-Transport-Security: max-age=604800
Expires: Mon, 16 Jul 2018 00:34:58 GMT

b'{\n  "identifier": {\n    "type": "dns",\n    "value": "www.artisanbakeryexpo.com"\n  },\n  "status": "valid",\n  "expires": "2018-08-15T00:34:55Z",\n  "challenges": [\n    {\n      "type": "tls-alpn-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719372",\n      "token": "wG_r4SB4r6-MBPGk1Jj_t5_P_rYx5rQDpn7IgYGcdVs"\n    },\n    {\n      "type": "tls-sni-01",\n      "status": "valid",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719373",\n      "token": "N8afy768VV8eWGADSA0sMHFOrC1HPv3_9eQCX56KUSw",\n      "keyAuthorization": "N8afy768VV8eWGADSA0sMHFOrC1HPv3_9eQCX56KUSw.ncyRLUkqvLcjposr0QGtoax39E9Z7udPWxPBpQVHF2Y",\n      "validationRecord": [\n        {\n          "hostname": "www.artisanbakeryexpo.com",\n          "port": "443",\n          "addressesResolved": [\n            "138.197.20.176"\n          ],\n          "addressUsed": "138.197.20.176"\n        }\n      ]\n    },\n    {\n      "type": "dns-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719374",\n      "token": "BG4GgKa4LEdaK6PalhdcgyEFhRWgNE3-yLTJ4nDrB_4"\n    },\n    {\n      "type": "http-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719375",\n      "token": "P9YF1Smd8p-Em8giONqQzCo_o1duBBS3lauaxjX_g0w"\n    }\n  ],\n  "combinations": [\n    [\n      1\n    ],\n    [\n      2\n    ],\n    [\n      3\n    ],\n    [\n      0\n    ]\n  ]\n}'
2018-07-16 00:34:58,592:DEBUG:acme.challenges:tls-alpn-01 was not recognized, full message: {'token': 'wG_r4SB4r6-MBPGk1Jj_t5_P_rYx5rQDpn7IgYGcdVs', 'type': 'tls-alpn-01', 'uri': 'https://acme-v01.api.letsencrypt.org/acme/challenge/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM/5694719372', 'status': 'pending'}
2018-07-16 00:35:01,596:DEBUG:acme.client:Sending GET request to https://acme-v01.api.letsencrypt.org/acme/authz/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA.
2018-07-16 00:35:01,719:DEBUG:requests.packages.urllib3.connectionpool:"GET /acme/authz/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA HTTP/1.1" 200 1390
2018-07-16 00:35:01,719:DEBUG:acme.client:Received response:
HTTP 200
Content-Length: 1390
Server: nginx
X-Frame-Options: DENY
Link: <https://acme-v01.api.letsencrypt.org/acme/new-cert>;rel="next"
Connection: keep-alive
Replay-Nonce: liIITkWieIbcevjhhdFweFGRZX5PxKLK9GMrsqRc79w
Content-Type: application/json
Date: Mon, 16 Jul 2018 00:35:01 GMT
Pragma: no-cache
Cache-Control: max-age=0, no-cache, no-store
Strict-Transport-Security: max-age=604800
Expires: Mon, 16 Jul 2018 00:35:01 GMT

b'{\n  "identifier": {\n    "type": "dns",\n    "value": "artisanbakeryexpo.com"\n  },\n  "status": "pending",\n  "expires": "2018-07-23T00:34:55Z",\n  "challenges": [\n    {\n      "type": "tls-alpn-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719402",\n      "token": "Sv7BlGqGlCfTpRQYMMIxtl_K4RCiJKWThF2mS2lfao8"\n    },\n    {\n      "type": "http-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719404",\n      "token": "JGV941ANPbXKR09YdJQtPitu-4bkN1PhyReXfiOVKRU"\n    },\n    {\n      "type": "dns-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719406",\n      "token": "rXIPrMrmxKrxBrGd9WF3-LiiCR3oVdBGwCOyIaL_z0Y"\n    },\n    {\n      "type": "tls-sni-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719408",\n      "token": "ZPokLe274xd0hGPbw1qf80q6lPNoTbdWsT_Qqy8ug0k",\n      "keyAuthorization": "ZPokLe274xd0hGPbw1qf80q6lPNoTbdWsT_Qqy8ug0k.ncyRLUkqvLcjposr0QGtoax39E9Z7udPWxPBpQVHF2Y"\n    }\n  ],\n  "combinations": [\n    [\n      2\n    ],\n    [\n      0\n    ],\n    [\n      1\n    ],\n    [\n      3\n    ]\n  ]\n}'
2018-07-16 00:35:01,720:DEBUG:acme.challenges:tls-alpn-01 was not recognized, full message: {'token': 'Sv7BlGqGlCfTpRQYMMIxtl_K4RCiJKWThF2mS2lfao8', 'type': 'tls-alpn-01', 'uri': 'https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719402', 'status': 'pending'}
2018-07-16 00:35:04,724:DEBUG:acme.client:Sending GET request to https://acme-v01.api.letsencrypt.org/acme/authz/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA.
2018-07-16 00:35:04,826:DEBUG:requests.packages.urllib3.connectionpool:"GET /acme/authz/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA HTTP/1.1" 200 1390
2018-07-16 00:35:04,827:DEBUG:acme.client:Received response:
HTTP 200
Content-Length: 1390
Server: nginx
X-Frame-Options: DENY
Link: <https://acme-v01.api.letsencrypt.org/acme/new-cert>;rel="next"
Connection: keep-alive
Replay-Nonce: tEawX_fAr1ZkJUviDbZBOLpDKa2WGFkl0ORtpVJG_Gs
Content-Type: application/json
Date: Mon, 16 Jul 2018 00:35:04 GMT
Pragma: no-cache
Cache-Control: max-age=0, no-cache, no-store
Strict-Transport-Security: max-age=604800
Expires: Mon, 16 Jul 2018 00:35:04 GMT

b'{\n  "identifier": {\n    "type": "dns",\n    "value": "artisanbakeryexpo.com"\n  },\n  "status": "pending",\n  "expires": "2018-07-23T00:34:55Z",\n  "challenges": [\n    {\n      "type": "tls-alpn-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719402",\n      "token": "Sv7BlGqGlCfTpRQYMMIxtl_K4RCiJKWThF2mS2lfao8"\n    },\n    {\n      "type": "http-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719404",\n      "token": "JGV941ANPbXKR09YdJQtPitu-4bkN1PhyReXfiOVKRU"\n    },\n    {\n      "type": "dns-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719406",\n      "token": "rXIPrMrmxKrxBrGd9WF3-LiiCR3oVdBGwCOyIaL_z0Y"\n    },\n    {\n      "type": "tls-sni-01",\n      "status": "pending",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719408",\n      "token": "ZPokLe274xd0hGPbw1qf80q6lPNoTbdWsT_Qqy8ug0k",\n      "keyAuthorization": "ZPokLe274xd0hGPbw1qf80q6lPNoTbdWsT_Qqy8ug0k.ncyRLUkqvLcjposr0QGtoax39E9Z7udPWxPBpQVHF2Y"\n    }\n  ],\n  "combinations": [\n    [\n      2\n    ],\n    [\n      0\n    ],\n    [\n      1\n    ],\n    [\n      3\n    ]\n  ]\n}'
2018-07-16 00:35:04,827:DEBUG:acme.challenges:tls-alpn-01 was not recognized, full message: {'token': 'Sv7BlGqGlCfTpRQYMMIxtl_K4RCiJKWThF2mS2lfao8', 'type': 'tls-alpn-01', 'uri': 'https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719402', 'status': 'pending'}
2018-07-16 00:35:07,831:DEBUG:acme.client:Sending GET request to https://acme-v01.api.letsencrypt.org/acme/authz/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA.
2018-07-16 00:35:07,940:DEBUG:requests.packages.urllib3.connectionpool:"GET /acme/authz/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA HTTP/1.1" 200 1812
2018-07-16 00:35:07,941:DEBUG:acme.client:Received response:
HTTP 200
Content-Length: 1812
Server: nginx
X-Frame-Options: DENY
Link: <https://acme-v01.api.letsencrypt.org/acme/new-cert>;rel="next"
Connection: keep-alive
Replay-Nonce: MJKFvCTM3PDVOfY9tF31tKvJQ4yUWeCxRBNCC-qNK98
Content-Type: application/json
Date: Mon, 16 Jul 2018 00:35:07 GMT
Pragma: no-cache
Cache-Control: max-age=0, no-cache, no-store
Strict-Transport-Security: max-age=604800
Expires: Mon, 16 Jul 2018 00:35:07 GMT

b'{\n  "identifier": {\n    "type": "dns",\n    "value": "artisanbakeryexpo.com"\n  },\n  "status": "invalid",\n  "expires": "2018-07-23T00:34:55Z",\n  "challenges": [\n    {\n      "type": "tls-alpn-01",\n      "status": "invalid",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719402",\n      "token": "Sv7BlGqGlCfTpRQYMMIxtl_K4RCiJKWThF2mS2lfao8"\n    },\n    {\n      "type": "http-01",\n      "status": "invalid",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719404",\n      "token": "JGV941ANPbXKR09YdJQtPitu-4bkN1PhyReXfiOVKRU"\n    },\n    {\n      "type": "dns-01",\n      "status": "invalid",\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719406",\n      "token": "rXIPrMrmxKrxBrGd9WF3-LiiCR3oVdBGwCOyIaL_z0Y"\n    },\n    {\n      "type": "tls-sni-01",\n      "status": "invalid",\n      "error": {\n        "type": "urn:acme:error:connection",\n        "detail": "Timeout after connect (your server may be slow or overloaded)",\n        "status": 400\n      },\n      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719408",\n      "token": "ZPokLe274xd0hGPbw1qf80q6lPNoTbdWsT_Qqy8ug0k",\n      "keyAuthorization": "ZPokLe274xd0hGPbw1qf80q6lPNoTbdWsT_Qqy8ug0k.ncyRLUkqvLcjposr0QGtoax39E9Z7udPWxPBpQVHF2Y",\n      "validationRecord": [\n        {\n          "hostname": "artisanbakeryexpo.com",\n          "port": "443",\n          "addressesResolved": [\n            "138.197.20.176"\n          ],\n          "addressUsed": "138.197.20.176"\n        }\n      ]\n    }\n  ],\n  "combinations": [\n    [\n      2\n    ],\n    [\n      0\n    ],\n    [\n      1\n    ],\n    [\n      3\n    ]\n  ]\n}'
2018-07-16 00:35:07,942:DEBUG:acme.challenges:tls-alpn-01 was not recognized, full message: {'token': 'Sv7BlGqGlCfTpRQYMMIxtl_K4RCiJKWThF2mS2lfao8', 'type': 'tls-alpn-01', 'uri': 'https://acme-v01.api.letsencrypt.org/acme/challenge/2dSWG0oWailkhhO1vZSRGVZAPLMr0IAGr4S5soc1WnA/5694719402', 'status': 'invalid'}
2018-07-16 00:35:07,943:DEBUG:certbot.reporter:Reporting to user: The following errors were reported by the server:

Domain: artisanbakeryexpo.com
Type:   connection
Detail: Timeout after connect (your server may be slow or overloaded)

To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address. Additionally, please check that your computer has a publicly routable IP address and that no firewalls are preventing the server from communicating with the client. If you're using the webroot plugin, you should also verify that you are serving files from the webroot path you provided.
2018-07-16 00:35:07,943:INFO:certbot.auth_handler:Cleaning up challenges
2018-07-16 00:35:07,943:DEBUG:certbot.plugins.standalone:Stopping server at :::443...

My domain is: Multiple domains on same server, all affected. Example: www.nynow.com

My web server is (include version): Ubuntu LAMP on 16.04

The operating system my web server runs on is (include version): Ubuntu LAMP on 16.04

My hosting provider, if applicable, is: Digital Ocean

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

JuergenAuer · July 17, 2018, 3:54pm

Hi @blancast

searching artisanbakeryexpo.com looks fine:

https://transparencyreport.google.com/https/certificates?cert_search_auth=&cert_search_cert=&cert_search=include_expired:false;include_subdomains:false;domain:artisanbakeryexpo.com&lu=cert_search

A certificate created 2018-05-16, now a new certificate created 2018-07-16

And your protocol confuses me a little bit. This

https://acme-v01.api.letsencrypt.org/acme/authz/AdpxPHOPJUx8WAxwW_HKSn0s_oZEZykhoI46xZCNehM

says, you have a valid tls-sni-01 - challenge. The complete authz is valid.

Bit tls-sni-01 needs a 443-port. So this challenge may stop your webserver, then the restart doesn’t work.

And: tls-sni-01 is deprecated.

Possible solution: Switch to version 2 and http-01, so the webserver can be used.

blancast · July 17, 2018, 4:03pm

Hi @JuergenAuer

Thanks for the quick response!

I had no idea that challenge was depreciated. We used this guide to install certbot: https://www.digitalocean.com/community/tutorials/how-to-secure-apache-with-let-s-encrypt-on-ubuntu-16-04

We’re a little new to this. Any suggested reading material on how to properly switch to version 2 and http-01?

schoen · July 17, 2018, 4:43pm

I think the particular problem is that one of the certificates uses standalone and perhaps has a pre-hook to stop the web server but not a post-hook to restart it.

Maybe try

grep -r hook /etc/letsencrypt/renewal

to see which hooks are present.

blancast · July 17, 2018, 5:13pm

Hey @schoen

Thanks for the response! I checked all sites hooks and it looks like all are in place. See this screenshot: https://www.screencast.com/t/b0N7ibjJlK

Any other ideas?

schoen · July 17, 2018, 5:28pm

Is there any reason that running the systemctl stop apache2 followed by systemctl start apache2 might fail to restart Apache? Are there other logs that show the result of this process anywhere?

blancast · July 17, 2018, 5:55pm

Here’s a snapshot of the apache logs during the systemctl start apache2 that seemed to have failed. Seems very suspect, but I have no idea what it means.

schoen · July 17, 2018, 6:09pm

Do you have logs from a timeframe when it failed to restart? What did you to do restart it after that?

blancast · July 17, 2018, 6:30pm

Yep!

Apache Logs:

That 2018-07-16 00:34:53 timestamp in the screenshot above was when it went down.

It coincides directly with this timestamp from the LetsEncrypt log where it stops apache: 2018-07-16 00:34:53,598:INFO:certbot.hooks:Running pre-hook command: systemctl stop apache2

It comes back up some 12 minutes later. Which was triggered by me doing a root password reset in the DigitalOcean dashboard believe it or not (that triggers a reboot of the whole server, so that explains how it brought it up).

JuergenAuer · July 17, 2018, 8:04pm

Looks bad. So many stop and start + downtime. Maybe because you use tls-sni-01 and you need the 443-port, so every time the apache has to stop and start.

http-01 - challenge means, that your client creates a special file, Letsencrypt loads this file. So there is no stop/start required. Maybe only at the end to reload -> using the new certificates.

schoen · July 17, 2018, 8:59pm

Surely this isn't consistent with having used

because that doesn't mention --standalone, --pre-hook, or --post-hook, and yet those were used on all of the certificates on @blancast's server! So somebody else must have done it some other way.

@JuergenAuer, I think you're spreading a minor confusion about the relationship between ACME validation methods and Certbot plugins. In particular, tls-sni-01 is an ACME validation method (also called a challenge type), as is http-01. However, neither is a Certbot plugin and some Certbot plugins can support different methods.

In this case, --apache historically supported only tls-sni-01 while in newer Certbot releases it also supports http-01. --standalone historically supported both tls-sni-01 and http-01. --webroot has always supported only http-01. Any plugin that supports it can still use tls-sni-01 for renewal purposes only, not for initial issuance, under the transition plan that Let's Encrypt is using for the deprecation.

Here, it appears that the (seemingly unsuccessful) web server restart is happening due to a hook that was set in conjunction with --standalone. Thus, this restart would happen regardless of what challenge type is used, because the hook always stops the web server. (It's true that the hook might not be necessary when using --standalone with http-01 challenges if the web server doesn't currently listen on port 80, but the hook doesn't know whether its action is necessary or not and is run regardless of what challenge type is used.)

The switch that you're suggesting involving creating a temporary file is not from tls-sni-01 to http-01, but rather from --standalone to --webroot (while also deleting the associated --pre-hook and --post-hook). For a renewal, this does also have the effect of switching the challenge that would be used from tls-sni-01 to http-01, but this is only incidental in some ways. For example, there are circumstances in which this configuration would already have been using http-01, but it would still be necessary to switch to --webroot in order to avoid the need for the hooks!

@blancast, I would also like to see the logs from /var/log/letsencrypt (for example to see if it tried to restart the web server and with what result) and the Apache error log (to see if Apache logged any error message when Certbot tried to restart it).

JuergenAuer · July 18, 2018, 9:29am

Hi @schoen

thanks for your clarification.

Good to know. I've startet with the version 2, so some historical restrictions are new.

Yes. @blancast has a lot of working websites like artisanbakeryexpo.com - so I'm wondering why creating a new webserver instead of using the existing site and only copying a file.

blancast · July 19, 2018, 7:15pm

Hey all!

Thanks for the detailed responses. Yeah it does look like our setup is configured a bit differently from the mentioned article.

Here is another server that went down this morning - apache shutdown triggered by certbot and never got spun back up

schoen · July 19, 2018, 8:13pm

Can we see the log further down, or the actual content of the log file? The part that you quoted is mostly about Certbot’s decision to attempt to renew the certificate, and not about what happened when it tried to do so.

system · August 18, 2018, 8:13pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.