Using certbot renew with pre-/posthook in windows 10 shows error

Help
1

Hi!

OS: Windows 10 Home, I'm admin
Domains: Some Dyndns, e.g. 24.dedyn.io
certbot 1.22.0

I've created some certificates with no problems.

  C:\Certbot>certbot certonly ^
    --standalone ^
    --preferred-challenges http ^
    --email me@whatever.de ^
    -d 24.dedyn.io

I don't think it matters because --standalone, but ...
Web server nginx 1.21.3 (WinNMP)

A test with renew also worked.

C:\Certbot> certbot renew --cert-name "24.dedyn.io" --debug --dry-run
Saving debug log to C:\Certbot\log\letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing C:\Certbot\renewal\24.dedyn.io.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Simulating renewal of an existing certificate for 24.dedyn.io

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all simulated renewals succeeded:
  C:\Certbot\live\24.dedyn.io\fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

But if I use pre-/posthook, the following error occurs, regardless of which program/script I specify.
(D:/WinNMP/WinNMP.exe /k = kill WinNMP, will be executed successfully
D:/WinNMP/WinNMP.exe /s = start WinNMP, will not be executed)

C:\Certbot> certbot renew --cert-name "24.dedyn.io" -v --debug --dry-run --pre-hook "D:/WinNMP/WinNMP.exe /k" --post-hook "D:/WinNMP/WinNMP.exe /s"
Saving debug log to C:\Certbot\log\letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing C:\Certbot\renewal\24.dedyn.io.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Certificate not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator standalone, Installer None
Running pre-hook command: D:/WinNMP/WinNMP.exe /k
Exception in thread Thread-2:
Traceback (most recent call last):
  File "threading.py", line 973, in _bootstrap_inner
  File "threading.py", line 910, in run
  File "subprocess.py", line 1479, in _readerthread
  File "encodings\cp1252.py", line 23, in decode
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 136: character maps to <undefined>
Failed to renew certificate 24.dedyn.io with error: 'NoneType' object has no attribute 'strip'

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All simulated renewals failed. The following certificates could not be renewed:
  C:\Certbot\live\24.dedyn.io\fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Exiting abnormally:
Traceback (most recent call last):
  File "runpy.py", line 197, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "C:\Program Files (x86)\Certbot\bin\certbot.exe\__main__.py", line 29, in <module>
    sys.exit(main())
  File "C:\Program Files (x86)\Certbot\pkgs\certbot\main.py", line 19, in main
    return internal_main.main(cli_args)
  File "C:\Program Files (x86)\Certbot\pkgs\certbot\_internal\main.py", line 1632, in main
    return config.func(config, plugins)
  File "C:\Program Files (x86)\Certbot\pkgs\certbot\_internal\main.py", line 1518, in renew
    renewal.handle_renewal_request(config)
  File "C:\Program Files (x86)\Certbot\pkgs\certbot\_internal\renewal.py", line 511, in handle_renewal_request
    raise errors.Error("{0} renew failure(s), {1} parse failure(s)".format(
certbot.errors.Error: 1 renew failure(s), 0 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile C:\Certbot\log\letsencrypt.log or re-run Certbot with -v for more details.

Can someone help me please?

1 Like
2

[abbreviated output]

curl -Ii6 24.dedyn.io
HTTP/1.1 403 Forbidden
Content-Length: 3433

curl -Ii4 24.dedyn.io
HTTP/1.1 301 Moved Permanently
Server: nginx
Content-Length: 162

Name:      24.dedyn.io
Addresses: 2003:f8:f706:3700:52f:91e6:f9f7:6a73
           217.251.233.83
1 Like
3

This looks like a bug in Certbot. I'll try investigate it soon.

In the meantime, you could try redirecting the output from WinNMP.exe to NUL (or to a file) to prevent the decoding error from happening.

On Windows, I think you would tack this onto the commands:

>NUL 2>NUL
2 Likes
4

Thanks for quick reply!

But error still there, and there is no output on command line for 'D:/WinNMP/WinNMP.exe /k'. It's a win gui program.

C:\Certbot> certbot renew --cert-name "24.dedyn.io" -v --debug --dry-run --pre-hook "D:/WinNMP/WinNMP.exe /k >NUL 2>NUL" --post-hook "D:/WinNMP/WinNMP.exe /s >NUL 2>NUL"
Saving debug log to C:\Certbot\log\letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing C:\Certbot\renewal\24.dedyn.io.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Certificate not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator standalone, Installer None
Running pre-hook command: D:/WinNMP/WinNMP.exe /k >NUL 2>NUL
Exception in thread Thread-2:
Traceback (most recent call last):
  File "threading.py", line 973, in _bootstrap_inner
  File "threading.py", line 910, in run
  File "subprocess.py", line 1479, in _readerthread
  File "encodings\cp1252.py", line 23, in decode
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 136: character maps to <undefined>
Failed to renew certificate 24.dedyn.io with error: 'NoneType' object has no attribute 'strip'

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All simulated renewals failed. The following certificates could not be renewed:
  C:\Certbot\live\24.dedyn.io\fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Exiting abnormally:
Traceback (most recent call last):
  File "runpy.py", line 197, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "C:\Program Files (x86)\Certbot\bin\certbot.exe\__main__.py", line 29, in <module>
    sys.exit(main())
  File "C:\Program Files (x86)\Certbot\pkgs\certbot\main.py", line 19, in main
    return internal_main.main(cli_args)
  File "C:\Program Files (x86)\Certbot\pkgs\certbot\_internal\main.py", line 1632, in main
    return config.func(config, plugins)
  File "C:\Program Files (x86)\Certbot\pkgs\certbot\_internal\main.py", line 1518, in renew
    renewal.handle_renewal_request(config)
  File "C:\Program Files (x86)\Certbot\pkgs\certbot\_internal\renewal.py", line 511, in handle_renewal_request
    raise errors.Error("{0} renew failure(s), {1} parse failure(s)".format(
certbot.errors.Error: 1 renew failure(s), 0 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile C:\Certbot\log\letsencrypt.log or re-run Certbot with -v for more details.
1 Like
5

Could you please post the C:\Certbot\logs\letsencrypt.log file which contains the crash stack trace, for later investigation?

Sorry that the workaround didn't work.

1 Like
6

@rg305
Sorry, unfortunately I don't understand what this is supposed to mean for me.

1 Like
7

letsencrypt.log.txt (7.9 KB)

Here you are.

1 Like
8

Thanks. We'll get this fixed for next release.

I'll try find a workaround for restarting WinNMP later tonight or tomorrow as well.

3 Likes
9

It means the IPv6 returns "403 Forbidden" while the IPv4 returns "301 Moved Permanently".
Are they being served from the same web server?

1 Like
10

Yes, they are both the same nginx server. I was probably testing certbot with --prehook and the web server was down or the certbot server was online. Now both return "301 Moved Permanently", http calls are redirected to https. Does curl have an option to address the server via https?

In any case, thanks for the explanation!

1 Like
13

Sorry... my own FW was blocking me from your IPv6:80 site :frowning:

2 Likes
14

@_az
I'm currently testing manual-auth-hook and manual-cleanup-hook for manual mode. Same mistake. If you are interested...

C:\Certbot\manual-hooks>certbot certonly --manual --dry-run --test-cert --manual-auth-hook "C:\Certbot\manual-hooks\authenticator.bat" --manual-cleanup-hook "C:\Certbot\manual-hooks\cleanup.bat" --preferred-challenges dns --email klemm.f@gmx.de -d 24.dedyn.io -d *.24.dedyn.io
Saving debug log to C:\Certbot\log\letsencrypt.log
Simulating a certificate request for 24.dedyn.io and *.24.dedyn.io
Exception in thread Thread-2:
Traceback (most recent call last):
  File "threading.py", line 973, in _bootstrap_inner
  File "threading.py", line 910, in run
  File "subprocess.py", line 1479, in _readerthread
  File "encodings\cp1252.py", line 23, in decode
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 138: character maps to <undefined>
Encountered exception during recovery: KeyError: KeyAuthorizationAnnotatedChallenge(challb=ChallengeBody(chall=DNS01(token=b'\x85+\xf4E\xd4\xb1\xb8\xa3\xf30\x1b\xd2\xd1CF\x1b \xfb\xcep #\x88l~&faG{F7'), uri='https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/1417707188/1k0yBw', _url='https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/1417707188/1k0yBw', status=Status(pending), validated=None, error=None), domain='24.dedyn.io', account_key=JWKRSA(key=<ComparableRSAKey(<cryptography.hazmat.backends.openssl.rsa._RSAPrivateKey object at 0x0502F778>)>))
An unexpected error occurred:
AttributeError: 'NoneType' object has no attribute 'strip'
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile C:\Certbot\log\letsencrypt.log or re-run Certbot with -v for more details.

It is important, however, that no arguments are passed to the script, which would be necessary to create the txt record. Is that a consequential failure of the first mistake?

I will now try the whole thing with wsl under windows with ubuntu to see how it actually works. Then I can roughly prepare the Windows scripts.

edit: Sorry, tried to upload log file, but don't worked. Is 17kb too big?
edit2: Ok, sorry, I've seen it, environment variables.

1 Like
15

Well, I'm trying to get this to work for myself as well (on Server 2019), but I'm having different issues.

The problem I'm encountering is that WinNMP.exe /s doesn't always terminate. Certbot ends up hanging, waiting for the process to exit.

Calling WinNMP.exe /s appears to behave differently based on what stdout handle is attached to the process. Since the WinNMP program itself appears to be closed source, it's not so easy to dig into "why?".

But this is all a separate issue to the two issues you've hit so far:

  • The encoding error
  • The NoneType error

so I might have to find a way to get a Windows 10 virtual machine and see whether that gets me closer to what you're seeing.

Could you please try these two commands for me and show me the output:

"C:\Program Files (x86)\Certbot\Python\python.exe" -c "import subprocess; subprocess.run(['powershell.exe', '-Command', 'D:/WinNMP/WinNMP.exe /s'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False, universal_newlines=True)"

and

"C:\Program Files (x86)\Certbot\Python\python.exe" -c "import subprocess; subprocess.run(['powershell.exe', '-Command', 'D:/WinNMP/WinNMP.exe /s'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False, universal_newlines=False)"
2 Likes
16

You are welcome.

First variant, WinNMP runs.
The command brings WinNMP to the foreground and immediately ends with the error message.

C:\Certbot>"C:\Program Files (x86)\Certbot\Python\python.exe" -c "import subprocess; subprocess.run(['powershell.exe', '-Command', 'D:/WinNMP/WinNMP.exe /s'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False, universal_newlines=True)"
Exception in thread Thread-2:
Traceback (most recent call last):
  File "threading.py", line 973, in _bootstrap_inner
  File "threading.py", line 910, in run
  File "subprocess.py", line 1479, in _readerthread
  File "encodings\cp1252.py", line 23, in decode
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 138: character maps to <undefined>

First variant, WinNMP doesn't run.
The command starts WinNMP and then waits for WinNMP to close, then ends with the error message.

C:\Certbot>"C:\Program Files (x86)\Certbot\Python\python.exe" -c "import subprocess; subprocess.run(['powershell.exe', '-Command', 'D:/WinNMP/WinNMP.exe /s'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False, universal_newlines=True)"
Exception in thread Thread-2:
Traceback (most recent call last):
  File "threading.py", line 973, in _bootstrap_inner
  File "threading.py", line 910, in run
  File "subprocess.py", line 1479, in _readerthread
  File "encodings\cp1252.py", line 23, in decode
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 138: character maps to <undefined>

Second variant, WinNMP runs.
The command brings WinNMP to the foreground and ends.

C:\Certbot>"C:\Program Files (x86)\Certbot\Python\python.exe" -c "import subprocess; subprocess.run(['powershell.exe', '-Command', 'D:/WinNMP/WinNMP.exe /s'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False, universal_newlines=False)"

(no output)

Second variant, WinNMP doesn't run.
The command starts WinNMP and ends.

C:\Certbot>"C:\Program Files (x86)\Certbot\Python\python.exe" -c "import subprocess; subprocess.run(['powershell.exe', '-Command', 'D:/WinNMP/WinNMP.exe /s'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False, universal_newlines=False)"

(no output)

That's easy. If certbot starts WinNMP when it's already running, a second instance of WinNMP is started, which sees that it's already running, brings the already running one to the foreground and exits again. The best way to run a program only once (single instance). Therefore, certbot does not have to wait because the program it started has already ended.

If WinNMP is not running, certbot starts the main program and has to wait for it to exit.

In Windows scripts, you can use an argument at the start command to specify whether it should wait for completion or continue executing. But I'm shure you know this. :grin:

If there's anything else I can do to help, just ask.

1 Like
17

I agree this is what should happen ordinarily, but it seems that connecting stdout to subprocess.PIPE causes it to hang instead. (At least, in my test environment).

Oops, sorry, can you do this one as well:

"C:\Program Files (x86)\Certbot\Python\python.exe" -c "import subprocess; print(subprocess.run(['powershell.exe', '-Command', 'D:/WinNMP/WinNMP.exe /s'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False, universal_newlines=False))"
2 Likes
18

Very strange.

First the error message:

C:\Certbot>"C:\Program Files (x86)\Certbot\Python\python.exe" -c "import subprocess; print(subprocess.run(['powershell.exe', '-Command', 'D:/WinNMP/WinNMP.exe /s'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False, universal_newlines=False))"
CompletedProcess(args=['powershell.exe', '-Command', 'D:/WinNMP/WinNMP.exe /s'], returncode=0, stdout=b'', stderr=b'. : Die Datei "D:\\Users\\Itsme\\Dokumente\\WindowsPowerShell\\Microsoft.PowerShell_profile.ps1" kann nicht geladen \r\nwerden, da die Ausf\x81hrung von Skripts auf diesem System deaktiviert ist. Weitere Informationen finden Sie unter \r\n"about_Execution_Policies" (https:/go.microsoft.com/fwlink/?LinkID=135170).\r\nIn Zeile:1 Zeichen:3\r\n+ . \'D:\\Users\\Itsme\\Dokumente\\WindowsPowerShell\\Microsoft.PowerSh ...\r\n+   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n    + CategoryInfo          : Sicherheitsfehler: (:) [], PSSecurityException\r\n    + FullyQualifiedErrorId : UnauthorizedAccess\r\n')

Third variant, WinNMP doesn't run
The command starts WinNMP and then waits for WinNMP to close, then ends with the error message.

When WinNMP starts, I hear a sound like a message box (on notification), but I don't see any. But only in a command prompt with admin rights, not a normal one. The error message is the same for both.

I can run locally created Powershell scripts without any problems.

I can't think of more, sorry.

2 Likes
19

Well, that's helpful. The earlier error:

looks to be directly related to the content of the PowerShell error:

Ausf\x81hrung

So we have two problems:

  1. Certbot code, when calling hooks, is not handling decoding the output of the hooks properly. This needs to be fixed, but won't make the hook work for you.
  2. Your system has disabled PowerShell scripting in some way which interferes with Certbot launching the hooks. I'm afraid I don't know much about Windows policies and it's not clear to me whether there's anything we can do about this in Certbot.
2 Likes
20

To run the powershell version you'd need to either disable execution policy (add -ExecutionPolicy Bypass) or set it to a less restricted level (that can be done at the user level or the machine level). You should find out what your policy level is (run Get-ExecutionPolicy in a powershell command line).

Instead of redirecting the command output directly, try putting the original command in a .bat file and call that instead. You should consider contacting the software vendor and point them to this thread.

Normally windows services run as actual windows services (so you could just stop and start them), but I think this app may be a GUI service (run on startup).

I assume the app doesn't actually setup a windows service for Apache etc? If it did you could just restart that rather than restarting WinNMP. You may be able to call httpd to restart just the webserver httpd -k restart. The reason I suggest this is that commanding an actual GUI via certbot smells like the wrong thing to have to do as you would also be unable to automate this renewal when not signed in.

4 Likes
21

@_az

I don't understand why the scripts don't work either.

So Certbot uses Powershell? Then please try to place another '-ExecutionPolicy', 'Bypass' between 'Powershell.exe' and '-Command'. I've unlocked everything here, but it still doesn't work.

2 Likes
22

What's the Get-ExecutionPolicy of your system?

1 Like