Requesting certs for Non-ASCII-Domain (IDN) via certbot causes trouble

Help
1

hey there,

I run Apache2 on my Debian 8 (Jessie) server, where I host two websites using two virtual-hosts.
Because I want to enable HTTPS-access to them, I installed certbot via

git clone git://github.com/certbot/certbot.git

and performed it using

root@myserver:~/certbot# ./certbot-auto

After inserting my domain as punycode (because my domain nürnberg.rocks is a Internationalized Domain Name and not ASCII-comaptible), the following error occurs:

An unexpected error occurred:
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 5: ordinal not in range(128).

I’m using the certbot version 0.13.0 and I thought, punycode-domains should be supported in this version…?

Can someone help me solving this problem or explain, how I can request certs for ounycode-domains?

Thanks!

2

Well, the error is quite clear IMO: on the fifth position, it encounters a non-ASCII character.

So it appears your punycode isn't punycoded enough? What exactly did you enter into certbot?

Edit:
My certbot works perfectly with the punycoded hostname xn--nrnberg-n2a.rocks?

1 Like
3

Could you post the log from /var/log/letsencrypt, which should include a full traceback showing where the error came from?

4

Hi @hotcore,

As @Osiris commented, I suppose you are trying to issue the cert using your domain in unicode nürnberg.rocks but you should issue it using punycode xn--nrnberg-n2a.rocks.

Cheers,
sahsanu

5

As others correctly mentioned above, you're probably entering those names as-is, rather than in "punycode" form (in which case it would work with most clients). Out of interest, could you please provide the output of the locale command (either in this thread or privately) - this weekend I was actually adding a way to handle exactly this case to my LE client and would be interested in reproducing the problem :slight_smile:

6

Hey guys, thanks for your answers so far!

As certbot asked for the domain names, i entered "xn--nrnberg-n2a.rocks" (should be punycode...), while the real domain is nürnberg.rocks.
And this "xn--nrnberg-n2a.rocks" seems to produces the error, the fifth position is an "n", which is IMO an ASCII-character, isn't it?

I will post the entire log that evening, as well as answering leader[quote="leader, post:5, topic:32637"]
Out of interest, could you please provide the output of the locale command (either in this thread or privately)
[/quote]

7

Considering that you are entering already punycoded (if there is such a word) domain name and it is encoded correctly and the error points to exactly that letter (ü), I would imagine that your input is not a problem, but decoding punycode somewhere down the pipeline is. I had a quick look at the python code and I believe the problem lies around line 187 of certbot_apache/tls_sni_01.py (similar one exists in tls-sni for Nginx):

    return self.VHOST_TEMPLATE.format(
        vhost=ips,
        server_name=achall.response(achall.account_key).z_domain.decode('ascii'),
        ssl_options_conf_path=self.configurator.mod_ssl_conf,
        cert_path=self.get_cert_path(achall),
        key_path=self.get_key_path(achall),
       document_root=document_root).replace("\n", os.linesep)

I would imagine that using some other challenge type instead of tls-sni or using a standalone mode should solve this for you (also make sure the domain name in the Apache config is encoded too).

NB: Please note that I’m not a certbot developer and I usually prefer Perl to Python, so the above is just a theory, but I believe a plausible one.

@schoen can probably say whether that theory holds any water. :slight_smile:

8

I would also like to see the log.

Differently from @leader, I suspect the problem is with some other Unicode character on your system (e.g. in a filename or as part of an Apache configuration file). Is it possible that you have a ü character in a filename or Apache configuration file that Certbot would be processing?

If so, and if that causes this crash, that is a but in Certbot that we’ll need to fix.

9

First at all, here is the logfile from /var/log/letsencrypt/letsencrypt.log

[code] 2017-04-24 20:33:30,603:DEBUG:certbot.log:Root logging level set at 20
2017-04-24 20:33:30,604:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2017-04-24 20:33:30,605:DEBUG:certbot.main:certbot version: 0.13.0
2017-04-24 20:33:30,605:DEBUG:certbot.main:Arguments: ['--apache']
2017-04-24 20:33:30,606:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#nginx,PluginEntryPoint#standalone,PluginEntryPoint#manual,PluginEntryPoint#webroot,PluginEntryPoin$
2017-04-24 20:33:30,607:DEBUG:certbot.plugins.selection:Requested authenticator apache and installer apache
2017-04-24 20:33:31,495:DEBUG:certbot_apache.configurator:Apache version is 2.4.10
2017-04-24 20:33:32,005:DEBUG:certbot.plugins.selection:Single candidate plugin: * apache
Description: Apache Web Server plugin - Beta
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.configurator:ApacheConfigurator
Initialized: <certbot_apache.configurator.ApacheConfigurator object at 0x7fb6ba1849d0>
Prep: True
2017-04-24 20:33:32,007:DEBUG:certbot.plugins.selection:Selected authenticator <certbot_apache.configurator.ApacheConfigurator object at 0x7fb6ba1849d0> and installer <certbot_apache.configurato$
2017-04-24 20:33:32,019:DEBUG:certbot.main:Picked account: <Account(4bc16a081291d8b57fe8463ce9697bf1)>
2017-04-24 20:33:32,022:DEBUG:acme.client:Sending GET request to https://acme-v01.api.letsencrypt.org/directory.
2017-04-24 20:33:32,084:DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
2017-04-24 20:35:02,839:DEBUG:requests.packages.urllib3.connectionpool:https://acme-v01.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 352
2017-04-24 20:35:02,844:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 352
Boulder-Request-Id: amQr9aw-5cgv1Kfh9iJXCEv29M8-V5PEOk5QA8dRcgA
Replay-Nonce: WXL_NhRovns_Yhu5uoDc8WwjP3IQ7nShmRqMWYKYvlI
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Mon, 24 Apr 2017 20:35:06 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 24 Apr 2017 20:35:06 GMT
Connection: keep-alive

{
"key-change": "https://acme-v01.api.letsencrypt.org/acme/key-change",
"new-authz": "https://acme-v01.api.letsencrypt.org/acme/new-authz",
"new-cert": "https://acme-v01.api.letsencrypt.org/acme/new-cert",
"new-reg": "https://acme-v01.api.letsencrypt.org/acme/new-reg",
"revoke-cert": "https://acme-v01.api.letsencrypt.org/acme/revoke-cert"
}
2017-04-24 20:35:02,846:DEBUG:certbot.util:Not suggesting name "www.nürnberg.rocks"
2017-04-24 20:35:02,849:DEBUG:certbot.util:Non-ASCII domain names not supported. To issue for an Internationalized Domain Name, use Punycode.
2017-04-24 20:35:02,850:DEBUG:certbot.util:Not suggesting name "gwsan.nürnberg.rocks"
2017-04-24 20:35:02,850:DEBUG:certbot.util:Non-ASCII domain names not supported. To issue for an Internationalized Domain Name, use Punycode.
2017-04-24 20:35:09,424:INFO:certbot.main:Obtaining a new certificate
2017-04-24 20:35:09,426:DEBUG:acme.client:Requesting fresh nonce
2017-04-24 20:35:09,426:DEBUG:acme.client:Sending HEAD request to https://acme-v01.api.letsencrypt.org/acme/new-authz.
2017-04-24 20:35:09,626:DEBUG:requests.packages.urllib3.connectionpool:https://acme-v01.api.letsencrypt.org:443 "HEAD /acme/new-authz HTTP/1.1" 405 0
2017-04-24 20:35:09,629:DEBUG:acme.client:Received response:
HTTP 405
Server: nginx
Content-Type: application/problem+json
Content-Length: 91
Allow: POST
Boulder-Request-Id: rk9nzhuCJ6xZxiS9ucj6JRT69vVHy1iFQzXbZkb2Y8k
Replay-Nonce: dfoF4bMa-srCC34vk7ZayGE_Z6vdYAmVN3fz0ZwkEss
Expires: Mon, 24 Apr 2017 20:35:13 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 24 Apr 2017 20:35:13 GMT
Connection: keep-alive

2017-04-24 20:35:09,629:DEBUG:acme.client:Storing nonce: dfoF4bMa-srCC34vk7ZayGE_Z6vdYAmVN3fz0ZwkEss
2017-04-24 20:35:09,630:DEBUG:acme.client:JWS payload:
{
"identifier": {
"type": "dns",
"value": "xn--nrnberg-n2a.rocks"
},
"resource": "new-authz"
}
2017-04-24 20:35:09,636:DEBUG:acme.client:Sending POST request to https://acme-v01.api.letsencrypt.org/acme/new-authz:
{
"header": {
"alg": "RS256",
"jwk": {
"e": "AQAB",
"kty": "RSA",
"n": "pUTd9HzSRfumd5I9YDkmETzpYqRjosi4KvfeAavhTJaJaAvRhNdoGx6h4TmMI3lcf0SvyG7jtF5g5BX7cgm3qZo18RsLaDzjjLN41GfrIOW5z03pjJHx0LCZ8a0N7jh1ghobL2DNwgkenQyn-SQbQ80bvRle15lNqjq_2PRFmzqUy4_tNU6sBf$
}
},
"protected": "eyJub25jZSI6ICJkZm9GNGJNYS1zckNDMzR2azdaYXlHRV9aNnZkWUFtVk4zZnowWndrRXNzIn0",
"payload": "ewogICJpZGVudGlmaWVyIjogewogICAgInR5cGUiOiAiZG5zIiwgCiAgICAidmFsdWUiOiAieG4tLW5ybmJlcmctbjJhLnJvY2tzIgogIH0sIAogICJyZXNvdXJjZSI6ICJuZXctYXV0aHoiCn0",
"signature": "hHRATgMMKjC60rTyHHqFoNXhAtvCeQSJLk3pB5KoWS8C-RQ7ZRTiOcGCOimWenX5HM2BsTxb43ScpN-1bgzXIioaAB_TEIp05OZQsFEFdSScFtMw5QCLieO-kZj7BFNJ0I7bcz1qhykfJ9EAspiAAZdBZiEN3yehy_df9186dkCnrI7Qp9$
}
2017-04-24 20:35:09,855:DEBUG:requests.packages.urllib3.connectionpool:https://acme-v01.api.letsencrypt.org:443 "POST /acme/new-authz HTTP/1.1" 201 1009
2017-04-24 20:35:09,857:DEBUG:acme.client:Received response:
HTTP 201
Server: nginx
Content-Type: application/json
Content-Length: 1009
Boulder-Request-Id: u7GV-7knyFYg1OBe-TuSf81Hs4Gd0K3uPi27e9Tp2KY
Boulder-Requester: 13159593
Link: https://acme-v01.api.letsencrypt.org/acme/new-cert;rel="next"
Location: https://acme-v01.api.letsencrypt.org/acme/authz/z7_DJyqHTCX0a8ubk3eHzwCXJoxyubCw1jeWy1KvvE8
Replay-Nonce: pIbJkWZ3IMwGRvBGuKJlRZX16xVxALGTKoaBtZaJxTk
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Mon, 24 Apr 2017 20:35:13 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 24 Apr 2017 20:35:13 GMT
Connection: keep-alive

{
"identifier": {
"type": "dns",
"value": "xn--nrnberg-n2a.rocks"
},
"status": "pending",
"expires": "2017-05-01T20:35:13.184024764Z",
"challenges": [
{
"type": "http-01",
"status": "pending",
"uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/z7_DJyqHTCX0a8ubk3eHzwCXJoxyubCw1jeWy1KvvE8/1078291794",
"token": "OurzFgq8UXRkcQzixp1hmpfynwybFhT5PUu4eXoKDx8"
},
{
"type": "dns-01",
"status": "pending",
"uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/z7_DJyqHTCX0a8ubk3eHzwCXJoxyubCw1jeWy1KvvE8/1078291795",
"token": "tgKpc-ul4sR6bPOmKhS4QRAGmPqHB7qCv_Va_tS_m6E"
},
{
"type": "tls-sni-01",
"status": "pending",
"uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/z7_DJyqHTCX0a8ubk3eHzwCXJoxyubCw1jeWy1KvvE8/1078291796",
"token": "vIgeIZWo02c2mdOfo1Nq17U7X5vujwmbkbYGRK4R-hA"
}
],
"combinations": [
[
0
],
[
2
],
[
1
]
]
}
2017-04-24 20:35:09,857:DEBUG:acme.client:Storing nonce: pIbJkWZ3IMwGRvBGuKJlRZX16xVxALGTKoaBtZaJxTk
2017-04-24 20:35:09,859:INFO:certbot.auth_handler:Performing the following challenges:
2017-04-24 20:35:09,860:INFO:certbot.auth_handler:tls-sni-01 challenge for xn--nrnberg-n2a.rocks
2017-04-24 20:35:10,110:INFO:certbot_apache.configurator:Enabled Apache socache_shmcb module
2017-04-24 20:35:10,423:INFO:certbot_apache.configurator:Enabled Apache ssl module
2017-04-24 20:35:10,911:DEBUG:certbot.error_handler:Encountered exception:
Traceback (most recent call last):
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/auth_handler.py", line 115, in _solve_challenges
resp = self.auth.perform(self.achalls)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/configurator.py", line 1762, in perform
sni_response = chall_doer.perform()
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/tls_sni_01.py", line 79, in perform
addrs = self._mod_config()
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/tls_sni_01.py", line 101, in _mod_config
achall_addrs = self._get_addrs(achall)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/tls_sni_01.py", line 126, in _get_addrs
vhost = self.configurator.choose_vhost(achall.domain, temp=True)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/configurator.py", line 331, in choose_vhost
return self._choose_vhost_from_list(target_name, temp)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/configurator.py", line 335, in _choose_vhost_from_list
vhost = display_ops.select_vhost(target_name, self.vhosts)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/display_ops.py", line 29, in select_vhost
code, tag = _vhost_menu(domain, vhosts)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/display_ops.py", line 79, in _vhost_menu
name_size=disp_name_size)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 5: ordinal not in range(128)

2017-04-24 20:35:10,911:DEBUG:certbot.error_handler:Calling registered functions
2017-04-24 20:35:10,911:INFO:certbot.auth_handler:Cleaning up challenges
2017-04-24 20:35:11,549:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
File "/root/.local/share/letsencrypt/bin/letsencrypt", line 11, in
sys.exit(main())
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/main.py", line 755, in main
return config.func(config, plugins)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/main.py", line 597, in run
certname, lineage)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/main.py", line 82, in _get_and_save_cert
lineage = le_client.obtain_and_enroll_certificate(domains, certname)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/client.py", line 316, in obtain_and_enroll_certificate
certr, chain, key, _ = self.obtain_certificate(domains)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/client.py", line 285, in obtain_certificate
self.config.allow_subset_of_names)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/auth_handler.py", line 74, in get_authorizations
resp = self._solve_challenges()
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/auth_handler.py", line 115, in _solve_challenges
resp = self.auth.perform(self.achalls)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/configurator.py", line 1762, in perform
sni_response = chall_doer.perform()
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/tls_sni_01.py", line 79, in perform
addrs = self._mod_config()
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/tls_sni_01.py", line 101, in _mod_config
achall_addrs = self._get_addrs(achall)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/tls_sni_01.py", line 126, in _get_addrs
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/configurator.py", line 331, in choose_vhost
return self._choose_vhost_from_list(target_name, temp)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/configurator.py", line 335, in _choose_vhost_from_list
vhost = display_ops.select_vhost(target_name, self.vhosts)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/display_ops.py", line 29, in select_vhost
code, tag = _vhost_menu(domain, vhosts)
File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/display_ops.py", line 79, in _vhost_menu
name_size=disp_name_size)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 5: ordinal not in range(128) [/code]

I have edited none of the apache-config-files, so I am quite shure, there is nowhere an ü-character in there, exept from the virtual-host-config file. At /etc/apache2/sites-available/myvhost.conf the domain written as-is (with ü) is set as ServerName-Directive.
Is certbot processing the vhost-file while creating the certs? If so, this might be the problem.

10

Yes, I'm almost positive that's the problem. I'll file a bug about this.

11

Submitted as

Thanks for letting us know about the problem and providing a log. Unfortunately, the best workaround until the problem is fixed in Certbot might be to represent your domain name with Punycode even within your Apache configuration file.

1 Like
12

Interesting, before the first edit I put there a question in bold if you have a domain name as is rather in punicode form in the Apache config, though considering that it might be not very likely with what you said about using punycode, I then re-phrased it to “make sure it’s encoded in Apache config” :slight_smile:

Anyways, it would be interesting to see what happens if you run it in standalone mode - technically there is no need for certbot to check configs then I guess?

13

Indded, this was nearly fixing the problem. I changed the ServerName-Directive in the vhost-config-file into the punycode domain, an ran again
/root/certbot/certbot-auto --apache

Afterwards I was asked to choose the domain, I want to request certificates for, and the only one shown was the "root"-domain "xn--nrnberg-n2a.rocks".
HTPPS-acces to that page works well now, but the only website I am hosting at the moment should be accessed via a subdomain "gwsan.xn--nrnberg-n2a.rocks".
The cert doesn't seem to work for the subdomain as well automatically, but creating a cert for that subdomain seems not to be easyly done by running

    cd /root/certbot
    ./certbot-auto --domains gwsan.xn--nrnberg-n2a.rocks

Feedback:

Obtaining a new certificate
Performing the following challenges:
tls-sni-01 challenge for gwsan.xn--nrnberg-n2a.rocks
Cleaning up challenges
An unexpected error occurred:
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 5: ordinal not in range(128)

The ServerName at the only enabled vhost-config at the moment (responsible for this website) is gwsan.xn--nrnberg-n2a.rocks.

How do i get certs for that subdomain, if creating one for the root-domain works now?

14

Sorry leader, I was not aware that you could also have ment a vhsot-file in the apache-configuration. Thats why I didn't thought about that at first :wink:

How do I start that standalone-mode?

15

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.