forked from github/pelican
03d9c38871
Adds a use_unicode kwarg to slugify to keep unicode characters as is (no ASCII-fying) and add tests for it. Also reworks how slugification logic. slugify started with the Django method for slugiying: - Normalize to compatibility decomposed from (NFKD) - Encode and decode with 'ascii' This works fine if the decomposed form contains ASCII characters (i.e. ç can be changed in to c+CEDILLA and ASCII would keep c only), but fails when decomposition doesn't result in ASCII characters (i.e. Chinese). To solve that 'unidecode' was added, which works fine for both cases. However, old method is now redundant but was kept. This commit removes the old method and adjusts logic slightly. Now slugify will normalize all text with composition mode (NFKC) to unify format for regex substitutions. And then if use_unicode is False, uses unidecode to convert it to ASCII. |
||
|---|---|---|
| .. | ||
| plugins | ||
| tests | ||
| themes | ||
| tools | ||
| __init__.py | ||
| __main__.py | ||
| cache.py | ||
| contents.py | ||
| generators.py | ||
| log.py | ||
| paginator.py | ||
| readers.py | ||
| rstdirectives.py | ||
| server.py | ||
| settings.py | ||
| urlwrappers.py | ||
| utils.py | ||
| writers.py | ||