This doesn't use polling unless absolutely necessarily, making it more efficient. It also reduces the amount of first-party code required, and simplifies working out which files are being watched.
`shutil.rmtree` would fail in `TestUtils.test_clean_output_dir` on some
filesystems if an application with a filewatcher had the pelican project
open while the test ran.
Using `tempfile.mkdtmp` for test directories circumvents this.
Improve _HTMLWordTruncator by using more than one unicode block in
_word_regex, making word count function behave properly with CJK,
Cyrillic, and more Latin characters when generating summary.
Combined file and folder watchers under a class and refactored
common watcher related code from __init__.py to the class.
This simplifies the main and autoreload functions in __init__
as well as fix the problem with crashes related to multiprocessing
on systems where default spawn mode is "spawn" instead of "fork".
Adds a use_unicode kwarg to slugify to keep unicode
characters as is (no ASCII-fying) and add tests for
it. Also reworks how slugification logic.
slugify started with the Django method for slugiying:
- Normalize to compatibility decomposed from (NFKD)
- Encode and decode with 'ascii'
This works fine if the decomposed form contains ASCII
characters (i.e. ç can be changed in to c+CEDILLA and
ASCII would keep c only), but fails when decomposition
doesn't result in ASCII characters (i.e. Chinese). To
solve that 'unidecode' was added, which works fine for
both cases. However, old method is now redundant but
was kept. This commit removes the old method and
adjusts logic slightly.
Now slugify will normalize all text with composition
mode (NFKC) to unify format for regex substitutions.
And then if use_unicode is False, uses unidecode to
convert it to ASCII.
Adds a `preserve_case` parameter to the `slugify()` function and uses it
to preserve capital letters in category names when using the Pelican
importer.
This commit removes Six as a dependency for Pelican, replacing the
relevant aliases with the proper Python 3 imports. It also removes
references to Python 2 logic that did not require Six.
Invalid references like those missing semicolons (e.g. `&mdash`) or
those causing overflows (e.g. `�`) are now gracefully
handled and no exception is thrown.
This commit also adds tests and comments where needed.
Also update HTML output by running (after making sure to have the fr_FR.utf8
locale installed):
```sh
LC_ALL=en_US.utf8 pelican -o pelican/tests/output/custom/ -s samples/pelican.conf.py samples/content/
LC_ALL=fr_FR.utf8 pelican -o pelican/tests/output/custom_locale/ -s samples/pelican.conf_FR.py samples/content/
LC_ALL=en_US.utf8 pelican -o pelican/tests/output/basic/ samples/content/
```
as described at
http://docs.getpelican.com/en/3.6.3/contribute.html#running-the-test-suite
* Fix {filename} links on Windows.
Otherwise '{filename}/foo/bar.jpg' doesn't work
* Clean up relative Posix path handling in contents.
* Use Posix paths in readers
* Environment for Popen must be strs, not unicodes.
* Ignore Git CRLF warnings.
* Replace CRLFs with LFs in inputs on Windows.
* Fix importer tests
* Fix test_contents
* Fix one last backslash in paginated output
* Skip the remaining failing locale tests on Windows.
* Document the use of forward slashes on Windows.
* Add some Fabric and ghp-import notes
reverts getpelican/pelican@ddcccfeaa9
If one used a locale that made use of unicode characters (like fr_FR.UTF-8)
the files on disk would be in correct locale while links would be to C.
Uses a SafeDatetime class that works with unicode format strigns
by using custom strftime to prevent ascii decoding errors with Python2.
Also added unicode decoding for the calendar module to fix period
archives.
The locale is a global state, and it was not properly reset to
whatever it was before the unitttest possibly changed it.
This is now fixed.
Not restoring the locale led to weird issues: depending on
the order chosen by "python -m unittest discover" to run
the unit tests, some tests would apparently randomly fail
due to the locale not being what was expected.
For example, test_period_in_timeperiod_archive would
call mock('posts/1970/ 1月/index.html',...) instead of
expected mock('posts/1970/Jan/index.html',...) and fail.
