diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 862cf7a7..a7b5a2a0 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -14,7 +14,7 @@ How to get help Before you ask for help, please make sure you do the following: 1. Read the documentation_ thoroughly. If in a hurry, at least use the search - field that is provided at top-right on the documentation_ pages. Make sure + field that is provided at top-left on the documentation_ pages. Make sure you read the docs for the Pelican version you are using. 2. Use a search engine (e.g., DuckDuckGo, Google) to search for a solution to your problem. Someone may have already found a solution, perhaps in the diff --git a/README.rst b/README.rst index f8b62732..a6341413 100644 --- a/README.rst +++ b/README.rst @@ -1,5 +1,5 @@ -Pelican |build-status| |coverage-status| |pypi-version| |pypi-downloads| -======================================================================== +Pelican |build-status| |pypi-version| |pypi-downloads| +====================================================== Pelican is a static site generator, written in Python_. @@ -59,9 +59,6 @@ Why the name "Pelican"? .. |build-status| image:: https://img.shields.io/travis/getpelican/pelican/master.svg :target: https://travis-ci.org/getpelican/pelican :alt: Travis CI: continuous integration status -.. |coverage-status| image:: https://img.shields.io/coveralls/getpelican/pelican.svg - :target: https://coveralls.io/r/getpelican/pelican - :alt: Coveralls: code coverage status .. |pypi-version| image:: https://img.shields.io/pypi/v/pelican.svg :target: https://pypi.python.org/pypi/pelican :alt: PyPI: the Python Package Index diff --git a/docs/changelog.rst b/docs/changelog.rst index f52d6449..6a4d65a4 100644 --- a/docs/changelog.rst +++ b/docs/changelog.rst @@ -4,7 +4,17 @@ Release history Next release ============ -- Nothing yet +* ``SLUG_SUBSTITUTIONS`` now accepts 3-tuple elements, allowing to keep + non-alphanum characters. Existing 2-tuple configurations will continue to work + without change in behavior. The new 3rd parameter has side effects when there + are multiple substitutions defined. Plese see the docs. +* Tag and category slugs can be controlled with greater precision using the + ``TAG_SUBSTITUTIONS`` and ``CATEGORY_SUBSTITUTIONS`` settings. These also + allow for keeping non-alphanum characters for backward compatibility with + existing URLs. +* Author slugs can be controlled with greater precision using the + ``AUTHOR_SUBSTITUTIONS`` setting. Keeping non-alphanum characters is supported + as well but discouraged. 3.6.3 (2015-08-14) ================== diff --git a/docs/content.rst b/docs/content.rst index 0fa89921..7f6a9591 100644 --- a/docs/content.rst +++ b/docs/content.rst @@ -455,7 +455,13 @@ Option Valid values Description ============= ============ ========================================= anchorlinenos N/A If present wrap line numbers in tags. classprefix string String to prepend to token class names -hl_lines numbers List of lines to be highlighted. +hl_lines numbers List of lines to be highlighted, where + line numbers to highlight are separated + by a space. This is similar to + ``emphasize-lines`` in Sphinx, but it + does not support a range of line numbers + separated by a hyphen, or comma-separated + line numbers. lineanchors string Wrap each line in an anchor using this string and -linenumber. linenos string If present or set to "table" output line diff --git a/docs/contribute.rst b/docs/contribute.rst index 2962ddb1..0ed82dfd 100644 --- a/docs/contribute.rst +++ b/docs/contribute.rst @@ -37,7 +37,7 @@ To clone the Pelican source:: To install the development dependencies:: $ cd src/pelican - $ pip install -r dev_requirements.txt + $ pip install -r requirements/developer.pip To install Pelican and its dependencies:: @@ -69,14 +69,15 @@ or bugfix. The tests live in ``pelican/tests`` and you can run them using the "discover" feature of ``unittest``:: - $ python -m unittest discover + $ python -Wd -m unittest discover After making your changes and running the tests, you may see a test failure mentioning that "some generated files differ from the expected functional tests output." If you have made changes that affect the HTML output generated by Pelican, and the changes to that output are expected and deemed correct given the nature of your changes, then you should update the output used by the -functional tests. To do so, you can use the following two commands:: +functional tests. To do so, **make sure you have both ``en_EN.utf8`` and +``fr_FR.utf8`` locales installed**, and then run the following two commands:: $ LC_ALL=en_US.utf8 pelican -o pelican/tests/output/custom/ \ -s samples/pelican.conf.py samples/content/ diff --git a/docs/install.rst b/docs/install.rst index 067a5cec..f05f3941 100644 --- a/docs/install.rst +++ b/docs/install.rst @@ -1,9 +1,8 @@ Installing Pelican ################## -Pelican currently runs best on Python 2.7.x; earlier versions of Python are -not supported. There is provisional support for Python 3.3+, although there may -be rough edges, particularly with regards to optional 3rd-party components. +Pelican currently runs best on Python 2.7.x and 3.3+; earlier versions of +Python are not supported. You can install Pelican via several different methods. The simplest is via `pip `_:: @@ -23,7 +22,7 @@ session and create a new virtual environment for Pelican:: source bin/activate Once the virtual environment has been created and activated, Pelican can be -be installed via ``pip install pelican`` as noted above. Alternatively, if +installed via ``pip install pelican`` as noted above. Alternatively, if you have the project source, you can install Pelican using the distutils method:: @@ -88,7 +87,7 @@ If you installed Pelican via distutils or the bleeding-edge method, simply perform the same step to install the most recent version. Kickstart your site -=================== +------------------- Once Pelican has been installed, you can create a skeleton project via the ``pelican-quickstart`` command, which begins by asking some questions about diff --git a/docs/settings.rst b/docs/settings.rst index 0adda992..6f695f90 100644 --- a/docs/settings.rst +++ b/docs/settings.rst @@ -61,6 +61,8 @@ Setting name (followed by default value, if any) If ``'fs'``, Pelican will use the file system timestamp information (mtime) if it can't get date information from the metadata. + If given any other string, it will be parsed by the same method + as article metadata. If set to a tuple object, the default datetime object will instead be generated by passing the tuple to the ``datetime.datetime`` constructor. @@ -306,8 +308,14 @@ Setting name (followed by default value, if any) What does it do? ``DAY_ARCHIVE_SAVE_AS = ''`` The location to save per-day archives of your posts. ``SLUG_SUBSTITUTIONS = ()`` Substitutions to make prior to stripping out non-alphanumerics when generating slugs. Specified - as a list of 2-tuples of ``(from, to)`` which are - applied in order. + as a list of 3-tuples of ``(from, to, skip)`` which are + applied in order. ``skip`` is a boolean indicating whether + or not to skip replacement of non-alphanumeric characters. + Useful for backward compatibility with existing URLs. +``AUTHOR_SUBSTITUTIONS = ()`` Substitutions for authors. ``SLUG_SUBSTITUTIONS`` is not + taken into account here! +``CATEGORY_SUBSTITUTIONS = ()`` Added to ``SLUG_SUBSTITUTIONS`` for categories. +``TAG_SUBSTITUTIONS = ()`` Added to ``SLUG_SUBSTITUTIONS`` for tags. ====================================================== ============================================================== .. note:: @@ -317,6 +325,20 @@ Setting name (followed by default value, if any) What does it do? set the corresponding ``*_SAVE_AS`` setting to ``''`` to prevent the relevant page from being generated. +.. note:: + + Substitutions are applied in order with the side effect that keeping + non-alphanum characters applies to the whole string when a replacement + is made. For example if you have the following setting + ``SLUG_SUBSTITUTIONS = (('C++', 'cpp'), ('keep dot', 'keep.dot', True))`` + the string ``Keep Dot`` will be converted to ``keep.dot``, however + ``C++ will keep dot`` will be converted to ``cpp will keep.dot`` instead + of ``cpp-will-keep.dot``! + + If you want to keep non-alphanum characters only for tags or categories + but not other slugs then configure ``TAG_SUBSTITUTIONS`` and + ``CATEGORY_SUBSTITUTIONS`` respectively! + Pelican can optionally create per-year, per-month, and per-day archives of your posts. These secondary archives are disabled by default but are automatically enabled if you supply format strings for their respective ``_SAVE_AS`` settings. @@ -700,15 +722,18 @@ Following are example ways to specify your preferred theme:: The built-in ``notmyidea`` theme can make good use of the following settings. Feel free to use them in your themes as well. -======================= ======================================================= +======================= ===================================================== Setting name What does it do? -======================= ======================================================= +======================= ===================================================== ``SITESUBTITLE`` A subtitle to appear in the header. ``DISQUS_SITENAME`` Pelican can handle Disqus comments. Specify the Disqus sitename identifier here. ``GITHUB_URL`` Your GitHub URL (if you have one). It will then use this information to create a GitHub ribbon. -``GOOGLE_ANALYTICS`` Set to 'UA-XXXX-YYYY' to activate Google Analytics. +``GOOGLE_ANALYTICS`` Set to ``UA-XXXXX-Y`` Property's tracking ID to + activate Google Analytics. +``GA_COOKIE_DOMAIN`` Set cookie domain field of Google Analytics tracking + code. Defaults to ``auto``. ``GOSQUARED_SITENAME`` Set to 'XXX-YYYYYY-X' to activate GoSquared. ``MENUITEMS`` A list of tuples (Title, URL) for additional menu items to appear at the beginning of the main menu. @@ -729,7 +754,7 @@ Setting name What does it do? If not specified, defaults to "links". ``SOCIAL_WIDGET_NAME`` Allows override of the name of the "social" widget. If not specified, defaults to "social". -======================= ======================================================= +======================= ===================================================== In addition, you can use the "wide" version of the ``notmyidea`` theme by adding the following to your configuration:: @@ -752,6 +777,18 @@ be filtered out. For example: ``[(logging.WARN, 'TAG_SAVE_AS is set to False')]`` +It is possible to filter out messages by a template. Check out source code to +obtain a template. + +For example: ``[(logging.WARN, 'Empty alt attribute for image %s in %s')]`` + +**Warning:** Silencing messages by templates is a dangerous feature. It is +possible to unintentionally filter out multiple message types with the same +template (including messages from future Pelican versions). Proceed with +caution. + +Note: This option does nothing ``--debug`` is passed. + .. _reading_only_modified_content: diff --git a/docs/tips.rst b/docs/tips.rst index 82018400..9f45f877 100644 --- a/docs/tips.rst +++ b/docs/tips.rst @@ -9,8 +9,8 @@ Custom 404 Pages When a browser requests a resource that the web server cannot find, the web server usually displays a generic "File not found" (404) error page that can be stark and unsightly. One way to provide an error page that matches the theme -of your site is to create a custom 404 page, such as this Markdown-formatted -example:: +of your site is to create a custom 404 page (*not* an article), such as this +Markdown-formatted example stored in ``content/pages/404.md``:: Title: Not Found Status: hidden @@ -28,8 +28,8 @@ configuration file's ``location`` block:: For Apache:: ErrorDocument 404 /404.html - -For Amazon S3, first navigate to the ``Static Site Hosting`` menu in the + +For Amazon S3, first navigate to the ``Static Site Hosting`` menu in the bucket settings on your AWS cosole. From there:: Error Document: 404.html diff --git a/pelican/__init__.py b/pelican/__init__.py index f27c930b..12db9adf 100644 --- a/pelican/__init__.py +++ b/pelican/__init__.py @@ -323,7 +323,7 @@ def parse_arguments(): help='Comma separated list of selected paths to write') parser.add_argument('--fatal', metavar='errors|warnings', - choices=('errors', 'warnings'), + choices=('errors', 'warnings'), default='', help=('Exit the program with non-zero status if any ' 'errors/warnings encountered.')) @@ -369,6 +369,7 @@ def get_instance(args): config_file = args.settings if config_file is None and os.path.isfile(DEFAULT_CONFIG_NAME): config_file = DEFAULT_CONFIG_NAME + args.settings = DEFAULT_CONFIG_NAME settings = read_settings(config_file, override=get_config(args)) diff --git a/pelican/contents.py b/pelican/contents.py index 0123384a..9b6aa971 100644 --- a/pelican/contents.py +++ b/pelican/contents.py @@ -172,6 +172,7 @@ class Content(object): 'lang': getattr(self, 'lang', 'en'), 'date': getattr(self, 'date', SafeDatetime.now()), 'author': self.author.slug if hasattr(self, 'author') else '', + 'tag': self.tag.slug if hasattr(self, 'tag') else '', 'category': self.category.slug if hasattr(self, 'category') else '' }) return metadata diff --git a/pelican/generators.py b/pelican/generators.py index ff9a9d7c..2107e338 100644 --- a/pelican/generators.py +++ b/pelican/generators.py @@ -5,7 +5,6 @@ import calendar import fnmatch import logging import os -import shutil from codecs import open from collections import defaultdict from functools import partial @@ -21,8 +20,9 @@ from pelican import signals from pelican.cache import FileStampDataCacher from pelican.contents import Article, Draft, Page, Static, is_valid_content from pelican.readers import Readers -from pelican.utils import (DateFormatter, copy, mkdir_p, posixize_path, - process_translations, python_2_unicode_compatible) +from pelican.utils import (DateFormatter, copy, copy_file_metadata, mkdir_p, + posixize_path, process_translations, + python_2_unicode_compatible) logger = logging.getLogger(__name__) @@ -309,24 +309,26 @@ class ArticlesGenerator(CachingGenerator): if self.settings.get('CATEGORY_FEED_ATOM'): writer.write_feed(arts, self.context, self.settings['CATEGORY_FEED_ATOM'] - % cat.slug) + % cat.slug, feed_title=cat.name) if self.settings.get('CATEGORY_FEED_RSS'): writer.write_feed(arts, self.context, self.settings['CATEGORY_FEED_RSS'] - % cat.slug, feed_type='rss') + % cat.slug, feed_title=cat.name, + feed_type='rss') for auth, arts in self.authors: arts.sort(key=attrgetter('date'), reverse=True) if self.settings.get('AUTHOR_FEED_ATOM'): writer.write_feed(arts, self.context, self.settings['AUTHOR_FEED_ATOM'] - % auth.slug) + % auth.slug, feed_title=auth.name) if self.settings.get('AUTHOR_FEED_RSS'): writer.write_feed(arts, self.context, self.settings['AUTHOR_FEED_RSS'] - % auth.slug, feed_type='rss') + % auth.slug, feed_title=auth.name, + feed_type='rss') if (self.settings.get('TAG_FEED_ATOM') or self.settings.get('TAG_FEED_RSS')): @@ -335,12 +337,12 @@ class ArticlesGenerator(CachingGenerator): if self.settings.get('TAG_FEED_ATOM'): writer.write_feed(arts, self.context, self.settings['TAG_FEED_ATOM'] - % tag.slug) + % tag.slug, feed_title=tag.name) if self.settings.get('TAG_FEED_RSS'): writer.write_feed(arts, self.context, self.settings['TAG_FEED_RSS'] % tag.slug, - feed_type='rss') + feed_title=tag.name, feed_type='rss') if (self.settings.get('TRANSLATION_FEED_ATOM') or self.settings.get('TRANSLATION_FEED_RSS')): @@ -725,8 +727,8 @@ class StaticGenerator(Generator): source_path = os.path.join(self.path, sc.source_path) save_as = os.path.join(self.output_path, sc.save_as) mkdir_p(os.path.dirname(save_as)) - shutil.copy2(source_path, save_as) logger.info('Copying %s to %s', sc.source_path, sc.save_as) + copy_file_metadata(source_path, save_as) class SourceFileGenerator(Generator): diff --git a/pelican/log.py b/pelican/log.py index 3d365607..fe14a29b 100644 --- a/pelican/log.py +++ b/pelican/log.py @@ -92,6 +92,7 @@ class LimitFilter(logging.Filter): """ _ignore = set() + _raised_messages = set() _threshold = 5 _group_count = defaultdict(int) @@ -105,12 +106,18 @@ class LimitFilter(logging.Filter): group_args = record.__dict__.get('limit_args', ()) # ignore record if it was already raised - # use .getMessage() and not .msg for string formatting - ignore_key = (record.levelno, record.getMessage()) - if ignore_key in self._ignore: + message_key = (record.levelno, record.getMessage()) + if message_key in self._raised_messages: return False else: - self._ignore.add(ignore_key) + self._raised_messages.add(message_key) + + # ignore LOG_FILTER records by templates when "debug" isn't enabled + logger_level = logging.getLogger().getEffectiveLevel() + if logger_level > logging.DEBUG: + ignore_key = (record.levelno, record.msg) + if ignore_key in self._ignore: + return False # check if we went over threshold if group: diff --git a/pelican/readers.py b/pelican/readers.py index e3b1012a..f130991f 100644 --- a/pelican/readers.py +++ b/pelican/readers.py @@ -288,7 +288,10 @@ class MarkdownReader(BaseReader): with pelican_open(source_path) as text: content = self._md.convert(text) - metadata = self._parse_metadata(self._md.Meta) + if hasattr(self._md, 'Meta'): + metadata = self._parse_metadata(self._md.Meta) + else: + metadata = {} return content, metadata @@ -545,6 +548,8 @@ class Readers(FileStampDataCacher): if content: content = typogrify_wrapper(content) + + if 'title' in metadata: metadata['title'] = typogrify_wrapper(metadata['title']) if 'summary' in metadata: @@ -607,7 +612,10 @@ def default_metadata(settings=None, process=None): metadata['category'] = value if settings.get('DEFAULT_DATE', None) and \ settings['DEFAULT_DATE'] != 'fs': - metadata['date'] = SafeDatetime(*settings['DEFAULT_DATE']) + if isinstance(settings['DEFAULT_DATE'], six.string_types): + metadata['date'] = get_date(settings['DEFAULT_DATE']) + else: + metadata['date'] = SafeDatetime(*settings['DEFAULT_DATE']) return metadata @@ -616,7 +624,7 @@ def path_metadata(full_path, source_path, settings=None): if settings: if settings.get('DEFAULT_DATE', None) == 'fs': metadata['date'] = SafeDatetime.fromtimestamp( - os.stat(full_path).st_ctime) + os.stat(full_path).st_mtime) metadata.update(settings.get('EXTRA_PATH_METADATA', {}).get( source_path, {})) return metadata @@ -651,15 +659,15 @@ def parse_path_metadata(source_path, settings=None, process=None): ('PATH_METADATA', source_path)]: checks.append((settings.get(key, None), data)) if settings.get('USE_FOLDER_AS_CATEGORY', None): - checks.insert(0, ('(?P.*)', subdir)) + checks.append(('(?P.*)', subdir)) for regexp, data in checks: if regexp and data: match = re.match(regexp, data) if match: # .items() for py3k compat. for k, v in match.groupdict().items(): + k = k.lower() # metadata must be lowercase if k not in metadata: - k = k.lower() # metadata must be lowercase if process: v = process(k, v) metadata[k] = v diff --git a/pelican/settings.py b/pelican/settings.py index 82c50c03..a996bf41 100644 --- a/pelican/settings.py +++ b/pelican/settings.py @@ -151,7 +151,7 @@ def read_settings(path=None, override=None): and not isabs(local_settings[p]): absp = os.path.abspath(os.path.normpath(os.path.join( os.path.dirname(path), local_settings[p]))) - if p not in ('THEME') or os.path.exists(absp): + if p != 'THEME' or os.path.exists(absp): local_settings[p] = absp if 'PLUGIN_PATH' in local_settings: diff --git a/pelican/tests/content/empty.md b/pelican/tests/content/empty.md new file mode 100644 index 00000000..e69de29b diff --git a/pelican/tests/content/empty_with_bom.md b/pelican/tests/content/empty_with_bom.md new file mode 100644 index 00000000..5f282702 --- /dev/null +++ b/pelican/tests/content/empty_with_bom.md @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/pelican/tests/output/basic/a-markdown-powered-article.html b/pelican/tests/output/basic/a-markdown-powered-article.html index dd92d691..f533fa3d 100644 --- a/pelican/tests/output/basic/a-markdown-powered-article.html +++ b/pelican/tests/output/basic/a-markdown-powered-article.html @@ -38,7 +38,7 @@ Published: Wed 20 April 2011 -

In cat1.

+

In cat1.

You're mutually oblivious.

a root-relative link to unbelievable @@ -66,4 +66,4 @@ - + \ No newline at end of file diff --git a/pelican/tests/output/basic/archives.html b/pelican/tests/output/basic/archives.html index 27b7c862..01f360eb 100644 --- a/pelican/tests/output/basic/archives.html +++ b/pelican/tests/output/basic/archives.html @@ -69,4 +69,4 @@ - + \ No newline at end of file diff --git a/pelican/tests/output/basic/article-1.html b/pelican/tests/output/basic/article-1.html index b09eef79..a5c92675 100644 --- a/pelican/tests/output/basic/article-1.html +++ b/pelican/tests/output/basic/article-1.html @@ -38,7 +38,7 @@ Published: Thu 17 February 2011 -

In cat1.

+

In cat1.

Article 1

@@ -65,4 +65,4 @@ - + \ No newline at end of file diff --git a/pelican/tests/output/basic/article-2.html b/pelican/tests/output/basic/article-2.html index e340a4f6..1a182c5f 100644 --- a/pelican/tests/output/basic/article-2.html +++ b/pelican/tests/output/basic/article-2.html @@ -38,7 +38,7 @@ Published: Thu 17 February 2011 -

In cat1.

+

In cat1.

Article 2

@@ -65,4 +65,4 @@ - + \ No newline at end of file diff --git a/pelican/tests/output/basic/article-3.html b/pelican/tests/output/basic/article-3.html index 08bf4fc1..7fab0edd 100644 --- a/pelican/tests/output/basic/article-3.html +++ b/pelican/tests/output/basic/article-3.html @@ -38,7 +38,7 @@ Published: Thu 17 February 2011 -

In cat1.

+

In cat1.

Article 3

@@ -65,4 +65,4 @@ - + \ No newline at end of file diff --git a/pelican/tests/output/basic/author/alexis-metaireau.html b/pelican/tests/output/basic/author/alexis-metaireau.html index eeca537a..3fe6c95e 100644 --- a/pelican/tests/output/basic/author/alexis-metaireau.html +++ b/pelican/tests/output/basic/author/alexis-metaireau.html @@ -38,10 +38,10 @@
- By Alexis Métaireau + By Alexis Métaireau
-

In yeah.

-

tags: foobarfoobar

+

In yeah.

+

tags: foo bar foobar

Some content here !

This is a simple title

@@ -74,10 +74,10 @@
- By Alexis Métaireau + By Alexis Métaireau
-

In bar.

-

tags: ohbaryeah

+

In bar.

+

tags: oh bar yeah

Why not ?

After all, why not ? It's pretty simple to do it, and it will allow me to write my blogposts in rst ! @@ -88,8 +88,8 @@ YEAH !

read more
- - + +
+ +