Merge branch 'master' into switch-to-pytest

.travis.yml

@@ -9,21 +9,17 @@ addons:
   apt_packages:
     - pandoc
 before_install:
- - sudo apt-get update -qq
+  - sudo apt-get update -qq
- - sudo locale-gen fr_FR.UTF-8 tr_TR.UTF-8
+  - sudo locale-gen fr_FR.UTF-8 tr_TR.UTF-8
 install:
-    - pip install python-coveralls
+    - pip install coveralls -U virtualenv py -e . -r dev_requirements.txt tox
-    - pip install -U virtualenv py
-    - pip install -e . -r dev_requirements.txt tox
 script:
-  - tox
+  - tox -e $TESTENV --sitepackages
-  - py.test --cov=pelican --cov-report=term-missing tests
 after_success:
-  # Report coverage results to coveralls.io
+  - py.test --cov=pelican --cov-report=term-missing tests
-  - pip install coveralls
   - coveralls
 notifications:
-    irc:
+  irc:
-        channels:
+    channels:
-            - "irc.freenode.org#pelican"
+      - "irc.freenode.org#pelican"
-        on_success: change
+    on_success: change

README.rst

@@ -3,57 +3,58 @@ Pelican |build-status| |coverage-status|
 
 Pelican is a static site generator, written in Python_.
 
-* Write your weblog entries directly with your editor of choice (vim!)
+* Write content in reStructuredText_ or Markdown_ using your editor of choice
-  in reStructuredText_ or Markdown_
+* Includes a simple command line tool to (re)generate site files
-* Includes a simple CLI tool to (re)generate the weblog
+* Easy to interface with version control systems and web hooks
-* Easy to interface with DVCSes and web hooks
+* Completely static output is simple to host anywhere
-* Completely static output is easy to host anywhere
+
 
 Features
 --------
 
 Pelican currently supports:
 
-* Blog articles and pages
+* Chronological content (e.g., articles, blog posts) as well as static pages
-* Comments, via an external service (Disqus). (Please note that while
+* Integration with external services (e.g., Google Analytics and Disqus)
-  useful, Disqus is an external service, and thus the comment data will be
+* Site themes (created using Jinja2_ templates)
-  somewhat outside of your control and potentially subject to data loss.)
-* Theming support (themes are created using Jinja2_ templates)
-* PDF generation of the articles/pages (optional)
 * Publication of articles in multiple languages
-* Atom/RSS feeds
+* Generation of Atom and RSS feeds
-* Code syntax highlighting
+* Syntax highlighting via Pygments_
-* Import from WordPress, Dotclear, or RSS feeds
+* Importing existing content from WordPress, Dotclear, and other services
-* Integration with external tools: Twitter, Google Analytics, etc. (optional)
+* Fast rebuild times due to content caching and selective output writing
-* Fast rebuild times thanks to content caching and selective output writing.
 
-Have a look at the `Pelican documentation`_ for more information.
+Check out `Pelican's documentation`_ for further information.
 
-Why the name "Pelican"?
------------------------
-
-"Pelican" is an anagram for *calepin*, which means "notebook" in French. ;)
-
-Source code
------------
-
-You can access the source code at: https://github.com/getpelican/pelican
-
-If you feel hackish, have a look at the explanation of `Pelican's internals`_.
 
 How to get help, contribute, or provide feedback
 ------------------------------------------------
 
 See our `contribution submission and feedback guidelines <CONTRIBUTING.rst>`_.
 
+
+Source code
+-----------
+
+Pelican's source code is `hosted on GitHub`_. If you feel like hacking,
+take a look at `Pelican's internals`_.
+
+
+Why the name "Pelican"?
+-----------------------
+
+"Pelican" is an anagram of *calepin*, which means "notebook" in French.
+
+
 .. Links
 
 .. _Python: http://www.python.org/
 .. _reStructuredText: http://docutils.sourceforge.net/rst.html
 .. _Markdown: http://daringfireball.net/projects/markdown/
 .. _Jinja2: http://jinja.pocoo.org/
-.. _`Pelican documentation`: http://docs.getpelican.com/
+.. _Pygments: http://pygments.org/
+.. _`Pelican's documentation`: http://docs.getpelican.com/
 .. _`Pelican's internals`: http://docs.getpelican.com/en/latest/internals.html
+.. _`hosted on GitHub`: https://github.com/getpelican/pelican
 
 .. |build-status| image:: https://img.shields.io/travis/getpelican/pelican/master.svg
    :target: https://travis-ci.org/getpelican/pelican

docs/settings.rst

@@ -847,13 +847,11 @@ can be invoked by passing the ``--archive`` flag).
 
 The cache files are Python pickles, so they may not be readable by
 different versions of Python as the pickle format often changes. If
-such an error is encountered, the cache files have to be rebuilt by
+such an error is encountered, it is caught and the cache file is 
-removing them and re-running Pelican, or by using the Pelican
+rebuilt automatically in the new format. The cache files will also be 
-command-line option ``--ignore-cache``. The cache files also have to
+rebuilt after the ``GZIP_CACHE`` setting has been changed.
-be rebuilt when changing the ``GZIP_CACHE`` setting for cache file
-reading to work properly.
 
-The ``--ignore-cache`` command-line option is also useful when the
+The ``--ignore-cache`` command-line option is useful when the
 whole cache needs to be regenerated, such as when making modifications
 to the settings file that will affect the cached content, or just for
 debugging purposes. When Pelican runs in autoreload mode, modification

docs/themes.rst

@@ -329,6 +329,108 @@ period_archives.html template
 <https://github.com/getpelican/pelican/blob/master/pelican/themes/simple/templates/period_archives.html>`_.
 
 
+Objects
+=======
+
+Detail objects attributes that are available and useful in templates. Not all
+attributes are listed here, this is a selection of attributes considered useful
+in a template.
+
+.. _object-article:
+
+Article
+-------
+
+The string representation of an Article is the `source_path` attribute.
+
+===================     ===================================================
+Attribute               Description
+===================     ===================================================
+author                  The :ref:`Author <object-author_cat_tag>` of
+                        this article.
+authors                 A list of :ref:`Authors <object-author_cat_tag>`
+                        of this article.
+category                The :ref:`Category <object-author_cat_tag>`
+                        of this article.
+content                 The rendered content of the article.
+date                    Datetime object representing the article date.
+date_format             Either default date format or locale date format.
+default_template        Default template name.
+in_default_lang         Boolean representing if the article is written
+                        in the default language.
+lang                    Language of the article.
+locale_date             Date formated by the `date_format`.
+metadata                Article header metadata `dict`.
+save_as                 Location to save the article page.
+slug                    Page slug.
+source_path             Full system path of the article source file.
+status                  The article status, can be any of 'published' or
+                        'draft'.
+summary                 Rendered summary content.
+tags                    List of :ref:`Tag <object-author_cat_tab>`
+                        objects.
+template                Template name to use for rendering.
+title                   Title of the article.
+translations            List of translations
+                        :ref:`Article <object-article>` objects.
+url                     URL to the article page.
+===================     ===================================================
+
+.. _object-author_cat_tag:
+
+Author / Category / Tag
+-----------------------
+
+The string representation of those objects is the `name` attribute.
+
+===================     ===================================================
+Attribute               Description
+===================     ===================================================
+name                    Name of this object [1]_.
+page_name               Author page name.
+save_as                 Location to save the author page.
+slug                    Page slug.
+url                     URL to the author page.
+===================     ===================================================
+
+.. [1] for Author object, coming from `:authors:` or `AUTHOR`.
+
+.. _object-page:
+
+Page
+----
+
+The string representation of a Page is the `source_path` attribute.
+
+===================     ===================================================
+Attribute               Description
+===================     ===================================================
+author                  The :ref:`Author <object-author_cat_tag>` of
+                        this page.
+content                 The rendered content of the page.
+date                    Datetime object representing the page date.
+date_format             Either default date format or locale date format.
+default_template        Default template name.
+in_default_lang         Boolean representing if the article is written
+                        in the default language.
+lang                    Language of the article.
+locale_date             Date formated by the `date_format`.
+metadata                Page header metadata `dict`.
+save_as                 Location to save the page.
+slug                    Page slug.
+source_path             Full system path of the page source file.
+status                  The page status, can be any of 'published' or
+                        'draft'.
+summary                 Rendered summary content.
+tags                    List of :ref:`Tag <object-author_cat_tab>`
+                        objects.
+template                Template name to use for rendering.
+title                   Title of the page.
+translations            List of translations
+                        :ref:`Article <object-article>` objects.
+url                     URL to the page.
+===================     ===================================================
+
 Feeds
 =====
 

pelican/__init__.py

@@ -321,7 +321,8 @@ def get_config(args):
         config['CACHE_PATH'] = args.cache_path
     if args.selected_paths:
         config['WRITE_SELECTED'] = args.selected_paths.split(',')
-    config['RELATIVE_URLS'] = args.relative_paths
+    if args.relative_paths:
+        config['RELATIVE_URLS'] = args.relative_paths
     config['DEBUG'] = args.verbosity == logging.DEBUG
 
     # argparse returns bytes in Py2. There is no definite answer as to which

pelican/contents.py

@@ -1,7 +1,7 @@
 # -*- coding: utf-8 -*-
 from __future__ import unicode_literals, print_function
 import six
-from six.moves.urllib.parse import (unquote, urlparse, urlunparse)
+from six.moves.urllib.parse import urlparse, urlunparse
 
 import copy
 import locale
@@ -53,7 +53,7 @@ class Content(object):
         self._context = context
         self.translations = []
 
-        local_metadata = dict(settings['DEFAULT_METADATA'])
+        local_metadata = dict()
         local_metadata.update(metadata)
 
         # set metadata as attributes
@@ -90,7 +90,7 @@ class Content(object):
 
             self.in_default_lang = (self.lang == default_lang)
 
-        # create the slug if not existing, generate slug according to 
+        # create the slug if not existing, generate slug according to
         # setting of SLUG_ATTRIBUTE
         if not hasattr(self, 'slug'):
             if settings['SLUGIFY_SOURCE'] == 'title' and hasattr(self, 'title'):
@@ -166,21 +166,13 @@ class Content(object):
         """Returns the URL, formatted with the proper values"""
         metadata = copy.copy(self.metadata)
         path = self.metadata.get('path', self.get_relative_source_path())
-        default_category = self.settings['DEFAULT_CATEGORY']
-        slug_substitutions = self.settings.get('SLUG_SUBSTITUTIONS', ())
         metadata.update({
             'path': path_to_url(path),
             'slug': getattr(self, 'slug', ''),
             'lang': getattr(self, 'lang', 'en'),
             'date': getattr(self, 'date', SafeDatetime.now()),
-            'author': slugify(
+            'author': self.author.slug if hasattr(self, 'author') else '',
-                getattr(self, 'author', ''),
+            'category': self.category.slug if hasattr(self, 'category') else ''
-                slug_substitutions
-            ),
-            'category': slugify(
-                getattr(self, 'category', default_category),
-                slug_substitutions
-            )
         })
         return metadata
 
@@ -316,8 +308,13 @@ class Content(object):
         """Dummy function"""
         pass
 
-    url = property(functools.partial(get_url_setting, key='url'))
+    @property
-    save_as = property(functools.partial(get_url_setting, key='save_as'))
+    def url(self):
+        return self.get_url_setting('url')
+
+    @property
+    def save_as(self):
+        return self.get_url_setting('save_as')
 
     def _get_template(self):
         if hasattr(self, 'template') and self.template is not None:

pelican/generators.py

@@ -544,10 +544,8 @@ class ArticlesGenerator(CachingGenerator):
             if hasattr(article, 'tags'):
                 for tag in article.tags:
                     self.tags[tag].append(article)
-            # ignore blank authors as well as undefined
             for author in getattr(article, 'authors', []):
-                if author.name != '':
+                self.authors[author].append(article)
-                    self.authors[author].append(article)
         # sort the articles by date
         self.articles.sort(key=attrgetter('date'), reverse=True)
         self.dates = list(self.articles)

pelican/readers.py

@@ -28,16 +28,44 @@ from pelican.contents import Page, Category, Tag, Author
 from pelican.utils import get_date, pelican_open, FileStampDataCacher, SafeDatetime, posixize_path
 
 
+def strip_split(text, sep=','):
+    """Return a list of stripped, non-empty substrings, delimited by sep."""
+    items = [x.strip() for x in text.split(sep)]
+    return [x for x in items if x]
+
+
+# Metadata processors have no way to discard an unwanted value, so we have
+# them return this value instead to signal that it should be discarded later.
+# This means that _filter_discardable_metadata() must be called on processed
+# metadata dicts before use, to remove the items with the special value.
+_DISCARD = object()
+
+
+def _process_if_nonempty(processor, name, settings):
+    """Removes extra whitespace from name and applies a metadata processor.
+    If name is empty or all whitespace, returns _DISCARD instead.
+    """
+    name = name.strip()
+    return processor(name, settings) if name else _DISCARD
+
+
 METADATA_PROCESSORS = {
-    'tags': lambda x, y: [Tag(tag, y) for tag in x.split(',')],
+    'tags': lambda x, y: [Tag(tag, y) for tag in strip_split(x)] or _DISCARD,
     'date': lambda x, y: get_date(x.replace('_', ' ')),
     'modified': lambda x, y: get_date(x),
-    'status': lambda x, y: x.strip(),
+    'status': lambda x, y: x.strip() or _DISCARD,
-    'category': Category,
+    'category': lambda x, y: _process_if_nonempty(Category, x, y),
-    'author': Author,
+    'author': lambda x, y: _process_if_nonempty(Author, x, y),
-    'authors': lambda x, y: [Author(author.strip(), y) for author in x.split(',')],
+    'authors': lambda x, y: [Author(a, y) for a in strip_split(x)] or _DISCARD,
+    'slug': lambda x, y: x.strip() or _DISCARD,
 }
 
+
+def _filter_discardable_metadata(metadata):
+    """Return a copy of a dict, minus any items marked as discardable."""
+    return {name: val for name, val in metadata.items() if val is not _DISCARD}
+
+
 logger = logging.getLogger(__name__)
 
 class BaseReader(object):
@@ -447,14 +475,14 @@ class Readers(FileStampDataCacher):
 
         reader = self.readers[fmt]
 
-        metadata = default_metadata(
+        metadata = _filter_discardable_metadata(default_metadata(
-            settings=self.settings, process=reader.process_metadata)
+            settings=self.settings, process=reader.process_metadata))
         metadata.update(path_metadata(
             full_path=path, source_path=source_path,
             settings=self.settings))
-        metadata.update(parse_path_metadata(
+        metadata.update(_filter_discardable_metadata(parse_path_metadata(
             source_path=source_path, settings=self.settings,
-            process=reader.process_metadata))
+            process=reader.process_metadata)))
         reader_name = reader.__class__.__name__
         metadata['reader'] = reader_name.replace('Reader', '').lower()
 
@@ -462,7 +490,7 @@ class Readers(FileStampDataCacher):
         if content is None:
             content, reader_metadata = reader.read(path)
             self.cache_data(path, (content, reader_metadata))
-        metadata.update(reader_metadata)
+        metadata.update(_filter_discardable_metadata(reader_metadata))
 
         if content:
             # find images with empty alt
@@ -537,6 +565,10 @@ def find_empty_alt(content, path):
 def default_metadata(settings=None, process=None):
     metadata = {}
     if settings:
+        for name, value in dict(settings.get('DEFAULT_METADATA', {})).items():
+            if process:
+                value = process(name, value)
+            metadata[name] = value
         if 'DEFAULT_CATEGORY' in settings:
             value = settings['DEFAULT_CATEGORY']
             if process:

pelican/themes/notmyidea/templates/index.html

@@ -23,7 +23,7 @@
             {% endif %}
         {# other items #}
         {% else %}
-            {% if loop.first and articles_page.has_previous %}
+            {% if loop.first %}
                 <section id="content" class="body">
                     <ol id="posts-list" class="hfeed" start="{{ articles_paginator.per_page -1 }}">
             {% endif %}

pelican/writers.py

@@ -3,7 +3,6 @@ from __future__ import with_statement, unicode_literals, print_function
 import six
 
 import os
-import locale
 import logging
 
 if not six.PY3:

tests/test_contents.py

@@ -8,7 +8,7 @@ import os.path
 
 from tests.support import unittest, get_settings
 
-from pelican.contents import Page, Article, Static, URLWrapper
+from pelican.contents import Page, Article, Static, URLWrapper, Author, Category
 from pelican.settings import DEFAULT_CONFIG
 from pelican.utils import path_to_url, truncate_html_words, SafeDatetime, posix_join
 from pelican.signals import content_object_init
@@ -33,7 +33,7 @@ class TestPage(unittest.TestCase):
             'metadata': {
                 'summary': TEST_SUMMARY,
                 'title': 'foo bar',
-                'author': 'Blogger',
+                'author': Author('Blogger', DEFAULT_CONFIG),
             },
             'source_path': '/path/to/file/foo.ext'
         }
@@ -374,7 +374,8 @@ class TestPage(unittest.TestCase):
         content = Page(**args)
         assert content.authors == [content.author]
         args['metadata'].pop('author')
-        args['metadata']['authors'] = ['First Author', 'Second Author']
+        args['metadata']['authors'] = [Author('First Author', DEFAULT_CONFIG),
+                                       Author('Second Author', DEFAULT_CONFIG)]
         content = Page(**args)
         assert content.authors
         assert content.author == content.authors[0]
@@ -396,8 +397,8 @@ class TestArticle(TestPage):
         settings['ARTICLE_URL'] = '{author}/{category}/{slug}/'
         settings['ARTICLE_SAVE_AS'] = '{author}/{category}/{slug}/index.html'
         article_kwargs = self._copy_page_kwargs()
-        article_kwargs['metadata']['author'] = "O'Brien"
+        article_kwargs['metadata']['author'] = Author("O'Brien", settings)
-        article_kwargs['metadata']['category'] = 'C# & stuff'
+        article_kwargs['metadata']['category'] = Category('C# & stuff', settings)
         article_kwargs['metadata']['title'] = 'fnord'
         article_kwargs['settings'] = settings
         article = Article(**article_kwargs)

tests/test_generators.py

@@ -413,6 +413,38 @@ class TestArticlesGenerator(unittest.TestCase):
         generator.generate_context()
         generator.readers.read_file.assert_called_count == orig_call_count
 
+    def test_standard_metadata_in_default_metadata(self):
+        settings = get_settings(filenames={})
+        settings['CACHE_CONTENT'] = False
+        settings['DEFAULT_CATEGORY'] = 'Default'
+        settings['DEFAULT_DATE'] = (1970, 1, 1)
+        settings['DEFAULT_METADATA'] = (('author', 'Blogger'),
+                                        # category will be ignored in favor of
+                                        # DEFAULT_CATEGORY
+                                        ('category', 'Random'),
+                                        ('tags', 'general, untagged'))
+        generator = ArticlesGenerator(
+            context=settings.copy(), settings=settings,
+            path=CONTENT_DIR, theme=settings['THEME'], output_path=None)
+        generator.generate_context()
+
+        authors = sorted([author.name for author, _ in generator.authors])
+        authors_expected = sorted(['Alexis Métaireau', 'Blogger',
+                                   'First Author', 'Second Author'])
+        self.assertEqual(authors, authors_expected)
+
+        categories = sorted([category.name
+                             for category, _ in generator.categories])
+        categories_expected = [
+            sorted(['Default', 'TestCategory', 'yeah', 'test', '指導書']),
+            sorted(['Default', 'TestCategory', 'Yeah', 'test', '指導書'])]
+        self.assertIn(categories, categories_expected)
+
+        tags = sorted([tag.name for tag in generator.tags])
+        tags_expected = sorted(['bar', 'foo', 'foobar', 'general', 'untagged',
+                                'パイソン', 'マック'])
+        self.assertEqual(tags, tags_expected)
+
 
 class TestPageGenerator(unittest.TestCase):
     # Note: Every time you want to test for a new field; Make sure the test

tests/test_paginator.py

@@ -5,7 +5,7 @@ import locale
 from tests.support import unittest, get_settings
 
 from pelican.paginator import Paginator
-from pelican.contents import Article
+from pelican.contents import Article, Author
 from pelican.settings import DEFAULT_CONFIG
 from jinja2.utils import generate_lorem_ipsum
 
@@ -26,7 +26,6 @@ class TestPage(unittest.TestCase):
             'metadata': {
                 'summary': TEST_SUMMARY,
                 'title': 'foo bar',
-                'author': 'Blogger',
             },
             'source_path': '/path/to/file/foo.ext'
         }
@@ -49,6 +48,7 @@ class TestPage(unittest.TestCase):
             key=lambda r: r[0],
         )
 
+        self.page_kwargs['metadata']['author'] = Author('Blogger', settings)
         object_list = [Article(**self.page_kwargs), Article(**self.page_kwargs)]
         paginator = Paginator('foobar.foo', object_list, settings)
         page = paginator.page(1)