Merge branch 'master' into html_list_tags

docs/changelog.rst

@@ -5,7 +5,16 @@ Next release
 ============
 
 * New signal: ``feed_generated``
-* Make the HTML reader parse multiple occurences of metadata tags as list
+* Replace Fabric by Invoke and ``fabfile.py`` template by ``tasks.py``.
+* Replace ``SLUG_SUBSTITUTIONS`` (and friends) by ``SLUG_REGEX_SUBSTITUTIONS``
+  for more finegrained control
+* ``'{base_name}'`` value in ``PAGINATION_PATTERNS`` setting no longer strips
+  ``'bar'`` from ``'foo/bar.html'`` (unless ``'bar' == 'index'``).
+* ``ARTICLE_ORDER_BY`` and ``PAGE_ORDER_BY`` now also affect 1) category, tag
+  and author pages 2) feeds 3) draft and hidden articles and pages
+* New ``ARTICLE_TRANSLATION_ID`` and ``PAGE_TRANSLATION_ID`` settings to specify
+  metadata attributes used to identify translations; or to disable translations
+* Make the HTML reader parse multiple occurrences of metadata tags as a list
 
 3.7.1 (2017-01-10)
 ==================

docs/content.rst

@@ -11,7 +11,7 @@ The idea behind "pages" is that they are usually not temporal in nature and are
 used for content that does not change very often (e.g., "About" or "Contact"
 pages).
 
-You can find sample content in the repository at: ``pelican/samples/content/``
+You can find sample content in the repository at ``samples/content/``.
 
 .. _internal_metadata:
 
@@ -70,6 +70,22 @@ Metadata syntax for Markdown posts should follow this pattern::
 
     This is the content of my super blog post.
 
+You can also have your own metadata keys (so long as they don't conflict with reserved metadata keywords) for use in your python templates. The following is the list of reserved metadata keywords:
+
+* `Title`
+* `Tags`
+* `Date`
+* `Modified`
+* `Status`
+* `Category`
+* `Author`
+* `Authors`
+* `Slug`
+* `Summary`
+* `Template`
+* `Save_as`
+* `Url`
+
 Readers for additional formats (such as AsciiDoc_) are available via plugins.
 Refer to `pelican-plugins`_ repository for those.
 
@@ -370,8 +386,9 @@ of available translations for that article.
    language. For such advanced functionality the `i18n_subsites
    plugin`_ can be used.
 
-Pelican uses the article's URL "slug" to determine if two or more articles are
-translations of one another. The slug can be set manually in the file's
+By default, Pelican uses the article's URL "slug" to determine if two or more
+articles are translations of one another. (This can be changed with the
+``ARTICLE_TRANSLATION_ID`` setting.) The slug can be set manually in the file's
 metadata; if not set explicitly, Pelican will auto-generate the slug from the
 title of the article.
 
@@ -517,9 +534,9 @@ your settings file.
 Publishing drafts
 =================
 
-If you want to publish an article as a draft (for friends to review before
-publishing, for example), you can add a ``Status: draft`` attribute to its
-metadata. That article will then be output to the ``drafts`` folder and not
+If you want to publish an article or a page as a draft (for friends to review
+before publishing, for example), you can add a ``Status: draft`` attribute to
+its metadata. That article will then be output to the ``drafts`` folder and not
 listed on the index page nor on any category or tag page.
 
 If your articles should be automatically published as a draft (to not accidentally
@@ -535,6 +552,6 @@ metadata to include ``Status: published``.
 .. _W3C ISO 8601: http://www.w3.org/TR/NOTE-datetime
 .. _AsciiDoc: http://www.methods.co.nz/asciidoc/
 .. _pelican-plugins: http://github.com/getpelican/pelican-plugins
-.. _Markdown Extensions: http://pythonhosted.org/Markdown/extensions/
-.. _CodeHilite extension: http://pythonhosted.org/Markdown/extensions/code_hilite.html#syntax
+.. _Markdown Extensions: https://python-markdown.github.io/extensions/
+.. _CodeHilite extension: https://python-markdown.github.io/extensions/code_hilite/#syntax
 .. _i18n_subsites plugin: http://github.com/getpelican/pelican-plugins/tree/master/i18n_subsites

docs/contribute.rst

@@ -14,32 +14,32 @@ When doing so, please adhere to the following guidelines.
 Setting up the development environment
 ======================================
 
-While there are many ways to set up one's development environment, following
-is a method that uses `virtualenv <https://virtualenv.pypa.io/en/stable/>`_.
+While there are many ways to set up one's development environment, we recommend
+using `Virtualenv <https://virtualenv.pypa.io/en/stable/>`_. This tool allows
+you to set up separate environments for separate Python projects that are
+isolated from one another so you can use different packages (and package
+versions) for each.
+
 If you don't have ``virtualenv`` installed, you can install it via::
 
     $ pip install virtualenv
 
-Virtual environments allow you to work on Python projects which are isolated
-from one another so you can use different packages (and package versions) with
-different projects.
-
-To create and activate a virtual environment, use the following syntax::
+Use ``virtualenv`` to create and activate a virtual environment::
 
     $ virtualenv ~/virtualenvs/pelican
     $ cd ~/virtualenvs/pelican
     $ . bin/activate
 
-To clone the Pelican source::
+Clone the Pelican source into a subfolder called ``src/pelican``::
 
     $ git clone https://github.com/getpelican/pelican.git src/pelican
-
-To install the development dependencies::
-
     $ cd src/pelican
+
+Install the development dependencies::
+
     $ pip install -r requirements/developer.pip
 
-To install Pelican and its dependencies::
+Install Pelican and its dependencies::
 
     $ python setup.py develop
 
@@ -47,7 +47,8 @@ Or using ``pip``::
 
     $ pip install -e .
 
-To conveniently test on multiple Python versions, we also provide a .tox file.
+To conveniently test on multiple Python versions, we also provide a ``.tox``
+file.
 
 
 Building the docs
@@ -57,7 +58,7 @@ If you make changes to the documentation, you should preview your changes
 before committing them::
 
     $ pip install -r requirements/docs.pip
-    $ cd src/pelican/docs
+    $ cd docs
     $ make html
 
 Open ``_build/html/index.html`` in your browser to preview the documentation.
@@ -79,8 +80,8 @@ mentioning that "some generated files differ from the expected functional tests
 output." If you have made changes that affect the HTML output generated by
 Pelican, and the changes to that output are expected and deemed correct given
 the nature of your changes, then you should update the output used by the
-functional tests. To do so, **make sure you have both ``en_EN.utf8`` and
-``fr_FR.utf8`` locales installed**, and then run the following two commands::
+functional tests. To do so, **make sure you have both** ``en_EN.utf8`` **and**
+``fr_FR.utf8`` **locales installed**, and then run the following two commands::
 
     $ LC_ALL=en_US.utf8 pelican -o pelican/tests/output/custom/ \
         -s samples/pelican.conf.py samples/content/
@@ -89,56 +90,60 @@ functional tests. To do so, **make sure you have both ``en_EN.utf8`` and
     $ LC_ALL=en_US.utf8 pelican -o pelican/tests/output/basic/ \
         samples/content/
 
-Testing on Python 2 and 3
--------------------------
+You may also find that some tests are skipped because some dependency (e.g.,
+Pandoc) is not installed. This does not automatically mean that these tests
+have passed; you should at least verify that any skipped tests are not
+affected by your changes.
 
-Testing on Python 3 currently requires some extra steps: installing
-Python 3-compatible versions of dependent packages and plugins.
+You should run the test suite under each of the supported versions of Python.
+This is best done by creating a separate Python environment for each version.
+Tox_ is a useful tool to automate running tests inside ``virtualenv``
+environments.
 
-Tox_ is a useful tool to run tests on both versions. It will install the
-Python 3-compatible version of dependent packages.
+.. _Tox: https://tox.readthedocs.io/en/latest/
 
-.. _Tox: http://testrun.org/tox/latest/
+Python 2/3 compatibility development tips
+=========================================
 
-Python 3 development tips
-=========================
+Here are some tips that may be useful for writing code that is compatible with
+both Python 2.7 and Python 3:
 
-Here are some tips that may be useful when doing some code for both Python 2.7
-and Python 3 at the same time:
+- Use new syntax. For example:
 
-- Assume every string and literal is unicode (import unicode_literals):
+  - ``print .. -> print(..)``
+  - ``except .., e -> except .. as e``
 
-  - Do not use prefix ``u'``.
-  - Do not encode/decode strings in the middle of sth. Follow the code to the
-    source (or target) of a string and encode/decode at the first/last possible
-    point.
-  - In other words, write your functions to expect and to return unicode.
-  - Encode/decode strings if e.g. the source is a Python function that is known
-    to handle this badly, e.g. strftime() in Python 2.
+- Use new methods. For example:
 
-- Use new syntax: print function, "except ... *as* e" (not comma), etc.
-- Refactor method calls like ``dict.iteritems()``, ``xrange()``, etc. in a way
-  that runs without code change in both Python versions.
-- Do not use magic method ``__unicode()__`` in new classes. Use only ``__str()__``
-  and decorate the class with ``@python_2_unicode_compatible``.
-- Do not start int literals with a zero. This is a syntax error in Py3k.
-- Unfortunately I did not find an octal notation that is valid in both
-  Pythons. Use decimal instead.
-- use six, e.g.:
+  - ``dict.iteritems() -> dict.items()``
+  - ``xrange(..) - > list(range(..))``
+
+- Use ``six`` where necessary. For example:
 
   - ``isinstance(.., basestring) -> isinstance(.., six.string_types)``
   - ``isinstance(.., unicode) -> isinstance(.., six.text_type)``
 
-- ``setlocale()`` in Python 2 bails when we give the locale name as unicode,
+- Assume every string and literal is Unicode:
+
+  - Use ``from __future__ import unicode_literals``
+  - Do not use the prefix ``u'`` before strings.
+  - Do not encode/decode strings in the middle of something. Follow the code to
+    the source/target of a string and encode/decode at the first/last possible
+    point.
+  - In particular, write your functions to expect and to return Unicode.
+  - Encode/decode strings if the string is the output of a Python function that
+    is known to handle this badly. For example, ``strftime()`` in Python 2.
+  - Do not use the magic method ``__unicode()__`` in new classes. Use only
+    ``__str()__`` and decorate the class with ``@python_2_unicode_compatible``.
+
+- ``setlocale()`` in Python 2 fails when we give the locale name as Unicode,
   and since we are using ``from __future__ import unicode_literals``, we do
-  that everywhere!  As a workaround, I enclosed the localename with ``str()``;
-  in Python 2 this casts the name to a byte string, in Python 3 this should do
-  nothing, because the locale name already had been unicode.
-
-- Kept range() almost everywhere as-is (2to3 suggests list(range())), just
-  changed it where I felt necessary.
-
-- Changed xrange() back to range(), so it is valid in both Python versions.
+  that everywhere! As a workaround, enclose the locale name with ``str()``;
+  in Python 2 this casts the name to a byte string, while in Python 3 this
+  should do nothing, because the locale name was already Unicode.
+- Do not start integer literals with a zero. This is a syntax error in Python 3.
+- Unfortunately there seems to be no octal notation that is valid in both
+  Python 2 and 3. Use decimal notation instead.
 
 
 Logging tips

docs/importer.rst

@@ -9,10 +9,11 @@ Description
 ``pelican-import`` is a command-line tool for converting articles from other
 software to reStructuredText or Markdown. The supported import formats are:
 
-- WordPress XML export
+- Blogger XML export
 - Dotclear export
 - Posterous API
 - Tumblr API
+- WordPress XML export
 - RSS/Atom feed
 
 The conversion from HTML to reStructuredText or Markdown relies on `Pandoc`_.
@@ -40,9 +41,9 @@ Usage
 
 ::
 
-    pelican-import [-h] [--wpfile] [--dotclear] [--posterous] [--tumblr] [--feed] [-o OUTPUT]
-                   [-m MARKUP] [--dir-cat] [--dir-page] [--strip-raw] [--disable-slugs]
-                   [-e EMAIL] [-p PASSWORD] [-b BLOGNAME]
+    pelican-import [-h] [--blogger] [--dotclear] [--posterous] [--tumblr] [--wpfile] [--feed]
+                   [-o OUTPUT] [-m MARKUP] [--dir-cat] [--dir-page] [--strip-raw] [--wp-custpost]
+                   [--wp-attach] [--disable-slugs] [-e EMAIL] [-p PASSWORD] [-b BLOGNAME]
                    input|api_token|api_key
 
 Positional arguments
@@ -57,10 +58,11 @@ Optional arguments
 ------------------
 
   -h, --help            Show this help message and exit
-  --wpfile              WordPress XML export (default: False)
+  --blogger             Blogger XML export (default: False)
   --dotclear            Dotclear export (default: False)
   --posterous           Posterous API (default: False)
   --tumblr              Tumblr API (default: False)
+  --wpfile              WordPress XML export (default: False)
   --feed                Feed to parse (default: False)
   -o OUTPUT, --output OUTPUT
                         Output path (default: content)
@@ -70,11 +72,24 @@ Optional arguments
   --dir-cat             Put files in directories with categories name
                         (default: False)
   --dir-page            Put files recognised as pages in "pages/" sub-
-                          directory (wordpress import only) (default: False)
-  --filter-author       Import only post from the specified author.
+                          directory (blogger and wordpress import only)
+                          (default: False)
+  --filter-author       Import only post from the specified author
   --strip-raw           Strip raw HTML code that can't be converted to markup
                         such as flash embeds or iframes (wordpress import
                         only) (default: False)
+  --wp-custpost         Put wordpress custom post types in directories. If
+                        used with --dir-cat option directories will be created
+                        as "/post_type/category/" (wordpress import only)
+  --wp-attach           Download files uploaded to wordpress as attachments.
+                        Files will be added to posts as a list in the post
+                        header and links to the files within the post will be
+                        updated. All files will be downloaded, even if they
+                        aren't associated with a post. Files will be downloaded
+                        with their original path inside the output directory,
+                        e.g. "output/wp-uploads/date/postname/file.jpg".
+                        (wordpress import only) (requires an internet
+                        connection)
   --disable-slugs       Disable storing slugs from imported posts within
                         output. With this disabled, your Pelican URLs may not
                         be consistent with your original posts. (default:
@@ -90,9 +105,9 @@ Optional arguments
 Examples
 ========
 
-For WordPress::
+For Blogger::
 
-    $ pelican-import --wpfile -o ~/output ~/posts.xml
+    $ pelican-import --blogger -o ~/output ~/posts.xml
 
 For Dotclear::
 
@@ -106,10 +121,14 @@ For Tumblr::
 
     $ pelican-import --tumblr -o ~/output --blogname=<blogname> <api_token>
 
+For WordPress::
+
+    $ pelican-import --wpfile -o ~/output ~/posts.xml
+
 Tests
 =====
 
 To test the module, one can use sample files:
 
-- for WordPress: http://wpcandy.com/made/the-sample-post-collection
+- for WordPress: http://www.wpbeginner.com/wp-themes/how-to-add-dummy-content-for-theme-development-in-wordpress/
 - for Dotclear: http://media.dotaddict.org/tda/downloads/lorem-backup.txt

docs/install.rst

@@ -101,10 +101,9 @@ can optionally add yourself if you plan to create non-chronological content)::
 
     yourproject/
     ├── content
-    │   └── (pages)
+    │   └── (pages)
     ├── output
-    ├── develop_server.sh
-    ├── fabfile.py
+    ├── tasks.py
     ├── Makefile
     ├── pelicanconf.py       # Main settings file
     └── publishconf.py       # Settings to use when ready to publish

docs/plugins.rst

@@ -108,6 +108,7 @@ page_generator_preread              page_generator                 invoked befor
                                                                    use if code needs to do something before every page is parsed.
 page_generator_init                 page_generator                 invoked in the PagesGenerator.__init__
 page_generator_finalized            page_generator                 invoked at the end of PagesGenerator.generate_context
+page_generator_write_page           page_generator, content        invoked before writing each page, the page is passed as content
 page_writer_finalized               page_generator, writer         invoked after all pages have been written, but before the page generator
                                                                    is closed.
 static_generator_context            static_generator, metadata
@@ -215,6 +216,7 @@ Adding a new generator is also really easy. You might want to have a look at
         # define a new generator here if you need to
         return MyGenerator
 
-    signals.get_generators.connect(get_generators)
+    def register():
+        signals.get_generators.connect(get_generators)
 
 .. _pelican-plugins bug #314: https://github.com/getpelican/pelican-plugins/issues/314

docs/publish.rst

@@ -25,8 +25,8 @@ argument, like so::
     pelican --write-selected output/posts/my-post-title.html
 
 Note that you must specify the path to the generated *output* file — not the
-source content. To determine the output file path, use the ``--debug`` flag to
-determine the correct file name and location. If desired, ``--write-selected``
+source content. To determine the output file name and location, use the
+``--debug`` flag. If desired, ``--write-selected``
 can take a comma-separated list of paths or can be configured as a setting.
 (See: :ref:`writing_only_selected_content`)
 
@@ -101,7 +101,7 @@ While the ``pelican`` command is the canonical way to generate your site,
 automation tools can be used to streamline the generation and publication
 flow. One of the questions asked during the ``pelican-quickstart`` process
 pertains to whether you want to automate site generation and publication.
-If you answered "yes" to that question, a ``fabfile.py`` and
+If you answered "yes" to that question, a ``tasks.py`` and
 ``Makefile`` will be generated in the root of your project. These files,
 pre-populated with certain information gleaned from other answers provided
 during the ``pelican-quickstart`` process, are meant as a starting point and
@@ -113,55 +113,43 @@ files can deleted at any time and will not affect usage of the canonical
 Following are automation tools that "wrap" the ``pelican`` command and can
 simplify the process of generating, previewing, and uploading your site.
 
-Fabric
+Invoke
 ------
 
-The advantage of Fabric_ is that it is written in Python and thus can be used
+The advantage of Invoke_ is that it is written in Python and thus can be used
 in a wide range of environments. The downside is that it must be installed
-separately. Use the following command to install Fabric, prefixing with
+separately. Use the following command to install Invoke, prefixing with
 ``sudo`` if your environment requires it::
 
-    pip install Fabric
+    pip install invoke
 
-.. note:: Installing PyCrypto on Windows
-
-    Fabric depends upon PyCrypto_, which is tricky to install
-    if your system doesn't have a C compiler.
-    For Windows users, before installing Fabric, use
-    ``easy_install http://www.voidspace.org.uk/downloads/pycrypto26/pycrypto-2.6.win32-py2.7.exe``
-    per this `StackOverflow suggestion <http://stackoverflow.com/a/11405769/6364>`_
-    You're more likely to have success
-    with the Win32 versions of Python 2.7 and PyCrypto,
-    than with the Win64—\
-    even if your operating system is a 64-bit version of Windows.
-
-Take a moment to open the ``fabfile.py`` file that was generated in your
+Take a moment to open the ``tasks.py`` file that was generated in your
 project root. You will see a number of commands, any one of which can be
 renamed, removed, and/or customized to your liking. Using the out-of-the-box
 configuration, you can generate your site via::
 
-    fab build
+    invoke build
 
 If you'd prefer to have Pelican automatically regenerate your site every time a
 change is detected (which is handy when testing locally), use the following
 command instead::
 
-    fab regenerate
+    invoke regenerate
 
 To serve the generated site so it can be previewed in your browser at
 http://localhost:8000/::
 
-    fab serve
+    invoke serve
 
 If during the ``pelican-quickstart`` process you answered "yes" when asked
 whether you want to upload your site via SSH, you can use the following command
 to publish your site via rsync over SSH::
 
-    fab publish
+    invoke publish
 
 These are just a few of the commands available by default, so feel free to
-explore ``fabfile.py`` and see what other commands are available. More
-importantly, don't hesitate to customize ``fabfile.py`` to suit your specific
+explore ``tasks.py`` and see what other commands are available. More
+importantly, don't hesitate to customize ``tasks.py`` to suit your specific
 needs and preferences.
 
 Make
@@ -201,10 +189,7 @@ separate terminal sessions, but you can run both at once via::
     make devserver
 
 The above command will simultaneously run Pelican in regeneration mode as well
-as serve the output at http://localhost:8000. Once you are done testing your
-changes, you should stop the development server via::
-
-    ./develop_server.sh stop
+as serve the output at http://localhost:8000.
 
 When you're ready to publish your site, you can upload it via the method(s) you
 chose during the ``pelican-quickstart`` questionnaire. For this example, we'll
@@ -219,5 +204,4 @@ That's it! Your site should now be live.
 executables, such as ``python3``, you can set the ``PY`` and ``PELICAN``
 environment variables, respectively, to override the default executable names.)
 
-.. _Fabric: http://fabfile.org/
-.. _PyCrypto: http://pycrypto.org
+.. _Invoke: http://www.pyinvoke.org

docs/quickstart.rst

@@ -61,11 +61,10 @@ ignored for now.)
 Preview your site
 -----------------
 
-Open a new terminal session and run the following commands to switch to your
-``output`` directory and launch Pelican's web server::
+Open a new terminal session, navigate to your site directory and run the
+following command to launch Pelican's web server::
 
-    cd ~/projects/yoursite/output
-    python -m pelican.server
+    pelican --listen
 
 Preview your site by navigating to http://localhost:8000/ in your browser.
 

docs/settings.rst

@@ -21,6 +21,12 @@ Settings are configured in the form of a Python module (a file). There is an
 <https://github.com/getpelican/pelican/raw/master/samples/pelican.conf.py>`_
 available for reference.
 
+To see a list of current settings in your environment, including both default
+and any customized values, run the following command (append one or more
+specific setting names as arguments to see values for those settings only)::
+
+    pelican --print-settings
+
 All the setting identifiers must be set in all-caps, otherwise they will not be
 processed. Setting values that are numbers (5, 20, etc.), booleans (True,
 False, None, etc.), dictionaries, or tuples should *not* be enclosed in
@@ -28,10 +34,9 @@ quotation marks. All other values (i.e., strings) *must* be enclosed in
 quotation marks.
 
 Unless otherwise specified, settings that refer to paths can be either absolute
-or relative to the configuration file.
-
-The settings you define in the configuration file will be passed to the
-templates, which allows you to use your settings to add site-wide content.
+or relative to the configuration file. The settings you define in the
+configuration file will be passed to the templates, which allows you to use your
+settings to add site-wide content.
 
 Here is a list of settings for Pelican:
 
@@ -131,7 +136,7 @@ Basic settings
 
    Extra configuration settings for the Markdown processor. Refer to the Python
    Markdown documentation's `Options section
-   <http://pythonhosted.org/Markdown/reference.html#markdown>`_ for a complete
+   <https://python-markdown.github.io/reference/#markdown>`_ for a complete
    list of supported options. The ``extensions`` option will be automatically
    computed from the ``extension_configs`` option.
 
@@ -328,6 +333,15 @@ Basic settings
    A list of metadata fields containing reST/Markdown content to be parsed and
    translated to HTML.
 
+.. data:: PORT = 8000
+
+   The TCP port to serve content from the output folder via HTTP when pelican
+   is run with --listen
+
+.. data:: BIND = ''
+
+   The IP to which to bind the HTTP server.
+
 
 URL settings
 ============
@@ -436,6 +450,24 @@ respectively.
 
    The location we will save the page which doesn't use the default language.
 
+.. data:: DRAFT_PAGE_URL = 'drafts/pages/{slug}.html'
+
+   The URL used to link to a page draft.
+
+.. data:: DRAFT_PAGE_SAVE_AS = 'drafts/pages/{slug}.html'
+
+   The actual location a page draft is saved at.
+
+.. data:: DRAFT_PAGE_LANG_URL = 'drafts/pages/{slug}-{lang}.html'
+
+   The URL used to link to a page draft which doesn't use the default
+   language.
+
+.. data:: DRAFT_PAGE_LANG_SAVE_AS = 'drafts/pages/{slug}-{lang}.html'
+
+   The actual location a page draft which doesn't use the default language is
+   saved at.
+
 .. data:: CATEGORY_URL = 'category/{slug}.html'
 
    The URL to use for a category.
@@ -464,34 +496,28 @@ respectively.
 
    The location to save per-year archives of your posts.
 
+.. data:: YEAR_ARCHIVE_URL = ''
+
+   The URL to use for per-year archives of your posts. Used only if you have
+   the ``{url}`` placeholder in ``PAGINATION_PATTERNS``.
+
 .. data:: MONTH_ARCHIVE_SAVE_AS = ''
 
    The location to save per-month archives of your posts.
 
+.. data:: MONTH_ARCHIVE_URL = ''
+
+   The URL to use for per-month archives of your posts. Used only if you have
+   the ``{url}`` placeholder in ``PAGINATION_PATTERNS``.
+
 .. data:: DAY_ARCHIVE_SAVE_AS = ''
 
    The location to save per-day archives of your posts.
 
-.. data:: SLUG_SUBSTITUTIONS = ()
+.. data:: DAY_ARCHIVE_URL = ''
 
-   Substitutions to make prior to stripping out non-alphanumerics when
-   generating slugs. Specified as a list of 3-tuples of ``(from, to, skip)``
-   which are applied in order. ``skip`` is a boolean indicating whether or not
-   to skip replacement of non-alphanumeric characters.  Useful for backward
-   compatibility with existing URLs.
-
-.. data:: AUTHOR_SUBSTITUTIONS = ()
-
-   Substitutions for authors. ``SLUG_SUBSTITUTIONS`` is not taken into account
-   here!
-
-.. data:: CATEGORY_SUBSTITUTIONS = ()
-
-   Added to ``SLUG_SUBSTITUTIONS`` for categories.
-
-.. data:: TAG_SUBSTITUTIONS = ()
-
-   Added to ``SLUG_SUBSTITUTIONS`` for tags.
+   The URL to use for per-day archives of your posts. Used only if you have the
+   ``{url}`` placeholder in ``PAGINATION_PATTERNS``.
 
 .. note::
 
@@ -500,24 +526,6 @@ respectively.
     set the corresponding ``*_SAVE_AS`` setting to ``''`` to prevent the
     relevant page from being generated.
 
-.. note::
-
-    Substitutions are applied in order with the side effect that keeping
-    non-alphanum characters applies to the whole string when a replacement
-    is made.
-
-    For example if you have the following setting::
-
-       SLUG_SUBSTITUTIONS = (('C++', 'cpp'), ('keep dot', 'keep.dot', True))
-
-    the string ``Keep Dot`` will be converted to ``keep.dot``, however
-    ``C++ will keep dot`` will be converted to ``cpp will keep.dot`` instead
-    of ``cpp-will-keep.dot``!
-
-    If you want to keep non-alphanum characters only for tags or categories
-    but not other slugs then configure ``TAG_SUBSTITUTIONS`` and
-    ``CATEGORY_SUBSTITUTIONS`` respectively!
-
 Pelican can optionally create per-year, per-month, and per-day archives of your
 posts. These secondary archives are disabled by default but are automatically
 enabled if you supply format strings for their respective ``_SAVE_AS`` settings.
@@ -579,6 +587,33 @@ URLs for direct template pages are theme-dependent. Some themes use
 corresponding ``*_URL`` setting as string, while others hard-code them:
 ``'archives.html'``, ``'authors.html'``, ``'categories.html'``, ``'tags.html'``.
 
+.. data:: SLUG_REGEX_SUBSTITUTIONS = [
+        (r'[^\w\s-]', ''),  # remove non-alphabetical/whitespace/'-' chars
+        (r'(?u)\A\s*', ''),  # strip leading whitespace
+        (r'(?u)\s*\Z', ''),  # strip trailing whitespace
+        (r'[-\s]+', '-'),  # reduce multiple whitespace or '-' to single '-'
+    ]
+
+   Regex substitutions to make when generating slugs of articles and pages.
+   Specified as a list of pairs of ``(from, to)`` which are applied in order,
+   ignoring case. The default substitutions have the effect of removing
+   non-alphanumeric characters and converting internal whitespace to dashes.
+   Apart from these substitutions, slugs are always converted to lowercase
+   ascii characters and leading and trailing whitespace is stripped. Useful for
+   backward compatibility with existing URLs.
+
+.. data:: AUTHOR_REGEX_SUBSTITUTIONS = SLUG_REGEX_SUBSTITUTIONS
+
+   Regex substitutions for author slugs. Defaults to ``SLUG_REGEX_SUBSTITUTIONS``.
+
+.. data:: CATEGORY_REGEX_SUBSTITUTIONS = SLUG_REGEX_SUBSTITUTIONS
+
+   Regex substitutions for category slugs. Defaults to ``SLUG_REGEX_SUBSTITUTIONS``.
+
+.. data:: TAG_REGEX_SUBSTITUTIONS = SLUG_REGEX_SUBSTITUTIONS
+
+   Regex substitutions for tag slugs. Defaults to ``SLUG_REGEX_SUBSTITUTIONS``.
+
 Time and Date
 =============
 
@@ -699,6 +734,10 @@ Template pages
                          'src/resume.html': 'dest/resume.html',
                          'src/contact.html': 'dest/contact.html'}
 
+.. data:: TEMPLATE_EXTENSION = ['.html']
+
+   The extensions to use when looking up template files from template names.
+
 .. data:: DIRECT_TEMPLATES = ['index', 'categories', 'authors', 'archives']
 
    List of templates that are used directly to render content. Typically direct
@@ -706,15 +745,8 @@ Template pages
    tags and category index pages). If the tag and category collections are not
    needed, set ``DIRECT_TEMPLATES = ['index', 'archives']``
 
-.. data:: PAGINATED_DIRECT_TEMPLATES = ['index']
-
-   Provides the direct templates that should be paginated.
-
-.. data:: EXTRA_TEMPLATES_PATHS = []
-
-   A list of paths you want Jinja2 to search for templates.  Can be used to
-   separate templates from the theme.  Example: projects, resume, profile ...
-   These templates need to use ``DIRECT_TEMPLATES`` setting.
+   ``DIRECT_TEMPLATES`` are searched for over paths maintained in
+   ``THEME_TEMPLATES_OVERRIDES``.
 
 
 Metadata
@@ -811,46 +843,95 @@ the ``TAG_FEED_ATOM`` and ``TAG_FEED_RSS`` settings:
 
 .. data:: FEED_ATOM = None, i.e. no Atom feed
 
-   Relative URL to output the Atom feed.
+   The location to save the Atom feed.
+
+.. data:: FEED_ATOM_URL = None
+
+   Relative URL of the Atom feed. If not set, ``FEED_ATOM`` is used both for
+   save location and URL.
 
 .. data:: FEED_RSS = None, i.e. no RSS
 
-   Relative URL to output the RSS feed.
+   The location to save the RSS feed.
+
+.. data:: FEED_RSS_URL = None
+
+   Relative URL of the RSS feed. If not set, ``FEED_RSS`` is used both for save
+   location and URL.
 
 .. data:: FEED_ALL_ATOM = 'feeds/all.atom.xml'
 
-   Relative URL to output the all-posts Atom feed: this feed will contain all
+   The location to save the all-posts Atom feed: this feed will contain all
    posts regardless of their language.
 
+.. data:: FEED_ALL_ATOM_URL = None
+
+   Relative URL of the all-posts Atom feed. If not set, ``FEED_ALL_ATOM`` is
+   used both for save location and URL.
+
 .. data:: FEED_ALL_RSS = None, i.e. no all-posts RSS
 
-   Relative URL to output the all-posts RSS feed: this feed will contain all
+   The location to save the the all-posts RSS feed: this feed will contain all
    posts regardless of their language.
 
+.. data:: FEED_ALL_RSS_URL = None
+
+   Relative URL of the all-posts RSS feed. If not set, ``FEED_ALL_RSS`` is used
+   both for save location and URL.
+
 .. data:: CATEGORY_FEED_ATOM = 'feeds/%s.atom.xml'
 
-   Where to put the category Atom feeds. [2]_
+   The location to save the category Atom feeds. [2]_
+
+.. data:: CATEGORY_FEED_ATOM_URL = None
+
+   Relative URL of the category Atom feeds, including the ``%s`` placeholder.
+   [2]_ If not set, ``CATEGORY_FEED_ATOM`` is used both for save location and
+   URL.
 
 .. data:: CATEGORY_FEED_RSS = None, i.e. no RSS
 
-   Where to put the category RSS feeds.
+   The location to save the category RSS feeds, including the ``%s``
+   placeholder. [2]_
+
+.. data:: CATEGORY_FEED_RSS_URL = None
+
+   Relative URL of the category RSS feeds, including the ``%s`` placeholder.
+   [2]_ If not set, ``CATEGORY_FEED_RSS`` is used both for save location and
+   URL.
 
 .. data:: AUTHOR_FEED_ATOM = 'feeds/%s.atom.xml'
 
-   Where to put the author Atom feeds. [2]_
+   The location to save the author Atom feeds. [2]_
+
+.. data:: AUTHOR_FEED_ATOM_URL = None
+
+   Relative URL of the author Atom feeds, including the ``%s`` placeholder.
+   [2]_ If not set, ``AUTHOR_FEED_ATOM`` is used both for save location and
+   URL.
 
 .. data:: AUTHOR_FEED_RSS = 'feeds/%s.rss.xml'
 
-   Where to put the author RSS feeds. [2]_
+   The location to save the author RSS feeds. [2]_
+
+.. data:: AUTHOR_FEED_RSS_URL = None
+
+   Relative URL of the author RSS feeds, including the ``%s`` placeholder. [2]_
+   If not set, ``AUTHOR_FEED_RSS`` is used both for save location and URL.
 
 .. data:: TAG_FEED_ATOM = None, i.e. no tag feed
 
-   Relative URL to output the tag Atom feed. It should be defined using a "%s"
-   match in the tag name.
+   The location to save the tag Atom feed, including the ``%s`` placeholder.
+   [2]_
+
+.. data:: TAG_FEED_ATOM_URL = None
+
+   Relative URL of the tag Atom feed, including the ``%s`` placeholder. [2]_
 
 .. data:: TAG_FEED_RSS = None, i.e. no RSS tag feed
 
-   Relative URL to output the tag RSS feed
+   Relative URL to output the tag RSS feed, including the ``%s`` placeholder.
+   If not set, ``TAG_FEED_RSS`` is used both for save location and URL.
 
 .. data:: FEED_MAX_ITEMS
 
@@ -865,7 +946,7 @@ the ``TAG_FEED_ATOM`` and ``TAG_FEED_RSS`` settings:
 
 If you don't want to generate some or any of these feeds, set the above variables to ``None``.
 
-.. [2] %s is the name of the category.
+.. [2] ``%s`` is replaced by name of the category / author / tag.
 
 
 FeedBurner
@@ -908,7 +989,15 @@ You can use the following settings to configure the pagination.
    The maximum number of articles to include on a page, not including orphans.
    False to disable pagination.
 
-.. data:: PAGINATION_PATTERNS
+.. data:: PAGINATED_TEMPLATES = {'index': None, 'tag': None, 'category': None, 'author': None}
+
+   The templates to use pagination with, and the number of articles to include
+   on a page. If this value is ``None``, it defaults to ``DEFAULT_PAGINATION``.
+
+.. data:: PAGINATION_PATTERNS = (
+      (1, '{name}{extension}', '{name}{extension}'),
+      (2, '{name}{number}{extension}', '{name}{number}{extension}'),
+  )
 
    A set of patterns that are used to determine advanced pagination output.
 
@@ -916,25 +1005,28 @@ You can use the following settings to configure the pagination.
 Using Pagination Patterns
 -------------------------
 
-The ``PAGINATION_PATTERNS`` setting can be used to configure where
-subsequent pages are created. The setting is a sequence of three
-element tuples, where each tuple consists of::
+By default, pages subsequent to ``.../foo.html`` are created as
+``.../foo2.html``, etc. The ``PAGINATION_PATTERNS`` setting can be used to
+change this. It takes a sequence of triples, where each triple consists of::
 
-  (minimum page, URL setting, SAVE_AS setting,)
+  (minimum_page, page_url, page_save_as,)
 
-For example, if you wanted the first page to just be ``/``, and the
-second (and subsequent) pages to be ``/page/2/``, you would set
-``PAGINATION_PATTERNS`` as follows::
+For ``page_url`` and ``page_save_as``, you may use a number of variables.
+``{url}`` and ``{save_as}`` correspond respectively to the ``*_URL`` and
+``*_SAVE_AS`` values of the corresponding page type (e.g. ``ARTICLE_SAVE_AS``).
+If ``{save_as} == foo/bar.html``, then ``{name} == foo/bar`` and
+``{extension} == .html``. ``{base_name}`` equals ``{name}`` except that it
+strips trailing ``/index`` if present. ``{number}`` equals the page number.
+
+For example, if you want to leave the first page unchanged, but place
+subsequent pages at ``.../page/2/`` etc, you could set ``PAGINATION_PATTERNS``
+as follows::
 
   PAGINATION_PATTERNS = (
-      (1, '{base_name}/', '{base_name}/index.html'),
+      (1, '{url}', '{save_as}`,
       (2, '{base_name}/page/{number}/', '{base_name}/page/{number}/index.html'),
   )
 
-This would cause the first page to be written to
-``{base_name}/index.html``, and subsequent ones would be written into
-``page/{number}`` directories.
-
 
 Translations
 ============
@@ -946,14 +1038,38 @@ more information.
 
    The default language to use.
 
+.. data:: ARTICLE_TRANSLATION_ID = 'slug'
+
+   The metadata attribute(s) used to identify which articles are translations
+   of one another. May be a string or a collection of strings. Set to ``None``
+   or ``False`` to disable the identification of translations.
+
+.. data:: PAGE_TRANSLATION_ID = 'slug'
+
+   The metadata attribute(s) used to identify which pages are translations
+   of one another. May be a string or a collection of strings. Set to ``None``
+   or ``False`` to disable the identification of translations.
+
 .. data:: TRANSLATION_FEED_ATOM = 'feeds/all-%s.atom.xml'
 
-   Where to put the Atom feed for translations. [3]_
+   The location to save the Atom feed for translations. [3]_
+
+.. data:: TRANSLATION_FEED_ATOM_URL = None
+
+   Relative URL of the Atom feed for translations, including the ``%s``
+   placeholder. [3]_ If not set, ``TRANSLATION_FEED_ATOM`` is used both for
+   save location and URL.
 
 .. data:: TRANSLATION_FEED_RSS = None, i.e. no RSS
 
    Where to put the RSS feed for translations.
 
+.. data:: TRANSLATION_FEED_RSS_URL = None
+
+   Relative URL of the RSS feed for translations, including the ``%s``
+   placeholder. [3]_ If not set, ``TRANSLATION_FEED_RSS`` is used both for save
+   location and URL.
+
 .. [3] %s is the language
 
 
@@ -981,7 +1097,7 @@ Ordering content
 
 .. data:: PAGE_ORDER_BY = 'basename'
 
-   Defines how the pages (``PAGES`` variable in the template) are sorted.
+   Defines how the pages (``pages`` variable in the template) are sorted.
    Options are same as ``ARTICLE_ORDER_BY``.  The default value, ``'basename'``
    will sort pages by their basename.
 
@@ -1011,6 +1127,21 @@ However, here are the settings that are related to themes.
    with the same names are included in the paths defined in this settings, they
    will be progressively overwritten.
 
+.. data:: THEME_TEMPLATES_OVERRIDES = []
+
+   A list of paths you want Jinja2 to search for templates before searching the
+   theme's ``templates/`` directory.  Allows for overriding individual theme
+   template files without having to fork an existing theme.  Jinja2 searches in
+   the following order: files in ``THEME_TEMPLATES_OVERRIDES`` first, then the
+   theme's ``templates/``.
+
+   You can also extend templates from the theme using the ``{% extends %}``
+   directive utilizing the ``!theme`` prefix as shown in the following example:
+
+   .. parsed-literal::
+
+      {% extends '!theme/article.html' %}
+
 .. data:: CSS_FILE = 'main.css'
 
    Specify the CSS file you want to load.
@@ -1130,15 +1261,17 @@ Simply populate the list with the log messages you want to hide, and they will
 be filtered out.
 
 For example::
-
-   [(logging.WARN, 'TAG_SAVE_AS is set to False')]
+    
+   import logging
+   LOG_FILTER = [(logging.WARN, 'TAG_SAVE_AS is set to False')]
 
 It is possible to filter out messages by a template. Check out source code to
 obtain a template.
 
 For example::
 
-   [(logging.WARN, 'Empty alt attribute for image %s in %s')]
+   import logging
+   LOG_FILTER = [(logging.WARN, 'Empty alt attribute for image %s in %s')]
 
 .. Warning::
    Silencing messages by templates is a dangerous feature. It is possible to

docs/themes.rst

@@ -86,6 +86,7 @@ categories      A list of (category, articles) tuples, containing
                 all the categories and corresponding articles (values)
 pages           The list of pages
 hidden_pages    The list of hidden pages
+draft_pages     The list of draft pages
 =============   ===================================================
 
 
@@ -154,8 +155,8 @@ author.html
 
 This template will be processed for each of the existing authors, with
 output generated according to the ``AUTHOR_SAVE_AS`` setting (`Default:`
-``author/{author_name}.html``). If pagination is active, subsequent pages will by
-default reside at ``author/{author_name}{number}.html``.
+``author/{slug}.html``). If pagination is active, subsequent pages will by
+default reside at ``author/{slug}{number}.html``.
 
 ======================  ===================================================
 Variable                Description
@@ -188,8 +189,8 @@ category.html
 
 This template will be processed for each of the existing categories, with
 output generated according to the ``CATEGORY_SAVE_AS`` setting (`Default:`
-``category/{category_name}.html``). If pagination is active, subsequent pages will by
-default reside at ``category/{category_name}{number}.html``.
+``category/{slug}.html``). If pagination is active, subsequent pages will by
+default reside at ``category/{slug}{number}.html``.
 
 ======================  ===================================================
 Variable                Description
@@ -222,7 +223,7 @@ article.html
 
 This template will be processed for each article, with
 output generated according to the ``ARTICLE_SAVE_AS`` setting (`Default:`
-``{article_name}.html``). The following variables are available when
+``{slug}.html``). The following variables are available when
 rendering.
 
 =============   ===================================================
@@ -263,7 +264,7 @@ page.html
 
 This template will be processed for each page, with
 output generated according to the ``PAGE_SAVE_AS`` setting (`Default:`
-``pages/{page_name}.html``). The following variables are available when
+``pages/{slug}.html``). The following variables are available when
 rendering.
 
 =============   ===================================================
@@ -279,8 +280,8 @@ tag.html
 
 This template will be processed for each tag, with
 output generated according to the ``TAG_SAVE_AS`` setting (`Default:`
-``tag/{tag_name}.html``). If pagination is active, subsequent pages will by
-default reside at ``tag/{tag_name}{number}.html``.
+``tag/{slug}.html``). If pagination is active, subsequent pages will by
+default reside at ``tag/{slug}{number}.html``.
 
 ======================  ===================================================
 Variable                Description
@@ -423,7 +424,7 @@ metadata                Page header metadata `dict`.
 save_as                 Location to save the page.
 slug                    Page slug.
 source_path             Full system path of the page source file.
-status                  The page status, can be any of 'published' or
+status                  The page status, can be any of 'published', 'hidden' or
                         'draft'.
 summary                 Rendered summary content.
 tags                    List of :ref:`Tag <object-author_cat_tag>`

docs/tips.rst

@@ -58,7 +58,7 @@ repository, and if you want to publish that Pelican site in the form of Project
 Pages to this repository, you can then use the following::
 
     $ pelican content -o output -s pelicanconf.py
-    $ ghp-import output
+    $ ghp-import output -b gh-pages
     $ git push origin gh-pages
 
 The ``ghp-import output`` command updates the local ``gh-pages`` branch with
@@ -67,7 +67,7 @@ already exist). The ``git push origin gh-pages`` command updates the remote
 ``gh-pages`` branch, effectively publishing the Pelican site.
 
 .. note::
-    The ``github`` target of the Makefile (and the ``gh_pages`` task of the Fabfile)
+    The ``github`` target of the Makefile (and the ``gh_pages`` task of ``tasks.py``)
     created by the ``pelican-quickstart`` command
     publishes the Pelican site as Project Pages, as described above.
 
@@ -87,7 +87,7 @@ your ``<username>.github.io`` repository on GitHub.
 Again, you can take advantage of ``ghp-import``::
 
     $ pelican content -o output -s pelicanconf.py
-    $ ghp-import output
+    $ ghp-import output -b gh-pages
     $ git push git@github.com:elemoine/elemoine.github.io.git gh-pages:master
 
 The ``git push`` command pushes the local ``gh-pages`` branch (freshly updated
@@ -98,6 +98,18 @@ by the ``ghp-import`` command) to the ``elemoine.github.io`` repository's
 
     To publish your Pelican site as User Pages, feel free to adjust the
     ``github`` target of the Makefile.
+    
+Another option for publishing to User Pages is to generate the output files in the root directory of the project.
+
+For example, your main project folder is ``<username>.github.io`` and you can create the Pelican project in a subdirectory called ``Pelican``. Then from inside the ``Pelican`` folder you can run::
+    
+    $ pelican content -o .. -s pelicanconf.py
+
+Now you can push the whole project ``<username>.github.io`` to the master branch of your GitHub repository::
+    
+    $ git push origin master
+    
+(assuming origin is set to your remote repository).
 
 Custom 404 Pages
 ----------------
@@ -147,3 +159,25 @@ embed videos in the markup. You can use `reST video directive
 <https://gist.github.com/dbrgn/2922648>`_ for reST or `mdx_video plugin
 <https://github.com/italomaia/mdx-video>`_ for Markdown.
 
+
+Develop Locally Using SSL
+==================================
+
+Here's how you can set up your local pelican server to support SSL.
+
+First, create a self-signed certificate and key using ``openssl`` (this creates ``cert.pem`` and ``key.pem``)::
+
+    $ openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes
+
+And use this command to launch the server (the server starts within your ``output`` directory)::
+
+    python -m pelican.server 8443 --key=../key.pem --cert=../cert.pem
+
+If you are using ``develop-server.sh``,  add this to the top::
+
+    CERT="$BASEDIR/cert.pem"
+    KEY="$BASEDIR/key.pem"
+
+and modify the ``pelican.server`` line as follows::
+
+    $PY -m pelican.server $port --ssl --cert="$CERT" --key="$KEY" &