git merge upstream/master

.gitattributes

@@ -0,0 +1,2 @@
+# Auto detect text files and perform LF normalization
+* text=auto

.gitignore

@@ -9,3 +9,7 @@ build
 dist
 tags
 .tox
+.coverage
+htmlcov
+six-*.egg/
+*.orig

.travis.yml

@@ -1,14 +1,20 @@
 language: python
 python:
-    - "2.6"
     - "2.7"
+    - "3.2"
+before_install:
+ - sudo apt-get update -qq
+ - sudo apt-get install -qq ruby-sass
 install:
-    - pip install nose unittest2 mock --use-mirrors
+    - pip install nose mock --use-mirrors
+    - if [[ $TRAVIS_PYTHON_VERSION == '2.7' ]]; then pip install --use-mirrors unittest2; else pip install --use-mirrors unittest2py3k; fi
     - pip install . --use-mirrors
     - pip install Markdown
+    - pip install webassets
+    - pip install cssmin
 script: nosetests -s tests
 notifications:
     irc:
-        channels: 
+        channels:
             - "irc.freenode.org#pelican"
         on_success: change

CHANGELOG

@@ -1,108 +0,0 @@
-3.0 - XX/XX/XXXX
-
-* Refactored the way URL are handled.
-* Improved the english documentation
-* Fixed packaging using setuptools entrypoints
-* Added typogrify support
-* Added a way to disable feed generation
-* Added support for DIRECT_TEMPLATES
-* Allow multiple extensions for content files
-* Added less support
-* Improved the import script
-* Fixed a bunch of bugs :-)
-* Added functional tests
-* Rsync support in the generated Makefile
-* Improved feed support (easily pluggable with feedburner for instance)
-
-2.8
-
-* dotclear importer
-* Allow the usage of markdown extensions
-* Themes are now easily extensible
-* Don't output pagination information if there is only one page.
-* Add a page per author, with all their articles
-* Improved the test suite
-* Made the themes more easy to extend
-* Removed Skribit support
-* Added a "pelican-quickstart" script
-* Fixed timezone-related issues
-* Add some scripts for windows support
-* Date can be specified in seconds
-* Never fail when generating posts (skip and continue)
-* Allow the use of future dates
-* Support having different timezones per languages.
-* Enhanced the documentation
-
-2.7
-
-* Uses logging rather than echoing to stdout
-* Support custom jinja filters
-* Compatibility with python 2.5
-* Add a theme manager
-* Packaged for debian
-* Add draft support
-
-2.6
-
-* changes in the output directory structure
-* makes templates easier to work with / create
-* Add RSS support (was only atom previously)
-* Add tag support for the feeds
-* Enhance the documentation
-* Add another theme (brownstone)
-* Add translations
-* Add a way to use "cleaner urls" with a rewrite url module (or equivalent)
-* Add a tag cloud
-* Add an autoreloading feature: the blog is automatically regenerated each time a modification is detected
-* Translate the documentation in french
-* import a blog from an rss feed
-* Pagination support
-* Add skribit support
-
-2.5
-
-* import from wordpress
-* add some new themes (martyalchin / wide-notmyidea)
-* first bug report !
-* linkedin support
-* added a FAQ
-* google analytics support
-* twitter support
-* use relative urls not static ones
-
-2.4
-
-* minor themes changes
-* add disqus support (so we have comments)
-* another code refactoring
-* add config settings about pages
-* blog entries can also be generated in pdf
-
-2.3
-
-* markdown support
-
-2.2
-
-* Prettify output
-* Manages static pages as well
-
-2.1
-
-* Put the notmyidea theme by default
-
-2.0
-
-* Refactoring to be more extensible
-* Change into the setting variables
-
-1.2
-
-* Add a debug option
-* Add feeds per category
-* Use filsystem to get dates if no metadata provided
-* Add pygment support
-
-1.1:
-
-* first working version

MANIFEST.in

@@ -1,4 +1,6 @@
 include *.rst
 global-include *.py
 recursive-include pelican *.html *.css *png *.in
-include LICENSE
+include LICENSE THANKS docs/changelog.rst
+recursive-include tests *
+recursive-exclude tests *.pyc

README.rst

@@ -1,12 +1,14 @@
 Pelican
-#######
+=======
 
-.. image:: https://secure.travis-ci.org/ametaireau/pelican.png?branch=master
+.. image:: https://secure.travis-ci.org/getpelican/pelican.png?branch=master
+   :target: http://travis-ci.org/#!/getpelican/pelican
+   :alt: Travis-ci: continuous integration status.
 
-Pelican is a simple weblog generator, written in `Python <http://www.python.org/>`_.
+Pelican is a static site generator, written in Python_.
 
-* Write your weblog entries directly with your editor of choice (vim!) 
+* Write your weblog entries directly with your editor of choice (vim!)
-  in `reStructuredText <http://docutils.sourceforge.net/rst.html>`_ or `Markdown <http://daringfireball.net/projects/markdown/>`_
+  in reStructuredText_ or Markdown_
 * Includes a simple CLI tool to (re)generate the weblog
 * Easy to interface with DVCSes and web hooks
 * Completely static output is easy to host anywhere
@@ -20,40 +22,49 @@ Pelican currently supports:
 * Comments, via an external service (Disqus). (Please note that while
   useful, Disqus is an external service, and thus the comment data will be
   somewhat outside of your control and potentially subject to data loss.)
-* Theming support (themes are created using `jinja2 <http://jinja.pocoo.org/>`_)
+* Theming support (themes are created using Jinja2_ templates)
 * PDF generation of the articles/pages (optional)
 * Publication of articles in multiple languages
 * Atom/RSS feeds
 * Code syntax highlighting
+* Asset management with `webassets`_ (optional)
 * Import from WordPress, Dotclear, or RSS feeds
 * Integration with external tools: Twitter, Google Analytics, etc. (optional)
 
-Have a look at `the documentation <http://pelican.notmyidea.org/en/latest/>`_ for
+Have a look at the `Pelican documentation`_ for more information.
-more information.
 
 Why the name "Pelican"?
-------------------------
+-----------------------
 
-Heh, you didn't notice? "Pelican" is an anagram for « Calepin » ;)
+"Pelican" is an anagram for *calepin*, which means "notebook" in French. ;)
 
 Source code
 -----------
 
-You can access the source code via git at: https://github.com/ametaireau/pelican
+You can access the source code at: https://github.com/getpelican/pelican
 
-If you feel hackish, have a look at the explanation of `Pelican's internals
+If you feel hackish, have a look at the explanation of `Pelican's internals`_.
-<http://pelican.notmyidea.org/en/latest/internals.html>`_.
 
 Feedback / Contact us
 ---------------------
 
-If you want to see new features in Pelican, don't hesitate to offer suggestions,
+If you want to see new features in Pelican, don't hesitate to offer
-clone the repository, etc. There are many ways to `contribute 
+suggestions, clone the repository, etc. There are many ways to contribute_.
-<http://pelican.notmyidea.org/en/latest/contribute.html>`_. That's open source, dude!
+That's open source, dude!
 
-Contact me at "alexis at notmyidea dot org" for any request/feedback! You can
+Send a message to "authors at getpelican dot com" with any requests/feedback! You
-also join the team at `#pelican on irc.freenode.org 
+can also join the team at `#pelican on Freenode`_ (or if you don't have an IRC
-<irc://irc.freenode.net/pelican>`_ 
+client handy, use the webchat_ for quick feedback.
-(or if you don't have any IRC client, use `the webchat
+
-<http://webchat.freenode.net/?channels=pelican&uio=d4>`_)
+.. Links
-for quick feedback.
+
+.. _Python: http://www.python.org/
+.. _reStructuredText: http://docutils.sourceforge.net/rst.html
+.. _Markdown: http://daringfireball.net/projects/markdown/
+.. _Jinja2: http://jinja.pocoo.org/
+.. _`Pelican documentation`: http://docs.getpelican.com/latest/
+.. _`Pelican's internals`: http://docs.getpelican.com/en/latest/internals.html
+.. _`#pelican on Freenode`: irc://irc.freenode.net/pelican
+.. _webchat: http://webchat.freenode.net/?channels=pelican&uio=d4
+.. _contribute: http://docs.getpelican.com/en/latest/contribute.html
+.. _webassets: https://github.com/miracle2k/webassets

THANKS

@@ -1,19 +1,122 @@
-Some people have helped to improve pelican by contributing features, reporting
+Pelican is a project by Alexis Métaireau <http://notmyidea.org> but there is
-bugs or giving ideas. Thanks to them !
+a quite big number of people that contributed or implemented key features over
+time.  We try to keep this list up to date, but you can have a look at the nice
+graphs proposed by github about contributors here:
+https://github.com/getpelican/pelican/graphs/contributors
 
-- Dan Jacka
+If you want to contibute, check the documentation section about how to do so
-- solsTiCe on linuxfr for reporting bugs
+<http://docs.getpelican.com/en/latest/contribute.html>
-- Guillaume B (Gui13)
+
-- Ronny Pfannschmidt
+Aaron Kavlie
-- Jérome Renard
+Abhishek L
-- Nicolas Martin
+Albrecht Mühlenschulte
-- David Kulak
+Aldiantoro Nugroho
-- Arnaud Bos
+Alen Mujezinovic
-- nblock (Florian)
+Alessandro Martin
-- Bruno Bord
+Alexander Artemenko
-- Laureline Guérin
+Alexandre RODIERE
-- Samuel Martin
+Alexis Daboville
-- Marcus Fredriksson
+Alexis Métaireau
-- Günter Kolousek
+Allan Whatmough
-- Simon Liedtke
+Andrea Crotti
-- Manuel F. Viera
+Andrew Laski
+Arnaud BOS
+asselinpaul
+Borgar
+Brandon W Maister
+Brendan Wholihan
+Brian C. Lane
+Brian Hsu
+Brian St. Pierre
+Bruno Binet
+BunnyMan
+Chris Streeter
+Christophe Chauvet
+Clint Howarth
+Dafydd Crosby
+Dana Woodman
+dave mankoff
+David Beitey
+David Marble
+derdon
+Dirkjan Ochtman
+Dirk Makowski
+draftcode
+Edward Delaporte
+epatters
+Eric Case
+Erik Hetzner
+FELD Boris
+Feth Arezki
+Florian Jacob
+Florian Preinstorfer
+Freeculture
+Guillaume
+Guillaume B
+Guillermo López
+guillermooo
+Ian Cordasco
+Iuri de Silvio
+James Rowe
+jawher
+Jerome
+Jiachen Yang
+Jochen Breuer
+joe di castro
+Jökull Sólberg Auðunsson
+Joshua Adelman
+Julian Berman
+justinmayer
+Justin Mayer
+Kyle Fuller
+Laureline Guerin
+Leonard Huang
+Marcel Hellkamp
+Marco Milanesi
+Marcus Fredriksson
+Mario Rodas
+Martin Brochhaus
+Massimo Santini
+Matt Bowcock
+Matt Layman
+Meir Kriheli
+Michael Guntsche
+Michael Reneer
+Michael Yanovich
+Mike Yumatov
+Mikhail Korobov
+m-r-r
+mviera
+Nico Di Rocco
+Nicolas Duhamel
+Nicolas Perriault
+Nicolas Steinmetz
+Pavel Puchkin
+Perry Roper
+Philippe Pepiot
+Rachid Belaid
+Ranjhith Kalisamy
+Remi Rampin
+Rémy HUBSCHER
+renhbo
+Roman Skvazh
+Ronny Pfannschmidt
+Rory McCann
+saghul
+sam
+Samrat Man Singh
+Simon Liedtke
+Skami18
+solsTiCe d'Hiver
+Stéphane Bunel
+Stéphane Raimbault
+Stuart Colville
+Tarek Ziade
+Thanos Lefteris
+the Bunny Man
+Tobias
+Tomi Pieviläinen
+Trae Blain
+Tshepang Lekhonkhobe
+Wladislaw Merezhko
+Zoresvit

TODO

@@ -1,9 +0,0 @@
-* Add a way to support pictures (see how sphinx makes that)
-* Make the program support UTF8-encoded files as input (and later: any encoding?)
-* Add status support (draft, published, hidden)
-* Add a serve + automatic generation behaviour.
-* Recompile only the changed files, not all.
-* Add a way to make the coffee (or not)
-* Add a sitemap generator.
-* read templates from the templates folder per default
-* add support of github via ghg import

dev_requirements.txt

@@ -1,10 +1,9 @@
-Jinja2>=2.4
+# Tests
-Pygments
-docutils
-feedgenerator
-unittest2
-pytz
 mock
+
+# Optional Packages
 Markdown
-blinker
+BeautifulSoup4
-BeautifulSoup
+lxml
+typogrify
+webassets

docs/changelog.rst

@@ -0,0 +1,161 @@
+Release history
+###############
+
+3.2 (XXXX-XX-XX)
+================
+
+* Support for Python 3!
+
+3.1 (2012-12-04)
+================
+
+* Importer now stores slugs within files by default. This can be disabled with
+  the ``--disable-slugs`` option.
+* Improve handling of links to intra-site resources
+* Ensure WordPress import adds paragraphs for all types of line endings
+  in post content
+* Decode HTML entities within WordPress post titles on import
+* Improve appearance of LinkedIn icon in default theme
+* Add GitHub and Google+ social icons support in default theme
+* Optimize social icons
+* Add ``FEED_ALL_ATOM`` and ``FEED_ALL_RSS`` to generate feeds containing all posts regardless of their language
+* Split ``TRANSLATION_FEED`` into ``TRANSLATION_FEED_ATOM`` and ``TRANSLATION_FEED_RSS``
+* Different feeds can now be enabled/disabled individually
+* Allow for blank author: if ``AUTHOR`` setting is not set, author won't
+  default to ``${USER}`` anymore, and a post won't contain any author
+  information if the post author is empty
+* Move LESS and Webassets support from Pelican core to plugin
+* The ``DEFAULT_DATE`` setting now defaults to ``None``, which means that
+  articles won't be generated unless date metadata is specified
+* Add ``FILENAME_METADATA`` setting to support metadata extraction from filename
+* Add ``gzip_cache`` plugin to compress common text files into a ``.gz``
+  file within the same directory as the original file, preventing the server
+  (e.g. Nginx) from having to compress files during an HTTP call
+* Add support for AsciiDoc-formatted content
+* Add ``USE_FOLDER_AS_CATEGORY`` setting so that feature can be toggled on/off
+* Support arbitrary Jinja template files
+* Restore basic functional tests
+* New signals: ``generator_init``, ``get_generators``, and
+  ``article_generate_preread``
+
+3.0 (2012-08-08)
+================
+
+* Refactored the way URLs are handled
+* Improved the English documentation
+* Fixed packaging using ``setuptools`` entrypoints
+* Added ``typogrify`` support
+* Added a way to disable feed generation
+* Added support for ``DIRECT_TEMPLATES``
+* Allow multiple extensions for content files
+* Added LESS support
+* Improved the import script
+* Added functional tests
+* Rsync support in the generated Makefile
+* Improved feed support (easily pluggable with Feedburner for instance)
+* Added support for ``abbr`` in reST
+* Fixed a bunch of bugs :-)
+
+2.8 (2012-02-28)
+==================
+
+* Dotclear importer
+* Allow the usage of Markdown extensions
+* Themes are now easily extensible
+* Don't output pagination information if there is only one page
+* Add a page per author, with all their articles
+* Improved the test suite
+* Made the themes easier to extend
+* Removed Skribit support
+* Added a ``pelican-quickstart`` script
+* Fixed timezone-related issues
+* Added some scripts for Windows support
+* Date can be specified in seconds
+* Never fail when generating posts (skip and continue)
+* Allow the use of future dates
+* Support having different timezones per language
+* Enhanced the documentation
+
+2.7 (2011-06-11)
+==================
+
+* Use ``logging`` rather than echoing to stdout
+* Support custom Jinja filters
+* Compatibility with Python 2.5
+* Added a theme manager
+* Packaged for Debian
+* Added draft support
+
+2.6 (2011-03-08)
+==================
+
+* Changes in the output directory structure
+* Makes templates easier to work with / create
+* Added RSS support (was Atom-only)
+* Added tag support for the feeds
+* Enhance the documentation
+* Added another theme (brownstone)
+* Added translations
+* Added a way to use cleaner URLs with a rewrite url module (or equivalent)
+* Added a tag cloud
+* Added an autoreloading feature: the blog is automatically regenerated each time a modification is detected
+* Translate the documentation into French
+* Import a blog from an RSS feed
+* Pagination support
+* Added Skribit support
+
+2.5 (2010-11-20)
+==================
+
+* Import from Wordpress
+* Added some new themes (martyalchin / wide-notmyidea)
+* First bug report!
+* Linkedin support
+* Added a FAQ
+* Google Analytics support
+* Twitter support
+* Use relative URLs, not static ones
+
+2.4 (2010-11-06)
+================
+
+* Minor themes changes
+* Add Disqus support (so we have comments)
+* Another code refactoring
+* Added config settings about pages
+* Blog entries can also be generated in PDF
+
+2.3 (2010-10-31)
+================
+
+* Markdown support
+
+2.2 (2010-10-30)
+================
+
+* Prettify output
+* Manages static pages as well
+
+2.1 (2010-10-30)
+================
+
+* Make notmyidea the default theme
+
+2.0 (2010-10-30)
+================
+
+* Refactoring to be more extensible
+* Change into the setting variables
+
+1.2 (2010-09-28)
+================
+
+* Added a debug option
+* Added per-category feeds
+* Use filesystem to get dates if no metadata is provided
+* Add Pygments support
+
+1.1 (2010-08-19)
+================
+
+* First working version

docs/conf.py

@@ -1,4 +1,5 @@
 # -*- coding: utf-8 -*-
+from __future__ import unicode_literals
 import sys, os
 
 sys.path.append(os.path.abspath('..'))
@@ -10,8 +11,8 @@ templates_path = ['_templates']
 extensions = ['sphinx.ext.autodoc',]
 source_suffix = '.rst'
 master_doc = 'index'
-project = u'Pelican'
+project = 'Pelican'
-copyright = u'2010, Alexis Metaireau and contributors'
+copyright = '2010, Alexis Metaireau and contributors'
 exclude_patterns = ['_build']
 version = __version__
 release = __major__
@@ -24,7 +25,7 @@ html_theme = 'pelican'
 html_theme_options = {
     'nosidebar': True,
     'index_logo': 'pelican.png',
-    'github_fork': 'ametaireau/pelican',
+    'github_fork': 'getpelican/pelican',
 }
 
 html_static_path = ['_static']
@@ -34,16 +35,16 @@ htmlhelp_basename = 'Pelicandoc'
 
 # -- Options for LaTeX output --------------------------------------------------
 latex_documents = [
-  ('index', 'Pelican.tex', u'Pelican Documentation',
+  ('index', 'Pelican.tex', 'Pelican Documentation',
-   u'Alexis Métaireau', 'manual'),
+   'Alexis Métaireau', 'manual'),
 ]
 
 # -- Options for manual page output --------------------------------------------
 man_pages = [
-    ('index', 'pelican', u'pelican documentation',
+    ('index', 'pelican', 'pelican documentation',
-     [u'Alexis Métaireau'], 1),
+     ['Alexis Métaireau'], 1),
-    ('pelican-themes', 'pelican-themes', u'A theme manager for Pelican',
+    ('pelican-themes', 'pelican-themes', 'A theme manager for Pelican',
-     [u'Mickaël Raybaud'], 1),
+     ['Mickaël Raybaud'], 1),
-    ('themes', 'pelican-theming', u'How to create themes for Pelican',
+    ('themes', 'pelican-theming', 'How to create themes for Pelican',
-     [u'The Pelican contributors'], 1)
+     ['The Pelican contributors'], 1)
 ]

docs/contribute.rst

@@ -3,14 +3,17 @@ How to contribute?
 There are many ways to contribute to Pelican. You can enhance the
 documentation, add missing features, and fix bugs (or just report them).
 
-Don't hesitate to fork and make a pull request on GitHub.
+Don't hesitate to fork and make a pull request on GitHub. When doing so, please
+create a new feature branch as opposed to making your commits in the master
+branch.
 
 Setting up the development environment
 ======================================
 
 You're free to set up your development environment any way you like. Here is a
-way using virtualenv and virtualenvwrapper. If you don't have them, you can
+way using the `virtualenv <http://www.virtualenv.org/>`_ and `virtualenvwrapper
-install these packages via::
+<http://www.doughellmann.com/projects/virtualenvwrapper/>`_ tools. If you don't
+have them, you can install these both of these packages via::
 
     $ pip install virtualenvwrapper
 
@@ -20,30 +23,103 @@ different projects.
 
 To create a virtual environment, use the following syntax::
 
-    $ mkvirtualenv pelican 
+    $ mkvirtualenv pelican
 
-To manually install the dependencies::
+To clone the Pelican source::
 
+    $ git clone https://github.com/getpelican/pelican.git src/pelican
+
+To install the development dependencies::
+
+    $ cd src/pelican
     $ pip install -r dev_requirements.txt
+
+To install Pelican and its dependencies::
+
     $ python setup.py develop
 
 Running the test suite
 ======================
 
 Each time you add a feature, there are two things to do regarding tests:
-checking that the existing tests pass, and adding tests for your new feature
+checking that the existing tests pass, and adding tests for the new feature
-or for the bug you're fixing.
+or bugfix.
 
 The tests live in "pelican/tests" and you can run them using the
 "discover" feature of unittest2::
 
     $ unit2 discover
 
+If you have made changes that affect the output of a Pelican-generated weblog,
+then you should update the output used by functional tests.
+To do so, you can use the following two commands::
+
+    $ LC_ALL="C" pelican -o tests/output/custom/ -s samples/pelican.conf.py \
+        samples/content/
+    $ LC_ALL="C" pelican -o tests/output/basic/ samples/content/
+
+testing for python3
+-------------------
+
+On Python 3, if you have installed the Py3k compatible versions of the
+plugins manual testing with ``unit2 discover`` is also straightforward.
+
+However, you must tell tox to use those Py3k libraries. If you forget this,
+tox will pull the regular packages from PyPi and the tests will fail.
+
+Tell tox about the local packages thusly: enter the source directory of
+smartypants and run tox there. Do this again for typogrify and webassets.
+Smartypants and typogrify do not have real tests, and webassets will fail
+noisily, but as a result we get these libraries neatly packaged in tox's
+distshare directory. And this we need to run tox for Pelican.
+
 Coding standards
 ================
 
-Try to respect what is described in the PEP8
+Try to respect what is described in the `PEP8 specification
-(http://www.python.org/dev/peps/pep-0008/) when providing patches. This can be
+<http://www.python.org/dev/peps/pep-0008/>`_ when providing patches. This can be
-eased by the pep8 tool (http://pypi.python.org/pypi/pep8) or by Flake8, which
+eased via the `pep8 <http://pypi.python.org/pypi/pep8>`_ or `flake8
-will give you some other cool hints about what's good or wrong
+<http://pypi.python.org/pypi/flake8/>`_ tools, the latter of which in
-(http://pypi.python.org/pypi/flake8/)
+particular will give you some useful hints about ways in which the
+code/formatting can be improved.
+
+Python3 support
+===============
+
+Here are some tips that may be useful when doing some code for both python2 and
+python3 at the same time:
+
+- Assume, every string and literal is unicode (import unicode_literals):
+ 
+  - Do not use prefix ``u'``.
+  - Do not encode/decode strings in the middle of sth. Follow the code to the
+    source (or target) of a string and encode/decode at the first/last possible
+    point.
+  - In other words, write your functions to expect and to return unicode.
+  - Encode/decode strings if e.g. the source is a Python function that is known
+    to handle this badly, e.g. strftime() in Python 2.
+
+- Use new syntax: print function, "except ... *as* e" (not comma) etc.
+- Refactor method calls like ``dict.iteritems()``, ``xrange()`` etc. in a way
+  that runs without code change in both Python versions.
+- Do not use magic method ``__unicode()__`` in new classes. Use only ``__str()__``
+  and decorate the class with ``@python_2_unicode_compatible``.
+- Do not start int literals with a zero. This is a syntax error in Py3k.
+- Unfortunately I did not find an octal notation that is valid in both
+  Pythons. Use decimal instead.
+- use six, e.g.:
+
+  - ``isinstance(.., basestring) -> isinstance(.., six.string_types)``
+  - ``isinstance(.., unicode) -> isinstance(.., six.text_type)``
+
+- ``setlocale()`` in Python 2 bails when we give the locale name as unicode,
+  and since we are using ``from __future__ import unicode_literals``, we do
+  that everywhere!  As a workaround, I enclosed the localename with ``str()``;
+  in Python 2 this casts the name to a byte string, in Python 3 this should do
+  nothing, because the locale name already had been unicode.
+
+- Kept range() almost everywhere as-is (2to3 suggests list(range())), just
+  changed it where I felt necessary.
+
+- Changed xrange() back to range(), so it is valid in both Python versions.
+

docs/faq.rst

@@ -1,7 +1,36 @@
 Frequently Asked Questions (FAQ)
 ################################
 
-Here is a summary of the frequently asked questions for Pelican.
+Here are some frequently asked questions about Pelican.
+
+What's the best way to communicate a problem, question, or suggestion?
+======================================================================
+
+If you have a problem, question, or suggestion, please start by striking up a
+conversation on `#pelican on Freenode <irc://irc.freenode.net/pelican>`_.
+Those who don't have an IRC client handy can jump in immediately via
+`IRC webchat <http://webchat.freenode.net/?channels=pelican&uio=d4>`_. Because
+of differing time zones, you may not get an immediate response to your question,
+but please be patient and stay logged into IRC — someone will almost always
+respond.
+
+If you are unable to resolve your issue or if you have a feature request, please
+refer to the `issue tracker <https://github.com/getpelican/pelican/issues>`_.
+
+How can I help?
+================
+
+There are several ways to help out. First, you can use Pelican and report any
+suggestions or problems you might have via IRC or the `issue tracker
+<https://github.com/getpelican/pelican/issues>`_.
+
+If you want to contribute, please fork `the git repository
+<https://github.com/getpelican/pelican/>`_, create a new feature branch, make
+your changes, and issue a pull request. Someone will review your changes as soon
+as possible. Please refer to the :doc:`How to Contribute <contribute>` section
+for more details.
+
+You can also contribute by creating themes and improving the documentation.
 
 Is it mandatory to have a configuration file?
 =============================================
@@ -15,7 +44,7 @@ I'm creating my own theme. How do I use Pygments for syntax highlighting?
 
 Pygments adds some classes to the generated content. These classes are used by
 themes to style code syntax highlighting via CSS. Specifically, you can
-customize the appearance of your syntax highlighting via the ``.codehilite pre`` 
+customize the appearance of your syntax highlighting via the ``.codehilite pre``
 class in your theme's CSS file. To see how various styles can be used to render
 Django code, for example, you can use the demo `on the project website
 <http://pygments.org/demo/15101/>`_.
@@ -25,27 +54,90 @@ How do I create my own theme?
 
 Please refer to :ref:`theming-pelican`.
 
-How can I help?
-================
-
-There are several ways to help out. First, you can use Pelican and report any
-suggestions or problems you might have on `the bugtracker
-<https://github.com/ametaireau/pelican/issues>`_.
-
-If you want to contribute, please fork `the git repository
-<https://github.com/ametaireau/pelican/>`_, make your changes, and issue
-a pull request. I'll review your changes as soon as possible.
-
-You can also contribute by creating themes and improving the documentation.
-
 I want to use Markdown, but I got an error.
 ===========================================
 
 Markdown is not a hard dependency for Pelican, so you will need to explicitly
-install it. You can do so by typing::
+install it. You can do so by typing the following, including ``sudo`` if
+required::
 
-    $ (sudo) pip install markdown
+    (sudo) pip install markdown
 
-In case you don't have pip installed, consider installing it via::
+If you don't have pip installed, consider installing the pip installer via::
 
-    $ (sudo) easy_install pip
+    (sudo) easy_install pip
+
+Can I use arbitrary meta-data in my templates?
+==============================================
+
+Yes. For example, to include a modified date in a Markdown post, one could
+include the following at the top of the article::
+
+    Modified: 2012-08-08
+
+That meta-data can then be accessed in the template::
+
+    {% if article.modified %}
+    Last modified: {{ article.modified }}
+    {% endif %}
+
+How do I assign custom templates on a per-page basis?
+=====================================================
+
+It's as simple as adding an extra line of metadata to any pages or articles you
+want to have its own template.
+
+    :template: template_name
+
+Then just make sure your theme contains the relevant template file (e.g.
+``template_name.html``).
+
+What if I want to disable feed generation?
+==========================================
+
+To disable all feed generation, all feed settings should be set to ``None``.
+All but two feed settings already default to ``None``, so if you want to disable
+all feed generation, you only need to specify the following settings::
+
+    FEED_ALL_ATOM = None
+    CATEGORY_FEED_ATOM = None
+
+Please note that ``None`` and ``''`` are not the same thing. The word ``None``
+should not be surrounded by quotes.
+
+I'm getting a warning about feeds generated without SITEURL being set properly
+==============================================================================
+
+`RSS and Atom feeds require all URLs and links in them to be absolute
+<http://validator.w3.org/feed/docs/rss2.html#comments>`_.
+In order to properly generate all URLs properly in Pelican you will need to set
+``SITEURL`` to the full path of your blog. When using ``make html`` and the
+default Makefile provided by the `pelican-quickstart` bootstrap script to test
+build your site, it's normal to see this warning since ``SITEURL`` is
+deliberately left undefined. If configured properly no other ``make`` commands
+should result in this warning.
+
+Feeds are still generated when this warning is displayed but may not validate.
+
+My feeds are broken since I upgraded to Pelican 3.x
+===================================================
+
+Starting in 3.0, some of the FEED setting names were changed to more explicitly
+refer to the Atom feeds they inherently represent (much like the FEED_RSS
+setting names). Here is an exact list of the renamed setting names::
+
+    FEED -> FEED_ATOM
+    TAG_FEED -> TAG_FEED_ATOM
+    CATEGORY_FEED -> CATEGORY_FEED_ATOM
+
+Starting in 3.1, the new feed ``FEED_ALL_ATOM`` has been introduced: this
+feed will aggregate all posts regardless of their language. This setting
+generates ``'feeds/all.atom.xml'`` by default and ``FEED_ATOM`` now defaults to
+``None``. The following feed setting has also been renamed::
+
+    TRANSLATION_FEED -> TRANSLATION_FEED_ATOM
+
+Older themes that referenced the old setting names may not link properly.
+In order to rectify this, please update your theme for compatibility by changing
+the relevant values in your template files. For an example of complete feed
+headers and usage please check out the ``simple`` theme.

docs/fr/configuration.rst

@@ -52,19 +52,25 @@ détails au prochain chapitre.
 Flux de syndication
 ===================
 
-CATEGORY_FEED : 
+CATEGORY_FEED_ATOM :
 	Chemin d’écriture des flux Atom liés aux catégories ;
 
 CATEGORY_FEED_RSS : 
 	Idem pour les flux rss (Optionnel);
 
-FEED :
+FEED_ATOM :
-	Chemin du flux Atom global ;
+	Chemin du flux Atom global;
 
 FEED_RSS :
 	Chemin du flux Rss global (Optionnel);
 
-TAG_FEED :
+FEED_ALL_ATOM :
+	Chemin du flux Atom global qui inclut la totalité des posts, indépendamment de la langue;
+
+FEED_ALL_RSS :
+	Chemin du flux Rss global  qui inclut la totalité des posts, indépendamment de la langue (Optionnel);
+
+TAG_FEED_ATOM :
 	Chemin des flux Atom pour les tags (Optionnel);
 
 TAG_FEED_RSS :
@@ -77,8 +83,11 @@ Traductions
 DEFAULT_LANG :
 	Le langage par défaut à utiliser. «*en*» par défaut ;
 
-TRANSLATION_FEED :
+TRANSLATION_FEED_ATOM :
-	Chemin du flux pour les traductions.
+	Chemin du flux Atom pour les traductions.
+
+TRANSLATION_FEED_RSS :
+	Chemin du flux RSS pour les traductions.
 
 
 Thèmes
@@ -111,7 +120,7 @@ LINKS :
 PDF_PROCESSOR :
 	Génère ou non les articles et pages au format pdf ;
 
-REVERSE_ARCHIVE_ORDER :
+NEWEST_FIRST_ARCHIVES :
 	Met les articles plus récent en tête de l'archive ;
 
 SOCIAL :
@@ -133,8 +142,12 @@ Pelican est fournit avec :doc:`pelican-themes`, un script permettant de gérer l
 Paramètres divers
 =================
 
-FALLBACK_ON_FS_DATE :
+DEFAULT_DATE:
-	Si *True*, Pelican se basera sur le *mtime* du fichier s'il n'y a pas de date spécifiée dans le fichier de l'article ;
+    Date par défaut à utiliser si l'information de date n'est pas spécifiée
+    dans les metadonnées de l'article.
+    Si 'fs', Pelican se basera sur le *mtime* du fichier.
+    Si c'est un tuple, il sera passé au constructeur datetime.datetime pour
+    générer l'objet datetime utilisé par défaut.
 
 KEEP_OUTPUT DIRECTORY :
 	Ne génère que les fichiers modifiés et n'efface pas le repertoire de sortie ;
@@ -150,8 +163,3 @@ SITEURL :
 
 STATIC_PATHS :
 	Les chemins statiques que vous voulez avoir accès sur le chemin de sortie "statique" ;
-
-
-
-
-

docs/fr/index.rst

@@ -30,7 +30,7 @@ Code source
 ===========
 
 Vous pouvez accéder au code source via git à l'adresse 
-http://github.com/ametaireau/pelican/
+http://github.com/getpelican/pelican/
 
 Feedback !
 ==========

docs/fr/installation.rst

@@ -24,7 +24,7 @@ Pour installer Pelican en reprenant le code via Github, nous aurons besoin du pa
 git-core pour récupérez les sources de Pelican. Puis nous procédons à l’installation ::
 
 	# apt-get install git-core
-	$ git clone https://github.com/ametaireau/pelican.git
+	$ git clone https://github.com/getpelican/pelican.git
 	$ cd pelican
 	# python setup.py install
 

docs/fr/themes.rst

@@ -10,7 +10,7 @@ Pelican utlise le très bon moteur de template `jinja2 <http://jinja.pocoo.org>`
 pour produire de l'HTML. La syntaxe de jinja2 est vraiment très simple. Si vous
 voulez créer votre propre thème, soyez libre de prendre inspiration sur le theme
 "simple" qui est disponible `ici
-<https://github.com/ametaireau/pelican/tree/master/pelican/themes/simple/templates>`_
+<https://github.com/getpelican/pelican/tree/master/pelican/themes/simple/templates>`_
 
 Structure
 =========

docs/getting_started.rst

@@ -14,17 +14,19 @@ If you don't have ``pip`` installed, an alternative method is ``easy_install``::
     $ easy_install pelican
 
 While the above is the simplest method, the recommended approach is to create
-a virtual environment for Pelican via `virtualenv <http://www.virtualenv.org/>`_
+a virtual environment for Pelican via virtualenv_ and virtualenvwrapper_ before
-and `virtualenvwrapper <http://www.doughellmann.com/projects/virtualenvwrapper/>`_
+installing Pelican. Assuming you've followed the virtualenvwrapper
-before installing Pelican::
+`installation <http://virtualenvwrapper.readthedocs.org/en/latest/install.html>`_
+and `shell configuration
+<http://virtualenvwrapper.readthedocs.org/en/latest/install.html#shell-startup-file>`_
+steps, you can then open a new terminal session and create a new virtual
+environment for Pelican::
 
-    $ sudo pip install --upgrade virtualenv virtualenvwrapper
     $ mkvirtualenv pelican
-    $ pip install pelican
 
 Once the virtual environment has been created and activated, Pelican can be
 be installed via ``pip`` or ``easy_install`` as noted above. Alternatively, if
-you have the project source, you can install Pelican using the distutils 
+you have the project source, you can install Pelican using the distutils
 method::
 
     $ cd path-to-Pelican-source
@@ -33,19 +35,23 @@ method::
 If you have Git installed and prefer to install the latest bleeding-edge
 version of Pelican rather than a stable release, use the following command::
 
-    $ pip install -e git://github.com/ametaireau/pelican#egg=pelican
+    $ pip install -e git://github.com/getpelican/pelican#egg=pelican
 
 If you plan on using Markdown as a markup format, you'll need to install the
 Markdown library as well::
 
     $ pip install Markdown
 
+If you want to use AsciiDoc you need to install it from `source
+<http://www.methods.co.nz/asciidoc/INSTALL.html>`_ or use your operating
+system's package manager.
+
 Upgrading
 ---------
 
-If you installed a stable Pelican release via pip or easy_install and wish to
+If you installed a stable Pelican release via ``pip`` or ``easy_install`` and
-upgrade to the latest stable release, you can do so by adding ``--upgrade`` to
+wish to upgrade to the latest stable release, you can do so by adding
-the relevant command. For pip, that would be::
+``--upgrade`` to the relevant command. For pip, that would be::
 
     $ pip install --upgrade pelican
 
@@ -55,28 +61,38 @@ perform the same step to install the most recent version.
 Dependencies
 ------------
 
-At this time, Pelican is dependent on the following Python packages:
+At this time, Pelican core is dependent on the following Python packages:
 
-* feedgenerator, to generate the Atom feeds
+* `feedgenerator <http://pypi.python.org/pypi/feedgenerator>`_, to generate the
-* jinja2, for templating support
+  Atom feeds
-* docutils, for supporting reStructuredText as an input format
+* `jinja2 <http://pypi.python.org/pypi/Jinja2>`_, for templating support
+* `pygments <http://pypi.python.org/pypi/Pygments>`_, for syntax highlighting
+* `docutils <http://pypi.python.org/pypi/docutils>`_, for supporting
+  reStructuredText as an input format
+* `pytz <http://pypi.python.org/pypi/pytz>`_, for timezone definitions
+* `blinker <http://pypi.python.org/pypi/blinker>`_, an object-to-object and
+  broadcast signaling system
+* `unidecode <http://pypi.python.org/pypi/Unidecode>`_, for ASCII
+  transliterations of Unicode text
 
 If you're not using Python 2.7, you will also need the ``argparse`` package.
 
 Optionally:
 
-* pygments, for syntax highlighting
+* `markdown <http://pypi.python.org/pypi/Markdown>`_, for supporting Markdown as
-* Markdown, for supporting Markdown as an input format
+  an input format
+* `typogrify <http://pypi.python.org/pypi/typogrify>`_, for typographical
+  enhancements
 
 Kickstart a blog
 ================
 
 Following is a brief tutorial for those who want to get started right away.
-We're going to assume Pelican was installed in a virtual environment via the
+We're going to assume that virtualenv_ and virtualenvwrapper_ are installed and
-following steps (if you're not using a virtual environment for Pelican, you can
+configured; if you've installed Pelican outside of a virtual environment,
-skip to the ``pelican-quickstart`` command)::
+you can skip to the ``pelican-quickstart`` command. Let's first create a new
+virtual environment and install Pelican into it::
 
-    $ sudo pip install --upgrade virtualenv virtualenvwrapper
     $ mkvirtualenv pelican
     $ pip install pelican Markdown
 
@@ -107,11 +123,21 @@ instead::
 
     $ make regenerate
 
-To serve the site so it can be previewed in your browser::
+To serve the site so it can be previewed in your browser at
+http://localhost:8000::
 
     $ make serve
 
-Visit http://localhost:8000 in your browser to see your site.
+Normally you would need to run ``make regenerate`` and ``make serve`` in two
+separate terminal sessions, but you can run both at once via::
+
+    $ make devserver
+
+The above command will simultaneously run Pelican in regeneration mode as well
+as serve the output at http://localhost:8000. Once you are done testing your
+changes, you should stop the development server via::
+
+    $ ./develop_server.sh stop
 
 When you're ready to publish your site, you can upload it via the method(s) you
 chose during the ``pelican-quickstart`` questionnaire. For this example, we'll
@@ -140,36 +166,62 @@ following syntax (give your file the ``.rst`` extension)::
     :date: 2010-10-03 10:20
     :tags: thats, awesome
     :category: yeah
+    :slug: my-super-post
     :author: Alexis Metaireau
+    :summary: Short version for index and feeds
 
-You can also use Markdown syntax (with a file ending in ``.md``).
+Pelican implements an extension to reStructuredText to enable support for the
-Markdown generation will not work until you explicitly install the ``Markdown``
+``abbr`` HTML tag. To use it, write something like this in your post::
-package, which can be done via ``pip install Markdown``. Metadata syntax for
+
-Markdown posts should follow this pattern::
+    This will be turned into :abbr:`HTML (HyperText Markup Language)`.
+
+You can also use Markdown syntax (with a file ending in ``.md``, ``.markdown``,
+or ``.mkd``). Markdown generation will not work until you explicitly install the
+``Markdown`` package, which can be done via ``pip install Markdown``. Metadata
+syntax for Markdown posts should follow this pattern::
 
-    Date: 2010-12-03
     Title: My super title
+    Date: 2010-12-03 10:20
     Tags: thats, awesome
+    Category: yeah
     Slug: my-super-post
+    Author: Alexis Metaireau
+    Summary: Short version for index and feeds
 
     This is the content of my super blog post.
 
-Note that, aside from the title, none of this metadata is mandatory: if the date
+Note that, aside from the title, none of this metadata is mandatory: if the
-is not specified, Pelican will rely on the file's "mtime" timestamp, and the
+date is not specified, Pelican can rely on the file's "mtime" timestamp through
-category can be determined by the directory in which the file resides. For
+the ``DEFAULT_DATE`` setting, and the category can be determined by the
-example, a file located at ``python/foobar/myfoobar.rst`` will have a category of
+directory in which the file resides. For example, a file located at
-``foobar``.
+``python/foobar/myfoobar.rst`` will have a category of ``foobar``. If you would
+like to organize your files in other ways where the name of the subfolder would
+not be a good category name, you can set the setting ``USE_FOLDER_AS_CATEGORY``
+to ``False``. If there is no summary metadata for a given post, the
+``SUMMARY_MAX_LENGTH`` setting can be used to specify how many words from the
+beginning of an article are used as the summary.
+
+You can also extract any metadata from the filename through a regular
+expression to be set in the ``FILENAME_METADATA`` setting.
+All named groups that are matched will be set in the metadata object. The
+default value for the ``FILENAME_METADATA`` setting will only extract the date
+from the filename. For example, if you would like to extract both the date and
+the slug, you could set something like:
+``'(?P<date>\d{4}-\d{2}-\d{2})_(?P<slug>.*)'``
+
+Please note that the metadata available inside your files takes precedence over
+the metadata extracted from the filename.
 
 Generate your blog
 ------------------
 
-The ``make`` shortcut commands mentioned in the ``Kickstart a blog`` section
+The ``make`` shortcut commands mentioned in the *Kickstart a blog* section
 are mostly wrappers around the ``pelican`` command that generates the HTML from
 the content. The ``pelican`` command can also be run directly::
 
     $ pelican /path/to/your/content/ [-s path/to/your/settings.py]
 
-The above command will generate your weblog and save it in the ``content/``
+The above command will generate your weblog and save it in the ``output/``
 folder, using the default theme to produce a simple site. The default theme is
 simple HTML without styling and is provided so folks may use it as a basis for
 creating their own themes.
@@ -189,20 +241,68 @@ run the ``pelican`` command with the ``-r`` or ``--autoreload`` option.
 Pages
 -----
 
-If you create a folder named ``pages``, all the files in it will be used to
+If you create a folder named ``pages`` inside the content folder, all the
-generate static pages.
+files in it will be used to generate static pages.
 
-Then, use the ``DISPLAY_PAGES_ON_MENU`` setting, which will add all the pages to 
+Then, use the ``DISPLAY_PAGES_ON_MENU`` setting to add all those pages to
-the menu.
+the primary navigation menu.
 
 If you want to exclude any pages from being linked to or listed in the menu
 then add a ``status: hidden`` attribute to its metadata. This is useful for
 things like making error pages that fit the generated theme of your site.
 
+Linking to internal content
+---------------------------
+
+From Pelican 3.1 onwards, it is now possible to specify intra-site links to
+files in the *source content* hierarchy instead of files in the *generated*
+hierarchy. This makes it easier to link from the current post to other posts
+and images that may be sitting alongside the current post (instead of having
+to determine where those resources will be placed after site generation).
+
+To link to internal content, use the following syntax:
+``|filename|path/to/file``.
+
+For example, you may want to add links between "article1" and "article2" given
+the structure::
+
+    website/
+    ├── content
+    │   ├── article1.rst
+    │   └── cat/
+    │       └── article2.md
+    └── pelican.conf.py
+
+In this example, ``article1.rst`` could look like::
+
+    Title: The first article
+    Date: 2012-12-01
+
+    See below intra-site link examples in reStructuredText format.
+
+    `a link relative to content root <|filename|/cat/article2.md>`_
+    `a link relative to current file <|filename|cat/article2.md>`_
+
+and ``article2.md``::
+
+    Title: The second article
+    Date: 2012-12-01
+
+    See below intra-site link examples in Markdown format.
+
+    [a link relative to content root](|filename|/article1.rst)
+    [a link relative to current file](|filename|../article1.rst)
+
+.. note::
+
+    You can use the same syntax to link to internal pages or even static
+    content (like images) which would be available in a directory listed in
+    ``settings["STATIC_PATHS"]``.
+
 Importing an existing blog
 --------------------------
 
-It is possible to import your blog from Dotclear, WordPress, and RSS feeds using 
+It is possible to import your blog from Dotclear, WordPress, and RSS feeds using
 a simple script. See :ref:`import`.
 
 Translations
@@ -248,25 +348,27 @@ then instead ensure that the translated article titles are identical, since the
 slug will be auto-generated from the article title.
 
 Syntax highlighting
----------------------
+-------------------
 
 Pelican is able to provide colorized syntax highlighting for your code blocks.
-To do so, you have to use the following conventions (you need to put this in
+To do so, you have to use the following conventions inside your content files.
-your content files).
 
-For RestructuredText::
+For reStructuredText, use the code-block directive::
 
     .. code-block:: identifier
 
-       your code goes here
+       <indented code block goes here>
 
-For Markdown, format your code blocks thusly::
+For Markdown, include the language identifier just above the code block,
+indenting both the identifier and code::
 
-    :::identifier
+    A block of text.
-    your code goes here
 
-The specified identifier should be one that appears on the 
+        :::identifier
-`list of available lexers <http://pygments.org/docs/lexers/>`_.
+        <code goes here>
+
+The specified identifier (e.g. ``python``, ``ruby``) should be one that
+appears on the `list of available lexers <http://pygments.org/docs/lexers/>`_.
 
 Publishing drafts
 -----------------
@@ -284,9 +386,11 @@ anything special to see what's happening with the generated files.
 
 You can either use your browser to open the files on your disk::
 
-    $ firefox output/index.html
+    firefox output/index.html
 
 Or run a simple web server using Python::
 
     cd output && python -m SimpleHTTPServer
 
+.. _virtualenv: http://www.virtualenv.org/
+.. _virtualenvwrapper: http://www.doughellmann.com/projects/virtualenvwrapper/

docs/importer.rst

@@ -4,71 +4,90 @@
  Import from other blog software
 =================================
 
+
 Description
 ===========
 
-``pelican-import`` is a command line tool for converting articles from other
+``pelican-import`` is a command-line tool for converting articles from other
-software to ReStructuredText. The supported formats are:
+software to ReStructuredText or Markdown. The supported import formats are:
 
 - WordPress XML export
 - Dotclear export
 - RSS/Atom feed
 
-The conversion from HTML to reStructuredText relies on `pandoc
+The conversion from HTML to reStructuredText or Markdown relies on `Pandoc`_.
-<http://johnmacfarlane.net/pandoc/>`_. For Dotclear, if the source posts are
+For Dotclear, if the source posts are written with Markdown syntax, they will
-written with Markdown syntax, they will not be converted (as Pelican also
+not be converted (as Pelican also supports Markdown).
-supports Markdown).
+
 
 Dependencies
-""""""""""""
+============
 
-``pelican-import`` has two dependencies not required by the rest of pelican:
+``pelican-import`` has some dependencies not required by the rest of pelican:
 
-- BeautifulSoup
+- *BeautifulSoup*, for WordPress and Dotclear import. Can be installed like
-- pandoc
+  any other Python package (``pip install BeautifulSoup``).
+- *Feedparser*, for feed import (``pip install feedparser``).
+- *Pandoc*, see the `Pandoc site`_ for installation instructions on your
+  operating system.
 
-BeatifulSoup can be installed like any other Python package::
+.. _Pandoc: http://johnmacfarlane.net/pandoc/
-
+.. _Pandoc site: http://johnmacfarlane.net/pandoc/installing.html
-    $ pip install BeautifulSoup
-
-For pandoc, install a package for your operating system from the 
-`pandoc site <http://johnmacfarlane.net/pandoc/installing.html>`_.
 
 
 Usage
-"""""
+=====
 
-| pelican-import [-h] [--wpfile] [--dotclear] [--feed] [-o OUTPUT]
+::
-|                [-m MARKUP][--dir-cat]
+
-|                input
+    pelican-import [-h] [--wpfile] [--dotclear] [--feed] [-o OUTPUT]
+                   [-m MARKUP] [--dir-cat] [--strip-raw] [--disable-slugs]
+                   input
+
+Positional arguments
+--------------------
+
+  input                 The input file to read
 
 Optional arguments
-""""""""""""""""""
+------------------
 
-  -h, --help            show this help message and exit
+  -h, --help            Show this help message and exit
-  --wpfile              Wordpress XML export
+  --wpfile              WordPress XML export (default: False)
-  --dotclear            Dotclear export
+  --dotclear            Dotclear export (default: False)
-  --feed                Feed to parse
+  --feed                Feed to parse (default: False)
   -o OUTPUT, --output OUTPUT
-                        Output path
+                        Output path (default: output)
-  -m MARKUP             Output markup
+  -m MARKUP, --markup MARKUP
+                        Output markup format (supports rst & markdown)
+                        (default: rst)
   --dir-cat             Put files in directories with categories name
+                        (default: False)
+  --strip-raw           Strip raw HTML code that can't be converted to markup
+                        such as flash embeds or iframes (wordpress import
+                        only) (default: False)
+  --disable-slugs       Disable storing slugs from imported posts within
+                        output. With this disabled, your Pelican URLs may not
+                        be consistent with your original posts. (default:
+                        False)
+
 
 Examples
 ========
 
-for WordPress::
+For WordPress::
 
     $ pelican-import --wpfile -o ~/output ~/posts.xml
 
-for Dotclear::
+For Dotclear::
 
     $ pelican-import --dotclear -o ~/output ~/backup.txt
 
+
 Tests
 =====
 
 To test the module, one can use sample files:
 
-- for Wordpress: http://wpcandy.com/made/the-sample-post-collection
+- for WordPress: http://wpcandy.com/made/the-sample-post-collection
 - for Dotclear: http://themes.dotaddict.org/files/public/downloads/lorem-backup.txt

docs/index.rst

@@ -1,16 +1,16 @@
 Pelican
-#######
+=======
 
-Pelican is a simple weblog generator, written in Python.
+Pelican is a static site generator, written in Python_.
 
-* Write your weblog entries directly with your editor of choice (vim!) in
+* Write your weblog entries directly with your editor of choice (vim!)
-  reStructuredText or Markdown
+  in reStructuredText_, Markdown_, or AsciiDoc_
-* A simple CLI tool to (re)generate the weblog
+* Includes a simple CLI tool to (re)generate the weblog
 * Easy to interface with DVCSes and web hooks
 * Completely static output is easy to host anywhere
 
 Features
-========
+--------
 
 Pelican currently supports:
 
@@ -18,40 +18,38 @@ Pelican currently supports:
 * Comments, via an external service (Disqus). (Please note that while
   useful, Disqus is an external service, and thus the comment data will be
   somewhat outside of your control and potentially subject to data loss.)
-* Theming support (themes are created using `jinja2 <http://jinja.pocoo.org/>`_)
+* Theming support (themes are created using Jinja2_ templates)
 * PDF generation of the articles/pages (optional)
 * Publication of articles in multiple languages
 * Atom/RSS feeds
 * Code syntax highlighting
-* Compilation of less css (optional)
+* Asset management with `webassets`_ (optional)
 * Import from WordPress, Dotclear, or RSS feeds
 * Integration with external tools: Twitter, Google Analytics, etc. (optional)
 
-Why the name "Pelican" ?
+Why the name "Pelican"?
-========================
+-----------------------
 
-Heh, you didn't notice? "Pelican" is an anagram for « Calepin » ;)
+"Pelican" is an anagram for *calepin*, which means "notebook" in French. ;)
 
 Source code
-===========
+-----------
 
-You can access the source code via git at http://github.com/ametaireau/pelican/
+You can access the source code at: https://github.com/getpelican/pelican
 
 Feedback / Contact us
-=====================
+---------------------
 
-If you want to see new features in Pelican, don't hesitate to tell me, to clone
+If you want to see new features in Pelican, don't hesitate to offer suggestions,
-the repository, etc. That's open source, dude!
+clone the repository, etc. There are many ways to :doc:`contribute<contribute>`.
+That's open source, dude!
 
-Contact me at "alexis at notmyidea dot org" for any request/feedback! You can
+Send a message to "authors at getpelican dot com" with any requests/feedback! You
-also join the team at `#pelican on irc.freenode.org
+can also join the team at `#pelican on Freenode`_ (or if you don't have an IRC
-<irc://irc.freenode.net/pelican>`_
+client handy, use the webchat_ for quick feedback.
-(or if you don't have any IRC client, use `the webchat
-<http://webchat.freenode.net/?channels=pelican&uio=d4>`_)
-for quick feedback.
 
 Documentation
-=============
+-------------
 
 A French version of the documentation is available at :doc:`fr/index`.
 
@@ -69,3 +67,17 @@ A French version of the documentation is available at :doc:`fr/index`.
    tips
    contribute
    report
+   changelog
+
+.. Links
+
+.. _Python: http://www.python.org/
+.. _reStructuredText: http://docutils.sourceforge.net/rst.html
+.. _Markdown: http://daringfireball.net/projects/markdown/
+.. _AsciiDoc: http://www.methods.co.nz/asciidoc/index.html
+.. _Jinja2: http://jinja.pocoo.org/
+.. _`Pelican documentation`: http://docs.getpelican.com/latest/
+.. _`Pelican's internals`: http://docs.getpelican.com/en/latest/internals.html
+.. _`#pelican on Freenode`: irc://irc.freenode.net/pelican
+.. _webchat: http://webchat.freenode.net/?channels=pelican&uio=d4
+.. _webassets: https://github.com/miracle2k/webassets

docs/internals.rst

@@ -12,8 +12,8 @@ original author wrote with some software design information.
 Overall structure
 =================
 
-What Pelican does is take a list of files and process them into some
+What Pelican does is take a list of files and process them into some sort of
-sort of output. Usually, the input files are reStructuredText and Markdown
+output. Usually, the input files are reStructuredText, Markdown and AsciiDoc
 files, and the output is a blog, but both input and output can be anything you
 want.
 
@@ -23,9 +23,9 @@ The logic is separated into different classes and concepts:
   on. Since those operations are commonly used, the object is created once and
   then passed to the generators.
 
-* **Readers** are used to read from various formats (Markdown and
+* **Readers** are used to read from various formats (AsciiDoc, Markdown and
-  reStructuredText for now, but the system is extensible). Given a file, they return
+  reStructuredText for now, but the system is extensible). Given a file, they
-  metadata (author, tags, category, etc.) and content (HTML-formatted).
+  return metadata (author, tags, category, etc.) and content (HTML-formatted).
 
 * **Generators** generate the different outputs. For instance, Pelican comes with
   ``ArticlesGenerator`` and ``PageGenerator``. Given a configuration, they can do
@@ -47,19 +47,17 @@ Take a look at the Markdown reader::
     class MarkdownReader(Reader):
         enabled = bool(Markdown)
 
-        def read(self, filename):
+        def read(self, source_path):
             """Parse content and metadata of markdown files"""
-            text = open(filename)
+            text = pelican_open(source_path)
             md = Markdown(extensions = ['meta', 'codehilite'])
             content = md.convert(text)
-            
+
             metadata = {}
             for name, value in md.Meta.items():
-                if name in _METADATA_FIELDS:
+                name = name.lower()
-                    meta = _METADATA_FIELDS[name](value[0])
+                meta = self.process_metadata(name, value[0])
-                else:
+                metadata[name] = meta
-                    meta = value[0]
-                metadata[name.lower()] = meta
             return content, metadata
 
 Simple, isn't it?
@@ -81,7 +79,7 @@ both; only the existing ones will be called.
   context is shared between all generators, and will be passed to the
   templates. For instance, the ``PageGenerator`` ``generate_context`` method
   finds all the pages, transforms them into objects, and populates the context
-  with them. Be careful *not* to output anything using this context at this 
+  with them. Be careful *not* to output anything using this context at this
   stage, as it is likely to change by the effect of other generators.
 
 * ``generate_output`` is then called. And guess what is it made for? Oh,

docs/pelican-themes.rst

@@ -64,7 +64,7 @@ In this example, we can see there are three themes available: ``notmyidea``, ``s
 Note that you can combine the ``--list`` option with the ``-v`` or ``--verbose`` option to get more verbose output, like this:
 
 .. code-block:: console
-    
+
     $ pelican-themes -v -l
     /usr/local/lib/python2.6/dist-packages/pelican-2.6.0-py2.6.egg/pelican/themes/notmyidea
     /usr/local/lib/python2.6/dist-packages/pelican-2.6.0-py2.6.egg/pelican/themes/two-column (symbolic link to `/home/skami/Dev/Python/pelican-themes/two-column')
@@ -118,7 +118,7 @@ Creating symbolic links
 To symbolically link a theme, you can use the ``-s`` or ``--symlink``, which works exactly as the ``--install`` option:
 
 .. code-block:: console
-    
+
     # pelican-themes --symlink ~/Dev/Python/pelican-themes/two-column
 
 In this example, the ``two-column`` theme is now symbolically linked to the Pelican themes path, so we can use it, but we can also modify it without having to reinstall it after each modification.
@@ -130,11 +130,11 @@ This is useful for theme development:
     $ sudo pelican-themes -s ~/Dev/Python/pelican-themes/two-column
     $ pelican ~/Blog/content -o /tmp/out -t two-column
     $ firefox /tmp/out/index.html
-    $ vim ~/Dev/Pelican/pelican-themes/two-coumn/static/css/main.css 
+    $ vim ~/Dev/Pelican/pelican-themes/two-coumn/static/css/main.css
     $ pelican ~/Blog/content -o /tmp/out -t two-column
     $ cp /tmp/bg.png ~/Dev/Pelican/pelican-themes/two-coumn/static/img/bg.png
     $ pelican ~/Blog/content -o /tmp/out -t two-column
-    $ vim ~/Dev/Pelican/pelican-themes/two-coumn/templates/index.html 
+    $ vim ~/Dev/Pelican/pelican-themes/two-coumn/templates/index.html
     $ pelican ~/Blog/content -o /tmp/out -t two-column
 
 
@@ -152,7 +152,7 @@ The ``--install``, ``--remove`` and ``--symlink`` option are not mutually exclus
                      --symlink ~/Dev/Python/pelican-themes/two-column \
                      --verbose
 
-In this example, the theme ``notmyidea-cms`` is replaced by the theme ``notmyidea-cms-fr`` 
+In this example, the theme ``notmyidea-cms`` is replaced by the theme ``notmyidea-cms-fr``
 
 
 
@@ -162,5 +162,3 @@ See also
 
 -   http://docs.notmyidea.org/alexis/pelican/
 -   ``/usr/share/doc/pelican/`` if you have installed Pelican using the `APT repository <http://skami18.github.com/pelican-packages/>`_
-
-

docs/plugins.rst

@@ -3,25 +3,25 @@
 Plugins
 #######
 
-Since version 3.0, Pelican manages plugins. Plugins are a way to add features
+Beginning with version 3.0, Pelican supports plugins. Plugins are a way to add
-to Pelican without having to directly hack Pelican code.
+features to Pelican without having to directly modify the Pelican core.
 
-Pelican is shipped with a set of core plugins, but you can easily implement
+Pelican is shipped with a set of bundled plugins, but you can easily implement
-your own (and this page describes how).
+your own. This page describes how to use and create plugins.
 
 How to use plugins
 ==================
 
-To load plugins, you have to specify them in your settings file. You have two
+To load plugins, you have to specify them in your settings file. There are two
-ways to do so.
+ways to do so. The first method is to specify strings with the path to the
-Either by specifying strings with the path to the callables::
+callables::
 
-    PLUGINS = ['pelican.plugins.gravatar',] 
+    PLUGINS = ['pelican.plugins.gravatar',]
 
-Or by importing them and adding them to the list::
+Alternatively, another method is to import them and add them to the list::
 
     from pelican.plugins import gravatar
-    PLUGINS = [gravatar, ]
+    PLUGINS = [gravatar,]
 
 If your plugins are not in an importable path, you can specify a ``PLUGIN_PATH``
 in the settings::
@@ -33,7 +33,7 @@ How to create plugins
 =====================
 
 Plugins are based on the concept of signals. Pelican sends signals, and plugins
-subscribe to those signals. The list of signals are defined in a following
+subscribe to those signals. The list of signals are defined in a subsequent
 section.
 
 The only rule to follow for plugins is to define a ``register`` callable, in
@@ -48,47 +48,172 @@ which you map the signals to your plugin logic. Let's take a simple example::
         signals.initialized.connect(test)
 
 
+
 List of signals
 ===============
 
 Here is the list of currently implemented signals:
 
-=========================   ============================   =========================================
+=============================   ============================   ===========================================================================
-Signal                      Arguments                      Description
+Signal                          Arguments                       Description
-=========================   ============================   =========================================
+=============================   ============================   ===========================================================================
-initialized                 pelican object
+initialized                     pelican object
-article_generate_context    article_generator, metadata
+finalized                       pelican object                  invoked after all the generators are executed and just before pelican exits
-article_generator_init      article_generator              invoked in the ArticlesGenerator.__init__
+                                                                usefull for custom post processing actions, such as:
-=========================   ============================   =========================================
+                                                                - minifying js/css assets.
+                                                                - notify/ping search engines with an updated sitemap.
+generator_init                  generator                       invoked in the Generator.__init__
+article_generate_context        article_generator, metadata
+article_generate_preread        article_generator               invoked before a article is read in ArticlesGenerator.generate_context;
+                                                                use if code needs to do something before every article is parsed
+article_generator_init          article_generator               invoked in the ArticlesGenerator.__init__
+article_generator_finalized     article_generator               invoked at the end of ArticlesGenerator.generate_context
+get_generators                  generators                      invoked in Pelican.get_generator_classes,
+                                                                can return a Generator, or several
+                                                                generator in a tuple or in a list.
+pages_generate_context          pages_generator, metadata
+pages_generator_init            pages_generator                 invoked in the PagesGenerator.__init__
+=============================   ============================   ===========================================================================
 
-The list is currently small, don't hesitate to add signals and make a pull
+The list is currently small, so don't hesitate to add signals and make a pull
 request if you need them!
 
+.. note::
+
+   The signal ``content_object_init`` can send a different type of object as
+   the argument. If you want to register only one type of object then you will
+   need to specify the sender when you are connecting to the signal.
+
+   ::
+
+       from pelican import signals
+       from pelican import contents
+
+       def test(sender, instance):
+               print "%s : %s content initialized !!" % (sender, instance)
+
+       def register():
+               signals.content_object_init.connect(test, sender=contents.Article)
+
+
+
 List of plugins
 ===============
 
-Not all the list are described here, but a few of them have been extracted from
+The following plugins are currently included with Pelican:
-the Pelican core and provided in ``pelican.plugins``. They are described here:
 
-Tag cloud
+* `Asset management`_ ``pelican.plugins.assets``
----------
+* `GitHub activity`_ ``pelican.plugins.github_activity``
+* `Global license`_ ``pelican.plugins.global_license``
+* `Gravatar`_ ``pelican.plugins.gravatar``
+* `Gzip cache`_ ``pelican.plugins.gzip_cache``
+* `HTML tags for reStructuredText`_ ``pelican.plugins.html_rst_directive``
+* `Related posts`_ ``pelican.plugins.related_posts``
+* `Sitemap`_ ``pelican.plugins.sitemap``
 
-Translation
+Ideas for plugins that haven't been written yet:
------------
 
-Github Activity
+* Tag cloud
+* Translation
+
+Plugin descriptions
+===================
+
+Asset management
+----------------
+
+This plugin allows you to use the `Webassets`_ module to manage assets such as
+CSS and JS files. The module must first be installed::
+
+    pip install webassets
+
+The Webassets module allows you to perform a number of useful asset management
+functions, including:
+
+* CSS minifier (``cssmin``, ``yui_css``, ...)
+* CSS compiler (``less``, ``sass``, ...)
+* JS minifier (``uglifyjs``, ``yui_js``, ``closure``, ...)
+
+Others filters include gzip compression, integration of images in CSS via data
+URIs, and more. Webassets can also append a version identifier to your asset
+URL to convince browsers to download new versions of your assets when you use
+far-future expires headers. Please refer to the `Webassets documentation`_ for
+more information.
+
+When used with Pelican, Webassets is configured to process assets in the
+``OUTPUT_PATH/theme`` directory. You can use Webassets in your templates by
+including one or more template tags. The Jinja variable ``{{ ASSET_URL }}`` can
+be used in templates and is relative to the ``theme/`` url. The
+``{{ ASSET_URL }}`` variable should be used in conjunction with the
+``{{ SITEURL }}`` variable in order to generate URLs properly. For example:
+
+.. code-block:: jinja
+
+    {% assets filters="cssmin", output="css/style.min.css", "css/inuit.css", "css/pygment-monokai.css", "css/main.css" %}
+        <link rel="stylesheet" href="{{ SITEURL }}/{{ ASSET_URL }}">
+    {% endassets %}
+
+... will produce a minified css file with a version identifier that looks like:
+
+.. code-block:: html
+
+    <link href="http://{SITEURL}/theme/css/style.min.css?b3a7c807" rel="stylesheet">
+
+These filters can be combined. Here is an example that uses the SASS compiler
+and minifies the output:
+
+.. code-block:: jinja
+
+    {% assets filters="sass,cssmin", output="css/style.min.css", "css/style.scss" %}
+        <link rel="stylesheet" href="{{ SITEURL }}/{{ ASSET_URL }}">
+    {% endassets %}
+
+Another example for Javascript:
+
+.. code-block:: jinja
+
+    {% assets filters="uglifyjs,gzip", output="js/packed.js", "js/jquery.js", "js/base.js", "js/widgets.js" %}
+        <script src="{{ SITEURL }}/{{ ASSET_URL }}"></script>
+    {% endassets %}
+
+The above will produce a minified and gzipped JS file:
+
+.. code-block:: html
+
+    <script src="http://{SITEURL}/theme/js/packed.js?00703b9d"></script>
+
+Pelican's debug mode is propagated to Webassets to disable asset packaging
+and instead work with the uncompressed assets.
+
+Many of Webasset's available compilers have additional configuration options
+(i.e. 'Less', 'Sass', 'Stylus', 'Closure_js').  You can pass these options to
+Webassets using the ``ASSET_CONFIG`` in your settings file.
+
+The following will handle Google Closure's compilation level and locate
+LessCSS's binary:
+
+.. code-block:: python
+
+    ASSET_CONFIG = (('closure_compressor_optimization', 'WHITESPACE_ONLY'),
+                    ('less_bin', 'lessc.cmd'), )
+
+.. _Webassets: https://github.com/miracle2k/webassets
+.. _Webassets documentation: http://webassets.readthedocs.org/en/latest/builtin_filters.html
+
+
+GitHub activity
 ---------------
 
-This plugin makes use of the ``feedparser`` library that you'll need to
+This plugin makes use of the `feedparser`_ library that you'll need to
 install.
 
-Set the ``GITHUB_ACTIVITY_FEED`` parameter to your Github activity feed.
+Set the ``GITHUB_ACTIVITY_FEED`` parameter to your GitHub activity feed.
-For example, my setting would look like::
+For example, to track Pelican project activity, the setting would be::
 
-     GITHUB_ACTIVITY_FEED = 'https://github.com/kpanic.atom'
+     GITHUB_ACTIVITY_FEED = 'https://github.com/getpelican.atom'
 
-On the templates side, you just have to iterate over the ``github_activity``
+On the template side, you just have to iterate over the ``github_activity``
-variable, as in the example::
+variable, as in this example::
 
      {% if GITHUB_ACTIVITY_FEED %}
         <div class="social">
@@ -102,7 +227,146 @@ variable, as in the example::
         </div><!-- /.github_activity -->
      {% endif %}
 
+``github_activity`` is a list of lists. The first element is the title,
+and the second element is the raw HTML from GitHub.
 
+.. _feedparser: https://crate.io/packages/feedparser/
 
-``github_activity`` is a list of lists. The first element is the title
+Global license
-and the second element is the raw HTML from Github.
+--------------
+
+This plugin allows you to define a ``LICENSE`` setting and adds the contents of that
+license variable to the article's context, making that variable available to use
+from within your theme's templates.
+
+Gravatar
+--------
+
+This plugin assigns the ``author_gravatar`` variable to the Gravatar URL and
+makes the variable available within the article's context. You can add
+``AUTHOR_EMAIL`` to your settings file to define the default author's email
+address. Obviously, that email address must be associated with a Gravatar
+account.
+
+Alternatively, you can provide an email address from within article metadata::
+
+    :email:  john.doe@example.com
+
+If the email address is defined via at least one of the two methods above,
+the ``author_gravatar`` variable is added to the article's context.
+
+Gzip cache
+----------
+
+Certain web servers (e.g., Nginx) can use a static cache of gzip-compressed
+files to prevent the server from compressing files during an HTTP call. Since
+compression occurs at another time, these compressed files can be compressed
+at a higher compression level for increased optimization.
+
+The ``gzip_cache`` plugin compresses all common text type files into a ``.gz``
+file within the same directory as the original file.
+
+HTML tags for reStructuredText
+------------------------------
+
+This plugin allows you to use HTML tags from within reST documents. Following
+is a usage example, which is in this case a contact form::
+
+    .. html::
+
+        <form method="GET" action="mailto:some email">
+          <p>
+            <input type="text" placeholder="Subject" name="subject">
+            <br />
+            <textarea name="body" placeholder="Message">
+            </textarea>
+            <br />
+            <input type="reset"><input type="submit">
+          </p>
+        </form>
+
+Related posts
+-------------
+
+This plugin adds the ``related_posts`` variable to the article's context.
+To enable, add the following to your settings file::
+
+    from pelican.plugins import related_posts
+    PLUGINS = [related_posts]
+
+You can then use the ``article.related_posts`` variable in your templates.
+For example::
+
+    {% if article.related_posts %}
+        <ul>
+        {% for related_post in article.related_posts %}
+            <li><a href="{{ related_post.url }}">{{ related_post.title }}</a></li>
+        {% endfor %}
+        </ul>
+    {% endif %}
+
+Sitemap
+-------
+
+The sitemap plugin generates plain-text or XML sitemaps. You can use the
+``SITEMAP`` variable in your settings file to configure the behavior of the
+plugin.
+
+The ``SITEMAP`` variable must be a Python dictionary and can contain three keys:
+
+- ``format``, which sets the output format of the plugin (``xml`` or ``txt``)
+
+- ``priorities``, which is a dictionary with three keys:
+
+  - ``articles``, the priority for the URLs of the articles and their
+    translations
+
+  - ``pages``, the priority for the URLs of the static pages
+
+  - ``indexes``, the priority for the URLs of the index pages, such as tags,
+     author pages, categories indexes, archives, etc...
+
+  All the values of this dictionary must be decimal numbers between ``0`` and ``1``.
+
+- ``changefreqs``, which is a dictionary with three items:
+
+  - ``articles``, the update frequency of the articles
+
+  - ``pages``, the update frequency of the pages
+
+  - ``indexes``, the update frequency of the index pages
+
+  Valid frequency values are ``always``, ``hourly``, ``daily``, ``weekly``, ``monthly``,
+  ``yearly`` and ``never``.
+
+If a key is missing or a value is incorrect, it will be replaced with the
+default value.
+
+The sitemap is saved in ``<output_path>/sitemap.<format>``.
+
+.. note::
+   ``priorities`` and ``changefreqs`` are information for search engines.
+   They are only used in the XML sitemaps.
+   For more information: <http://www.sitemaps.org/protocol.html#xmlTagDefinitions>
+
+**Example**
+
+Here is an example configuration (it's also the default settings):
+
+.. code-block:: python
+
+    PLUGINS=['pelican.plugins.sitemap',]
+
+    SITEMAP = {
+        'format': 'xml',
+        'priorities': {
+            'articles': 0.5,
+            'indexes': 0.5,
+            'pages': 0.5
+        },
+        'changefreqs': {
+            'articles': 'monthly',
+            'indexes': 'daily',
+            'pages': 'monthly'
+        }
+    }

docs/report.rst

@@ -4,7 +4,7 @@ Some history about Pelican
 .. warning::
 
     This page comes from a report the original author (Alexis Métaireau) wrote
-    right after writing Pelican, in December 2010. The information may not be 
+    right after writing Pelican, in December 2010. The information may not be
     up-to-date.
 
 Pelican is a simple static blog generator. It parses markup files
@@ -113,7 +113,7 @@ concepts. Here is what happens when calling the ``generate_context``
 method:
 
 * Read the folder “path”, looking for restructured text files, load
-  each of them, and construct a content object (``Article``) with it. To do so, 
+  each of them, and construct a content object (``Article``) with it. To do so,
   use ``Reader`` objects.
 * Update the ``context`` with all those articles.
 

docs/settings.rst

@@ -8,10 +8,16 @@ the command line::
 
 Settings are configured in the form of a Python module (a file). You can see an
 example by looking at `/samples/pelican.conf.py
-<https://github.com/ametaireau/pelican/raw/master/samples/pelican.conf.py>`_
+<https://github.com/getpelican/pelican/raw/master/samples/pelican.conf.py>`_
 
 All the setting identifiers must be set in all-caps, otherwise they will not be
-processed.
+processed. Setting values that are numbers (5, 20, etc.), booleans (True,
+False, None, etc.), dictionaries, or tuples should *not* be enclosed in
+quotation marks. All other values (i.e., strings) *must* be enclosed in
+quotation marks.
+
+Unless otherwise specified, settings that refer to paths can be either absolute
+or relative to the configuration file.
 
 The settings you define in the configuration file will be passed to the
 templates, which allows you to use your settings to add site-wide content.
@@ -25,40 +31,65 @@ Basic settings
 Setting name (default value)                                            What does it do?
 =====================================================================   =====================================================================
 `AUTHOR`                                                                Default author (put your name)
-`DATE_FORMATS` (``{}``)                                                 If you do manage multiple languages, you can
+`DATE_FORMATS` (``{}``)                                                 If you manage multiple languages, you can set the date formatting
-                                                                        set the date formatting here. See "Date format and locales"
+                                                                        here. See the "Date format and locales" section below for details.
-                                                                        section below for details.
+`USE_FOLDER_AS_CATEGORY` (``True``)                                     When you don't specify a category in your post metadata, set this
+                                                                        setting to ``True``, and organize your articles in subfolders, the
+                                                                        subfolder will become the category of your post. If set to ``False``,
+                                                                        ``DEFAULT_CATEGORY`` will be used as a fallback.
 `DEFAULT_CATEGORY` (``'misc'``)                                         The default category to fall back on.
 `DEFAULT_DATE_FORMAT` (``'%a %d %B %Y'``)                               The default date format you want to use.
 `DISPLAY_PAGES_ON_MENU` (``True``)                                      Whether to display pages on the menu of the
                                                                         template. Templates may or not honor this
                                                                         setting.
-`FALLBACK_ON_FS_DATE` (``True``)                                        If True, Pelican will use the file system
+`DEFAULT_DATE` (``None``)                                               The default date you want to use.
+                                                                        If ``fs``, Pelican will use the file system
                                                                         timestamp information (mtime) if it can't get
                                                                         date information from the metadata.
-`JINJA_EXTENSIONS` (``[]``)                                             A list of any Jinja2 extensions you want to use.
+                                                                        If set to a tuple object, the default datetime object will instead
+                                                                        be generated by passing the tuple to the
+                                                                        ``datetime.datetime`` constructor.
+`DEFAULT_METADATA` (``()``)                                             The default metadata you want to use for all articles
+                                                                        and pages.
+`FILENAME_METADATA` (``'(?P<date>\d{4}-\d{2}-\d{2}).*'``)               The regexp that will be used to extract any metadata
+                                                                        from the filename. All named groups that are matched
+                                                                        will be set in the metadata object.
+                                                                        The default value will only extract the date from
+                                                                        the filename.
+                                                                        For example, if you would like to extract both the
+                                                                        date and the slug, you could set something like:
+                                                                        ``'(?P<date>\d{4}-\d{2}-\d{2})_(?P<slug>.*)'``.
 `DELETE_OUTPUT_DIRECTORY` (``False``)                                   Delete the content of the output directory before
                                                                         generating new files.
+`FILES_TO_COPY` (``()``)                                                A list of files (or directories) to copy from the source (inside the
+                                                                        content directory) to the destination (inside the output directory).
+                                                                        For example: ``(('extra/robots.txt', 'robots.txt'),)``.
+`JINJA_EXTENSIONS` (``[]``)                                             A list of any Jinja2 extensions you want to use.
 `LOCALE` (''[#]_)                                                       Change the locale. A list of locales can be provided
                                                                         here or a single string representing one locale.
                                                                         When providing a list, all the locales will be tried
                                                                         until one works.
 `MARKUP` (``('rst', 'md')``)                                            A list of available markup languages you want
                                                                         to use. For the moment, the only available values
-                                                                        are `rst` and `md`.
+                                                                        are `rst`, `md`, `markdown`, `mkd`, `html`, and `htm`.
 `MD_EXTENSIONS` (``['codehilite','extra']``)                            A list of the extensions that the Markdown processor
                                                                         will use. Refer to the extensions chapter in the
                                                                         Python-Markdown documentation for a complete list of
                                                                         supported extensions.
 `OUTPUT_PATH` (``'output/'``)                                           Where to output the generated files.
-`PATH` (``None``)                                                       Path to look at for input files.
+`PATH` (``None``)                                                       Path to content directory to be processed by Pelican.
-`PAGE_DIR` (``'pages'``)                                                Directory to look at for pages.
+`PAGE_DIR` (``'pages'``)                                                Directory to look at for pages, relative to `PATH`.
 `PAGE_EXCLUDES` (``()``)                                                A list of directories to exclude when looking for pages.
-`ARTICLE_DIR` (``''``)                                                  Directory to look at for articles.
+`ARTICLE_DIR` (``''``)                                                  Directory to look at for articles, relative to `PATH`.
 `ARTICLE_EXCLUDES`: (``('pages',)``)                                    A list of directories to exclude when looking for articles.
-`PDF_GENERATOR` (``False``)                                             Set to True if you want to have PDF versions
+`PDF_GENERATOR` (``False``)                                             Set to ``True`` if you want PDF versions of your documents to be.
-                                                                        of your documents. You will need to install
+                                                                        generated. You will need to install ``rst2pdf``.
-                                                                        `rst2pdf`.
+`OUTPUT_SOURCES` (``False``)                                            Set to True if you want to copy the articles and pages in their
+                                                                        original format (e.g. Markdown or reStructuredText) to the
+                                                                        specified OUTPUT_PATH.
+`OUTPUT_SOURCES_EXTENSION` (``.text``)                                  Controls the extension that will be used by the SourcesGenerator.
+                                                                        Defaults to ``.text``. If not a valid string the default value
+                                                                        will be used.
 `RELATIVE_URLS` (``True``)                                              Defines whether Pelican should use document-relative URLs or
                                                                         not. If set to ``False``, Pelican will use the SITEURL
                                                                         setting to construct absolute URLs.
@@ -66,41 +97,42 @@ Setting name (default value)                                            What doe
 `SITENAME` (``'A Pelican Blog'``)                                       Your site name
 `SITEURL`                                                               Base URL of your website. Not defined by default,
                                                                         so it is best to specify your SITEURL; if you do not, feeds
-                                                                        will not be generated with properly-formed URLs. You should 
+                                                                        will not be generated with properly-formed URLs. You should
                                                                         include ``http://`` and your domain, with no trailing
                                                                         slash at the end. Example: ``SITEURL = 'http://mydomain.com'``
+`TEMPLATE_PAGES` (``None``)                                             A mapping containing template pages that will be rendered with
+                                                                        the blog entries. See :ref:`template_pages`.
 `STATIC_PATHS` (``['images']``)                                         The static paths you want to have accessible
                                                                         on the output path "static". By default,
-                                                                        Pelican will copy the 'images' folder to the
+                                                                        Pelican will copy the "images" folder to the
                                                                         output folder.
 `TIMEZONE`                                                              The timezone used in the date information, to
-                                                                        generate Atom and RSS feeds. See the "timezone"
+                                                                        generate Atom and RSS feeds. See the *Timezone*
                                                                         section below for more info.
-`TYPOGRIFY` (``False``)                                                 If set to true, some
+`TYPOGRIFY` (``False``)                                                 If set to True, several typographical improvements will be
-                                                                        additional transformations will be done on the
+                                                                        incorporated into the generated HTML via the `Typogrify
-                                                                        generated HTML, using the `Typogrify
                                                                         <http://static.mintchaos.com/projects/typogrify/>`_
-                                                                        library
+                                                                        library, which can be installed via: ``pip install typogrify``
-`LESS_GENERATOR` (``FALSE``)                                            Set to True or complete path to `lessc` (if not
-                                                                        found in system PATH) to enable compiling less
-                                                                        css files. Requires installation of `less css`_.
 `DIRECT_TEMPLATES` (``('index', 'tags', 'categories', 'archives')``)    List of templates that are used directly to render
                                                                         content. Typically direct templates are used to generate
-                                                                        index pages for collections of content e.g. tags and
+                                                                        index pages for collections of content (e.g. tags and
-                                                                        category index pages.
+                                                                        category index pages).
 `PAGINATED_DIRECT_TEMPLATES` (``('index',)``)                           Provides the direct templates that should be paginated.
-`SUMMARY_MAX_LENGTH` (``50``)                                           When creating a short summary of an article, this will 
+`SUMMARY_MAX_LENGTH` (``50``)                                           When creating a short summary of an article, this will
                                                                         be the default length in words of the text created.
-                                                                        This only applies if your content does not otherwise 
+                                                                        This only applies if your content does not otherwise
-                                                                        specify a summary. Setting to None will cause the summary 
+                                                                        specify a summary. Setting to ``None`` will cause the summary
                                                                         to be a copy of the original content.
-                                                                        
+`EXTRA_TEMPLATES_PATHS` (``[]``)                                        A list of paths you want Jinja2 to search for templates.
+                                                                        Can be used to separate templates from the theme.
+                                                                        Example: projects, resume, profile ...
+                                                                        These templates need to use ``DIRECT_TEMPLATES`` setting.
+`ASCIIDOC_OPTIONS` (``[]``)                                             A list of options to pass to AsciiDoc. See the `manpage
+                                                                        <http://www.methods.co.nz/asciidoc/manpage.html>`_
 =====================================================================   =====================================================================
 
 .. [#] Default is the system locale.
 
-.. _less css: http://lesscss.org/
-
 
 URL settings
 ------------
@@ -111,13 +143,13 @@ when testing locally, and absolute URLs are reliable and most useful when
 publishing. One method of supporting both is to have one Pelican configuration
 file for local development and another for publishing. To see an example of this
 type of setup, use the ``pelican-quickstart`` script as described at the top of
-the :doc:`Getting Started<getting_started>` page, which will produce two separate
+the :doc:`Getting Started <getting_started>` page, which will produce two separate
 configuration files for local development and publishing, respectively.
 
 You can customize the URLs and locations where files will be saved. The URLs and
 SAVE_AS variables use Python's format strings. These variables allow you to place
-your articles in a location such as '{slug}/index.html' and link to them as
+your articles in a location such as ``{slug}/index.html`` and link to them as
-'{slug}' for clean URLs. These settings give you the flexibility to place your
+``{slug}`` for clean URLs. These settings give you the flexibility to place your
 articles and pages anywhere you want.
 
 .. note::
@@ -138,41 +170,41 @@ Also, you can use other file metadata attributes as well:
 
 Example usage:
 
-* ARTICLE_URL = 'posts/{date:%Y}/{date:%b}/{date:%d}/{slug}/'
+* ARTICLE_URL = ``'posts/{date:%Y}/{date:%b}/{date:%d}/{slug}/'``
-* ARTICLE_SAVE_AS = 'posts/{date:%Y}/{date:%b}/{date:%d}/{slug}/index.html'
+* ARTICLE_SAVE_AS = ``'posts/{date:%Y}/{date:%b}/{date:%d}/{slug}/index.html'``
 
-This would save your articles in something like '/posts/2011/Aug/07/sample-post/index.html',
+This would save your articles in something like ``/posts/2011/Aug/07/sample-post/index.html``,
-and the URL to this would be '/posts/2011/Aug/07/sample-post/'.
+and the URL to this would be ``/posts/2011/Aug/07/sample-post/``.
 
-================================================    =====================================================
+====================================================    =====================================================
-Setting name (default value)                        what does it do?
+Setting name (default value)                            What does it do?
-================================================    =====================================================
+====================================================    =====================================================
-`ARTICLE_URL` ('{slug}.html')                       The URL to refer to an ARTICLE.
+`ARTICLE_URL` (``'{slug}.html'``)                       The URL to refer to an ARTICLE.
-`ARTICLE_SAVE_AS` ('{slug}.html')                   The place where we will save an article.
+`ARTICLE_SAVE_AS` (``'{slug}.html'``)                   The place where we will save an article.
-`ARTICLE_LANG_URL` ('{slug}-{lang}.html')           The URL to refer to an ARTICLE which doesn't use the
+`ARTICLE_LANG_URL` (``'{slug}-{lang}.html'``)           The URL to refer to an ARTICLE which doesn't use the
-                                                    default language.
+                                                        default language.
-`ARTICLE_LANG_SAVE_AS` ('{slug}-{lang}.html'        The place where we will save an article which
+`ARTICLE_LANG_SAVE_AS` (``'{slug}-{lang}.html'``)       The place where we will save an article which
-                                                    doesn't use the default language.
+                                                        doesn't use the default language.
-`PAGE_URL` ('pages/{slug}.html')                    The URL we will use to link to a page.
+`PAGE_URL` (``'pages/{slug}.html'``)                    The URL we will use to link to a page.
-`PAGE_SAVE_AS` ('pages/{slug}.html')                The location we will save the page.
+`PAGE_SAVE_AS` (``'pages/{slug}.html'``)                The location we will save the page.
-`PAGE_LANG_URL` ('pages/{slug}-{lang}.html')        The URL we will use to link to a page which doesn't
+`PAGE_LANG_URL` (``'pages/{slug}-{lang}.html'``)        The URL we will use to link to a page which doesn't
-                                                    use the default language.
+                                                        use the default language.
-`PAGE_LANG_SAVE_AS` ('pages/{slug}-{lang}.html')    The location we will save the page which doesn't
+`PAGE_LANG_SAVE_AS` (``'pages/{slug}-{lang}.html'``)    The location we will save the page which doesn't
-                                                    use the default language.
+                                                        use the default language.
-`AUTHOR_URL` ('author/{name}.html')                 The URL to use for an author.
+`AUTHOR_URL` (``'author/{slug}.html'``)                 The URL to use for an author.
-`AUTHOR_SAVE_AS` ('author/{name}.html')             The location to save an author.
+`AUTHOR_SAVE_AS` (``'author/{slug}.html'``)             The location to save an author.
-`CATEGORY_URL` ('category/{name}.html')             The URL to use for a category.
+`CATEGORY_URL` (``'category/{slug}.html'``)             The URL to use for a category.
-`CATEGORY_SAVE_AS` ('category/{name}.html')         The location to save a category.
+`CATEGORY_SAVE_AS` (``'category/{slug}.html'``)         The location to save a category.
-`TAG_URL` ('tag/{name}.html')                       The URL to use for a tag.
+`TAG_URL` (``'tag/{slug}.html'``)                       The URL to use for a tag.
-`TAG_SAVE_AS` ('tag/{name}.html')                   The location to save the tag page.
+`TAG_SAVE_AS` (``'tag/{slug}.html'``)                   The location to save the tag page.
-`<DIRECT_TEMPLATE_NAME>_SAVE_AS`                    The location to save content generated from direct
+`<DIRECT_TEMPLATE_NAME>_SAVE_AS`                        The location to save content generated from direct
-                                                    templates. Where <DIRECT_TEMPLATE_NAME> is the
+                                                        templates. Where <DIRECT_TEMPLATE_NAME> is the
-                                                    upper case template name.
+                                                        upper case template name.
-================================================    =====================================================
+====================================================    =====================================================
 
 .. note::
 
-    When any of `*_SAVE_AS` is set to False, files will not be created.
+    When any of the `*_SAVE_AS` settings is set to False, files will not be created.
 
 Timezone
 --------
@@ -191,14 +223,14 @@ Have a look at `the wikipedia page`_ to get a list of valid timezone values.
 Date format and locale
 ----------------------
 
-If no DATE_FORMAT is set, fall back to DEFAULT_DATE_FORMAT. If you need to
+If no DATE_FORMATS is set, fall back to DEFAULT_DATE_FORMAT. If you need to
 maintain multiple languages with different date formats, you can set this dict
 using language name (``lang`` in your posts) as key. Regarding available format
 codes, see `strftime document of python`_ :
 
 .. parsed-literal::
 
-    DATE_FORMAT = {
+    DATE_FORMATS = {
         'en': '%a, %d %b %Y',
         'jp': '%Y-%m-%d(%a)',
     }
@@ -217,13 +249,13 @@ above:
 
 .. parsed-literal::
     # On Unix/Linux
-    DATE_FORMAT = {
+    DATE_FORMATS = {
         'en': ('en_US','%a, %d %b %Y'),
         'jp': ('ja_JP','%Y-%m-%d(%a)'),
     }
 
     # On Windows
-    DATE_FORMAT = {
+    DATE_FORMATS = {
         'en': ('usa','%a, %d %b %Y'),
         'jp': ('jpn','%Y-%m-%d(%a)'),
     }
@@ -239,6 +271,23 @@ can get a list of available locales via the ``locale -a`` command; see manpage
 
 .. _locale(1): http://linux.die.net/man/1/locale
 
+
+.. _template_pages:
+
+Template pages
+==============
+
+If you want to generate custom pages besides your blog entries, you can point
+any Jinja2 template file with a path pointing to the file and the destination
+path for the generated file.
+
+For instance, if you have a blog with three static pages — a list of books,
+your resume, and a contact page — you could have::
+
+    TEMPLATE_PAGES = {'src/books.html': 'dest/books.html',
+                      'src/resume.html': 'dest/resume.html',
+                      'src/contact.html': 'dest/contact.html'}
+
 Feed settings
 =============
 
@@ -247,7 +296,7 @@ feeds if you prefer.
 
 Pelican generates category feeds as well as feeds for all your articles. It does
 not generate feeds for tags by default, but it is possible to do so using
-the ``TAG_FEED`` and ``TAG_FEED_RSS`` settings:
+the ``TAG_FEED_ATOM`` and ``TAG_FEED_RSS`` settings:
 
 ================================================    =====================================================
 Setting name (default value)                        What does it do?
@@ -257,20 +306,25 @@ Setting name (default value)                        What does it do?
                                                     to define this (e.g., "http://feeds.example.com"). If
                                                     you have already explicitly defined SITEURL (see
                                                     above) and want to use the same domain for your
-                                                    feeds, you can just set:  `FEED_DOMAIN = SITEURL`
+                                                    feeds, you can just set:  ``FEED_DOMAIN = SITEURL``.
-`FEED` (``'feeds/all.atom.xml'``)                   Relative URL to output the Atom feed.
+`FEED_ATOM` (``None``, i.e. no Atom feed)           Relative URL to output the Atom feed.
 `FEED_RSS` (``None``, i.e. no RSS)                  Relative URL to output the RSS feed.
-`CATEGORY_FEED` ('feeds/%s.atom.xml'[2]_)           Where to put the category Atom feeds.
+`FEED_ALL_ATOM` (``'feeds/all.atom.xml'``)          Relative URL to output the all posts Atom feed:
+                                                    this feed will contain all posts regardless of their
+                                                    language.
+`FEED_ALL_RSS` (``None``, i.e. no all RSS)          Relative URL to output the all posts RSS feed:
+                                                    this feed will contain all posts regardless of their
+                                                    language.
+`CATEGORY_FEED_ATOM` ('feeds/%s.atom.xml'[2]_)      Where to put the category Atom feeds.
 `CATEGORY_FEED_RSS` (``None``, i.e. no RSS)         Where to put the category RSS feeds.
-`TAG_FEED` (``None``, i.e. no tag feed)             Relative URL to output the tag Atom feed. It should
+`TAG_FEED_ATOM` (``None``, i.e. no tag feed)        Relative URL to output the tag Atom feed. It should
                                                     be defined using a "%s" match in the tag name.
 `TAG_FEED_RSS` (``None``, ie no RSS tag feed)       Relative URL to output the tag RSS feed
 `FEED_MAX_ITEMS`                                    Maximum number of items allowed in a feed. Feed item
                                                     quantity is unrestricted by default.
 ================================================    =====================================================
 
-If you don't want to generate some of these feeds, set ``None`` to the
+If you don't want to generate some or any of these feeds, set the above variables to ``None``.
-variables above.
 
 .. [2] %s is the name of the category.
 
@@ -281,7 +335,7 @@ If you want to use FeedBurner for your feed, you will likely need to decide
 upon a unique identifier. For example, if your site were called "Thyme" and
 hosted on the www.example.com domain, you might use "thymefeeds" as your
 unique identifier, which we'll use throughout this section for illustrative
-purposes. In your Pelican settings, set the `FEED` attribute to
+purposes. In your Pelican settings, set the `FEED_ATOM` attribute to
 "thymefeeds/main.xml" to create an Atom feed with an original address of
 `http://www.example.com/thymefeeds/main.xml`. Set the `FEED_DOMAIN` attribute
 to `http://feeds.feedburner.com`, or `http://feeds.example.com` if you are
@@ -306,10 +360,10 @@ You can use the following settings to configure the pagination.
 ================================================    =====================================================
 Setting name (default value)                        What does it do?
 ================================================    =====================================================
-`DEFAULT_ORPHANS` (0)                               The minimum number of articles allowed on the
+`DEFAULT_ORPHANS` (``0``)                           The minimum number of articles allowed on the
                                                     last page. Use this when you don't want to
                                                     have a last page with very few articles.
-`DEFAULT_PAGINATION` (False)                        The maximum number of articles to include on a
+`DEFAULT_PAGINATION` (``False``)                    The maximum number of articles to include on a
                                                     page, not including orphans. False to disable
                                                     pagination.
 ================================================    =====================================================
@@ -323,9 +377,9 @@ following settings.
 ================================================    =====================================================
 Setting name (default value)                        What does it do?
 ================================================    =====================================================
-`TAG_CLOUD_STEPS` (4)                               Count of different font sizes in the tag
+`TAG_CLOUD_STEPS` (``4``)                           Count of different font sizes in the tag
                                                     cloud.
-`TAG_CLOUD_MAX_ITEMS` (100)                         Maximum number of tags in the cloud.
+`TAG_CLOUD_MAX_ITEMS` (``100``)                     Maximum number of tags in the cloud.
 ================================================    =====================================================
 
 The default theme does not support tag clouds, but it is pretty easy to add::
@@ -342,15 +396,16 @@ N matches `TAG_CLOUD_STEPS` -1).
 Translations
 ============
 
-Pelican offers a way to translate articles. See the Getting Started section for
+Pelican offers a way to translate articles. See the :doc:`Getting Started <getting_started>` section for
 more information.
 
-================================================    =====================================================
+=====================================================    =====================================================
-Setting name (default value)                        What does it do?
+Setting name (default value)                             What does it do?
-================================================    =====================================================
+=====================================================    =====================================================
-`DEFAULT_LANG` (``'en'``)                           The default language to use.
+`DEFAULT_LANG` (``'en'``)                                The default language to use.
-`TRANSLATION_FEED` ('feeds/all-%s.atom.xml'[3]_)    Where to put the feed for translations.
+`TRANSLATION_FEED_ATOM` ('feeds/all-%s.atom.xml'[3]_)    Where to put the Atom feed for translations.
-================================================    =====================================================
+`TRANSLATION_FEED_RSS` (``None``, i.e. no RSS)           Where to put the RSS feed for translations.
+=====================================================    =====================================================
 
 .. [3] %s is the language
 
@@ -360,48 +415,57 @@ Ordering content
 ================================================    =====================================================
 Setting name (default value)                        What does it do?
 ================================================    =====================================================
-`REVERSE_ARCHIVE_ORDER` (``False``)                 Reverse the archives list order. (True: orders by date
+`NEWEST_FIRST_ARCHIVES` (``True``)                  Order archives by newest first by date. (False:
-                                                    in descending order, with newer articles first.)
+                                                    orders by date with older articles first.)
 `REVERSE_CATEGORY_ORDER` (``False``)                Reverse the category order. (True: lists by reverse
                                                     alphabetical order; default lists alphabetically.)
 ================================================    =====================================================
 
-Theming
+Themes
-=======
+======
 
-Theming is addressed in a dedicated section (see :ref:`theming-pelican`).
+Creating Pelican themes is addressed in a dedicated section (see :ref:`theming-pelican`).
-However, here are the settings that are related to theming.
+However, here are the settings that are related to themes.
 
 ================================================    =====================================================
 Setting name (default value)                        What does it do?
 ================================================    =====================================================
-`THEME`                                             Theme to use to produce the output. Can be the
+`THEME`                                             Theme to use to produce the output. Can be a relative
-                                                    complete static path to a theme folder, or
+                                                    or absolute path to a theme folder, or the name of a
-                                                    chosen between the list of default themes (see
+                                                    default theme or a theme installed via
-                                                    below)
+                                                    ``pelican-themes`` (see below).
 `THEME_STATIC_PATHS` (``['static']``)               Static theme paths you want to copy. Default
                                                     value is `static`, but if your theme has
                                                     other static paths, you can put them here.
 `CSS_FILE` (``'main.css'``)                         Specify the CSS file you want to load.
-`WEBASSETS` (``False``)                             Asset management with `webassets` (see below)
 ================================================    =====================================================
 
-By default, two themes are available. You can specify them using the `-t` option:
+
+By default, two themes are available. You can specify them using the `THEME` setting or by passing the
+``-t`` option to the ``pelican`` command:
 
 * notmyidea
-* simple (a synonym for "full text" :)
+* simple (a synonym for "plain text" :)
-
-You can define your own theme too, and specify its placement in the same
-manner. (Be sure to specify the full absolute path to it.)
-
-Here is :doc:`a guide on how to create your theme <themes>`
-
-You can find a list of themes at http://github.com/ametaireau/pelican-themes.
 
+There are a number of other themes available at http://github.com/getpelican/pelican-themes.
 Pelican comes with :doc:`pelican-themes`, a small script for managing themes.
 
-The `notmyidea` theme can make good use of the following settings. I recommend
+You can define your own theme, either by starting from scratch or by duplicating
-using them in your themes as well.
+and modifying a pre-existing theme. Here is :doc:`a guide on how to create your theme <themes>`.
+
+Following are example ways to specify your preferred theme::
+
+    # Specify name of a built-in theme
+    THEME = "notmyidea"
+    # Specify name of a theme installed via the pelican-themes tool
+    THEME = "chunk"
+    # Specify a customized theme, via path relative to the settings file
+    THEME = "themes/mycustomtheme"
+    # Specify a customized theme, via absolute path
+    THEME = "~/projects/mysite/themes/mycustomtheme"
+
+The built-in ``notmyidea`` theme can make good use of the following settings. Feel
+free to use them in your themes as well.
 
 =======================   =======================================================
 Setting name              What does it do ?
@@ -429,64 +493,11 @@ Setting name              What does it do ?
                           if you want this button to appear.
 =======================   =======================================================
 
-In addition, you can use the "wide" version of the `notmyidea` theme by
+In addition, you can use the "wide" version of the ``notmyidea`` theme by
 adding the following to your configuration::
 
     CSS_FILE = "wide.css"
 
-Asset management
-----------------
-
-The `WEBASSETS` setting allows to use the `webassets`_ module to manage assets
-(css, js). The module must first be installed::
-
-    pip install webassets
-
-`webassets` allows to concatenate your assets and to use almost all of the
-hype tools of the moment (see the `documentation`_):
-
-* css minifier (`cssmin`, `yuicompressor`, ...)
-* css compiler (`less`, `sass`, ...)
-* js minifier (`uglifyjs`, `yuicompressor`, `closure`, ...)
-
-Others filters include gzip compression, integration of images in css with
-`datauri` and more. Webassets also append a version identifier to your asset
-url to convince browsers to download new versions of your assets when you use
-far future expires headers.
-
-When using it with Pelican, `webassets` is configured to process assets in the
-``OUTPUT_PATH/theme`` directory. You can use it in your templates with a
-template tag, for example:
-
-.. code-block:: jinja
-
-    {% assets filters="cssmin", output="css/style.min.css", "css/inuit.css", "css/pygment-monokai.css", "css/main.css" %}
-        <link rel="stylesheet" href="{{ ASSET_URL }}">
-    {% endassets %}
-
-will produce a minified css file with the version identifier:
-
-.. code-block:: html
-
-    <link href="http://{SITEURL}/theme/css/style.min.css?b3a7c807" rel="stylesheet">
-
-Another example for javascript:
-
-.. code-block:: jinja
-
-    {% assets filters="uglifyjs,gzip", output="js/packed.js", "js/jquery.js", "js/base.js", "js/widgets.js" %}
-        <script src="{{ ASSETS_URL }}"></script>
-    {% endassets %}
-
-will produce a minified and gzipped js file:
-
-.. code-block:: html
-
-    <script src="http://{SITEURL}/theme/js/packed.js?00703b9d"></script>
-
-.. _webassets: https://github.com/miracle2k/webassets
-.. _documentation: http://webassets.readthedocs.org/en/latest/builtin_filters.html
-
 Example settings
 ================
 

docs/themes.rst

@@ -6,7 +6,7 @@ How to create themes for Pelican
 Pelican uses the great `Jinja2 <http://jinja.pocoo.org/>`_ templating engine to
 generate its HTML output. Jinja2 syntax is really simple. If you want to
 create your own theme, feel free to take inspiration from the `"simple" theme
-<https://github.com/ametaireau/pelican/tree/master/pelican/themes/simple/templates>`_.
+<https://github.com/getpelican/pelican/tree/master/pelican/themes/simple/templates>`_.
 
 Structure
 =========
@@ -29,13 +29,13 @@ To make your own theme, you must follow the following structure::
         └── tags.html        // must list all the tags. Can be a tag cloud.
 
 * `static` contains all the static assets, which will be copied to the output
-  `theme/static` folder. I've put the CSS and image folders here, but they are
+  `theme` folder. I've put the CSS and image folders here, but they are
   just examples. Put what you need here.
 
 * `templates` contains all the templates that will be used to generate the content.
   I've just put the mandatory templates here; you can define your own if it helps
   you keep things organized while creating your theme.
- 
+
 Templates and variables
 =======================
 
@@ -44,7 +44,7 @@ This document describes which templates should exist in a theme, and which
 variables will be passed to each template at generation time.
 
 All templates will receive the variables defined in your settings file, if they
-are in all-caps. You can access them directly. 
+are in all-caps. You can access them directly.
 
 Common variables
 ----------------
@@ -55,18 +55,32 @@ All of these settings will be available to all templates.
 Variable        Description
 =============   ===================================================
 articles        The list of articles, ordered descending by date
-                All the elements are `Article` objects, so you can 
+                All the elements are `Article` objects, so you can
                 access their attributes (e.g. title, summary, author
                 etc.)
 dates           The same list of articles, but ordered by date,
                 ascending
-tags            A key-value dict containing the tags (the keys) and
+tags            A list of (tag, articles) tuples, containing all
-                the list of respective articles (the values)
+                the tags.
-categories      A key-value dict containing the categories (keys) 
+categories      A list of (category, articles) tuples, containing
+                all the categories.
                 and the list of respective articles (values)
 pages           The list of pages
 =============   ===================================================
 
+Sorting
+-------
+
+URL wrappers (currently categories, tags, and authors), have
+comparison methods that allow them to be easily sorted by name:
+
+    {% for tag, articles in tags|sort %}
+
+If you want to sort based on different criteria, `Jinja's sort
+command`__ has a number of options.
+
+__ http://jinja.pocoo.org/docs/templates/#sort
+
 index.html
 ----------
 
@@ -92,8 +106,8 @@ author.html
 This template will be processed for each of the existing authors, with
 output generated at output/author/`author_name`.html.
 
-If pagination is active, subsequent pages will reside at
+If pagination is active, subsequent pages will reside as defined by setting
-output/author/`author_name``n`.html.
+AUTHOR_SAVE_AS (`Default:` output/author/`author_name'n'`.html).
 
 ===================     ===================================================
 Variable                Description
@@ -108,8 +122,8 @@ dates_paginator         A paginator object for the article list, ordered by
                         date, ascending.
 dates_page              The current page of articles, ordered by date,
                         ascending.
-page_name               'author/`author_name`' -- useful for pagination
+page_name               AUTHOR_URL where everything after `{slug}` is
-                        links
+                        removed -- useful for pagination links
 ===================     ===================================================
 
 category.html
@@ -118,8 +132,8 @@ category.html
 This template will be processed for each of the existing categories, with
 output generated at output/category/`category_name`.html.
 
-If pagination is active, subsequent pages will reside at
+If pagination is active, subsequent pages will reside as defined by setting
-output/category/`category_name``n`.html.
+CATEGORY_SAVE_AS (`Default:` output/category/`category_name'n'`.html).
 
 ===================     ===================================================
 Variable                Description
@@ -134,8 +148,8 @@ dates_paginator         A paginator object for the list of articles,
                         ordered by date, ascending
 dates_page              The current page of articles, ordered by date,
                         ascending
-page_name               'category/`category_name`' -- useful for pagination
+page_name               CATEGORY_URL where everything after `{slug}` is
-                        links
+                        removed -- useful for pagination links
 ===================     ===================================================
 
 article.html
@@ -170,8 +184,8 @@ tag.html
 This template will be processed for each tag, with corresponding .html files
 saved as output/tag/`tag_name`.html.
 
-If pagination is active, subsequent pages will reside at
+If pagination is active, subsequent pages will reside as defined in setting
-output/tag/`tag_name``n`.html.
+TAG_SAVE_AS (`Default:` output/tag/`tag_name'n'`.html).
 
 ===================     ===================================================
 Variable                Description
@@ -182,13 +196,33 @@ dates                   Articles related to this tag, but ordered by date,
                         ascending
 articles_paginator      A paginator object for the list of articles
 articles_page           The current page of articles
-dates_paginator         A paginator object for the list of articles, 
+dates_paginator         A paginator object for the list of articles,
                         ordered by date, ascending
 dates_page              The current page of articles, ordered by date,
                         ascending
-page_name               'tag/`tag_name`' -- useful for pagination links
+page_name               TAG_URL where everything after `{slug}` is removed
+                        -- useful for pagination links
 ===================     ===================================================
 
+Feeds
+=====
+
+The feed variables changed in 3.0. Each variable now explicitly lists ATOM or
+RSS in the name. ATOM is still the default. Old themes will need to be updated.
+Here is a complete list of the feed variables::
+
+    FEED_ATOM
+    FEED_RSS
+    FEED_ALL_ATOM
+    FEED_ALL_RSS
+    CATEGORY_FEED_ATOM
+    CATEGORY_FEED_RSS
+    TAG_FEED_ATOM
+    TAG_FEED_RSS
+    TRANSLATION_FEED_ATOM
+    TRANSLATION_FEED_RSS
+
+
 Inheritance
 ===========
 

docs/tips.rst

@@ -6,47 +6,80 @@ Here are some tips about Pelican that you might find useful.
 Publishing to GitHub
 ====================
 
-GitHub comes with an interesting "pages" feature: you can upload things there
+`GitHub Pages <https://help.github.com/categories/20/articles>`_ offer an easy
-and it will be available directly from their servers. As Pelican is a static
+and convenient way to publish Pelican sites. There are `two types of GitHub
-file generator, we can take advantage of this.
+Pages <https://help.github.com/articles/user-organization-and-project-pages>`_:
-
+*Project Pages* and *User Pages*. Pelican sites can be published as both
-User Pages
+Project Pages and User Pages.
-----------
-GitHub allows you to create user pages in the form of ``username.github.com``. 
-Whatever is created in the master branch will be published. For this purpose,
-just the output generated by Pelican needs to pushed to GitHub.
-
-So given a repository containing your articles, just run Pelican over the posts 
-and deploy the master branch to GitHub::
-
-    $ pelican -s pelican.conf.py ./path/to/posts -o /path/to/output
-
-Now add all the files in the output directory generated by Pelican::
-
-    $ git add /path/to/output/*
-    $ git commit -am "Your Message"
-    $ git push origin master
 
 Project Pages
 -------------
-For creating Project pages, a branch called ``gh-pages`` is used for publishing.
-The excellent `ghp-import <https://github.com/davisp/ghp-import>`_ makes this
-really easy, which can be installed via::
 
-    $ pip install ghp-import
+To publish a Pelican site as Project Pages you need to *push* the content of
+the ``output`` dir generated by Pelican to a repository's ``gh-pages`` branch
+on GitHub.
 
-Then, given a repository containing your articles, you would simply run 
+The excellent `ghp-import <https://github.com/davisp/ghp-import>`_, which can
-Pelican and upload the output to GitHub::
+be installed with ``easy_install`` or ``pip``, makes this process really easy.
 
-    $ pelican -s pelican.conf.py .
+For example, if the sources of your Pelican site are contained in a GitHub
+repository, and if you want to publish your Pelican site as Project Pages of
+this repository, you can then use the following::
+
+    $ pelican content -o output pelicanconf.py
     $ ghp-import output
     $ git push origin gh-pages
 
-And that's it.
+The ``ghp-import output`` command updates the local ``gh-pages`` branch with
+the content of the ``output`` directory (creating the branch if it doesn't
+already exist). The ``git push origin gh-pages`` command updates the remote
+``gh-pages`` branch, effectively publishing the Pelican site.
 
-If you want, you can put that directly into a post-commit hook, so each time you
+.. note::
-commit, your blog is up-to-date on GitHub!
 
-Put the following into ``.git/hooks/post-commit``::
+    The ``github`` target of the Makefile created by the ``pelican-quickstart``
+    command publishes the Pelican site as Project Pages as described above.
 
-    pelican -s pelican.conf.py . && ghp-import output && git push origin gh-pages
+User Pages
+----------
+
+To publish a Pelican site as User Pages you need to *push* the content of the
+``output`` dir generated by Pelican to the ``master`` branch of your
+``<username>.github.com`` repository on GitHub.
+
+Again, you can take advantage of ``ghp-import``::
+
+    $ pelican content -o output pelicanconf.py
+    $ ghp-import output
+    $ git push git@github.com:elemoine/elemoine.github.com.git gh-pages:master
+
+The ``git push`` command pushes the local ``gh-pages`` branch (freshly updated
+by the ``ghp-import`` command) to the ``elemoine.github.com`` repository's
+``master`` branch on GitHub.
+
+.. note::
+
+    To publish your Pelican site as User Pages feel free to adjust the the
+    ``github`` target of the Makefile.
+
+Extra Tips
+----------
+
+Tip #1:
+
+To automatically update your Pelican site on each commit you can create
+a post-commit hook. For example, you can add the following to
+``.git/hooks/post-commit``::
+
+    pelican pelican content -o output pelicanconf.py && ghp-import output && git push origin gh-pages
+
+Tip #2:
+
+To use a `custom domain
+<https://help.github.com/articles/setting-up-a-custom-domain-with-pages>`_ with
+GitHub Pages you need to have a ``CNAME`` file at the root of your pages. For
+that you will add ``CNAME`` file to your ``content``, dir and use the
+``FILES_TO_COPY`` setting variable to tell Pelican to copy that file
+to the ``output`` dir. For example::
+
+    FILES_TO_COPY = (('extra/CNAME', 'CNAME'),)

pelican/__init__.py

@@ -1,3 +1,7 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
+import six
+
 import os
 import re
 import sys
@@ -8,70 +12,59 @@ import argparse
 from pelican import signals
 
 from pelican.generators import (ArticlesGenerator, PagesGenerator,
-        StaticGenerator, PdfGenerator, LessCSSGenerator)
+                                StaticGenerator, PdfGenerator,
+                                SourceFileGenerator, TemplatePagesGenerator)
 from pelican.log import init
-from pelican.settings import read_settings, _DEFAULT_CONFIG
+from pelican.settings import read_settings
-from pelican.utils import clean_output_dir, files_changed, file_changed
+from pelican.utils import (clean_output_dir, files_changed, file_changed,
+                           NoFilesError)
 from pelican.writers import Writer
 
 __major__ = 3
-__minor__ = 0
+__minor__ = 2
-__version__ = "{0}.{1}".format(__major__, __minor__)
+__micro__ = 0
+__version__ = "{0}.{1}.{2}".format(__major__, __minor__, __micro__)
 
 
 logger = logging.getLogger(__name__)
 
 
 class Pelican(object):
-    def __init__(self, settings=None, path=None, theme=None, output_path=None,
+    def __init__(self, settings):
-            markup=None, delete_outputdir=False, plugin_path=None):
+        """
-        """Read the settings, and performs some checks on the environment
+        Pelican initialisation, performs some checks on the environment before
-        before doing anything else.
+        doing anything else.
         """
-        if settings is None:
-            settings = _DEFAULT_CONFIG
-
-        self.path = path or settings['PATH']
-        if not self.path:
-            raise Exception('You need to specify a path containing the content'
-                    ' (see pelican --help for more information)')
-
-        if self.path.endswith('/'):
-            self.path = self.path[:-1]
 
         # define the default settings
         self.settings = settings
-
         self._handle_deprecation()
 
-        self.theme = theme or settings['THEME']
+        self.path = settings['PATH']
-        output_path = output_path or settings['OUTPUT_PATH']
+        self.theme = settings['THEME']
-        self.output_path = os.path.realpath(output_path)
+        self.output_path = settings['OUTPUT_PATH']
-        self.markup = markup or settings['MARKUP']
+        self.markup = settings['MARKUP']
-        self.delete_outputdir = delete_outputdir \
+        self.ignore_files = settings['IGNORE_FILES']
-                                    or settings['DELETE_OUTPUT_DIRECTORY']
+        self.delete_outputdir = settings['DELETE_OUTPUT_DIRECTORY']
-
-        # find the theme in pelican.theme if the given one does not exists
-        if not os.path.exists(self.theme):
-            theme_path = os.sep.join([os.path.dirname(
-                os.path.abspath(__file__)), "themes/%s" % self.theme])
-            if os.path.exists(theme_path):
-                self.theme = theme_path
-            else:
-                raise Exception("Impossible to find the theme %s" % theme)
 
+        self.init_path()
         self.init_plugins()
         signals.initialized.send(self)
 
+    def init_path(self):
+        if not any(p in sys.path for p in ['', '.']):
+            logger.debug("Adding current directory to system path")
+            sys.path.insert(0, '')
+
     def init_plugins(self):
         self.plugins = self.settings['PLUGINS']
         for plugin in self.plugins:
             # if it's a string, then import it
-            if isinstance(plugin, basestring):
+            if isinstance(plugin, six.string_types):
                 logger.debug("Loading plugin `{0}' ...".format(plugin))
                 plugin = __import__(plugin, globals(), locals(), 'module')
 
-            logger.debug("Registering plugin `{0}' ...".format(plugin.__name__))
+            logger.debug("Registering plugin `{0}'".format(plugin.__name__))
             plugin.register()
 
     def _handle_deprecation(self):
@@ -114,10 +107,41 @@ class Pelican(object):
                                                       self.settings[setting])
                 logger.warning("%s = '%s'" % (setting, self.settings[setting]))
 
+        if self.settings.get('FEED', False):
+            logger.warning('Found deprecated `FEED` in settings. Modify FEED'
+            ' to FEED_ATOM in your settings and theme for the same behavior.'
+            ' Temporarily setting FEED_ATOM for backwards compatibility.')
+            self.settings['FEED_ATOM'] = self.settings['FEED']
+
+        if self.settings.get('TAG_FEED', False):
+            logger.warning('Found deprecated `TAG_FEED` in settings. Modify '
+            ' TAG_FEED to TAG_FEED_ATOM in your settings and theme for the '
+            'same behavior. Temporarily setting TAG_FEED_ATOM for backwards '
+            'compatibility.')
+            self.settings['TAG_FEED_ATOM'] = self.settings['TAG_FEED']
+
+        if self.settings.get('CATEGORY_FEED', False):
+            logger.warning('Found deprecated `CATEGORY_FEED` in settings. '
+            'Modify CATEGORY_FEED to CATEGORY_FEED_ATOM in your settings and '
+            'theme for the same behavior. Temporarily setting '
+            'CATEGORY_FEED_ATOM for backwards compatibility.')
+            self.settings['CATEGORY_FEED_ATOM'] =\
+                    self.settings['CATEGORY_FEED']
+
+        if self.settings.get('TRANSLATION_FEED', False):
+            logger.warning('Found deprecated `TRANSLATION_FEED` in settings. '
+            'Modify TRANSLATION_FEED to TRANSLATION_FEED_ATOM in your '
+            'settings and theme for the same behavior. Temporarily setting '
+            'TRANSLATION_FEED_ATOM for backwards compatibility.')
+            self.settings['TRANSLATION_FEED_ATOM'] =\
+                    self.settings['TRANSLATION_FEED']
+
     def run(self):
         """Run the generators and return"""
 
         context = self.settings.copy()
+        context['filenames'] = {}  # share the dict between all the generators
+        context['localsiteurl'] = self.settings.get('SITEURL')  # share
         generators = [
             cls(
                 context,
@@ -126,7 +150,6 @@ class Pelican(object):
                 self.theme,
                 self.output_path,
                 self.markup,
-                self.delete_outputdir
             ) for cls in self.get_generator_classes()
         ]
 
@@ -142,21 +165,33 @@ class Pelican(object):
 
         writer = self.get_writer()
 
-        # pass the assets environment to the generators
-        if self.settings['WEBASSETS']:
-            generators[1].env.assets_environment = generators[0].assets_env
-            generators[2].env.assets_environment = generators[0].assets_env
-
         for p in generators:
             if hasattr(p, 'generate_output'):
                 p.generate_output(writer)
 
+        signals.finalized.send(self)
+
     def get_generator_classes(self):
         generators = [StaticGenerator, ArticlesGenerator, PagesGenerator]
+
+        if self.settings['TEMPLATE_PAGES']:
+            generators.append(TemplatePagesGenerator)
         if self.settings['PDF_GENERATOR']:
             generators.append(PdfGenerator)
-        if self.settings['LESS_GENERATOR']:  # can be True or PATH to lessc
+        if self.settings['OUTPUT_SOURCES']:
-            generators.append(LessCSSGenerator)
+            generators.append(SourceFileGenerator)
+
+        for pair in signals.get_generators.send(self):
+            (funct, value) = pair
+
+            if not isinstance(value, (tuple, list)):
+                value = (value, )
+
+            for v in value:
+                if isinstance(v, type):
+                    logger.debug('Found generator: {0}'.format(v))
+                    generators.append(v)
+
         return generators
 
     def get_writer(self):
@@ -213,31 +248,44 @@ def parse_arguments():
     return parser.parse_args()
 
 
-def get_instance(args):
+def get_config(args):
-    markup = [a.strip().lower() for a in args.markup.split(',')]\
+    config = {}
-              if args.markup else None
+    if args.path:
+        config['PATH'] = os.path.abspath(os.path.expanduser(args.path))
+    if args.output:
+        config['OUTPUT_PATH'] = \
+                os.path.abspath(os.path.expanduser(args.output))
+    if args.markup:
+        config['MARKUP'] = [a.strip().lower() for a in args.markup.split(',')]
+    if args.theme:
+        abstheme = os.path.abspath(os.path.expanduser(args.theme))
+        config['THEME'] = abstheme if os.path.exists(abstheme) else args.theme
+    if args.delete_outputdir is not None:
+        config['DELETE_OUTPUT_DIRECTORY'] = args.delete_outputdir
+    return config
 
-    settings = read_settings(args.settings)
+
+def get_instance(args):
+
+    settings = read_settings(args.settings, override=get_config(args))
 
     cls = settings.get('PELICAN_CLASS')
-    if isinstance(cls, basestring):
+    if isinstance(cls, six.string_types):
         module, cls_name = cls.rsplit('.', 1)
         module = __import__(module)
         cls = getattr(module, cls_name)
 
-    return cls(settings, args.path, args.theme, args.output, markup,
+    return cls(settings)
-               args.delete_outputdir)
 
 
 def main():
     args = parse_arguments()
     init(args.verbosity)
-    # Split the markup languages only if some have been given. Otherwise,
-    # populate the variable with None.
     pelican = get_instance(args)
 
     try:
         if args.autoreload:
+            files_found_error = True
             while True:
                 try:
                     # Check source dir for changed files ending with the given
@@ -245,8 +293,10 @@ def main():
                     # restriction; all files are recursively checked if they
                     # have changed, no matter what extension the filenames
                     # have.
-                    if files_changed(pelican.path, pelican.markup) or \
+                    if files_changed(pelican.path, pelican.markup, pelican.ignore_files) or \
-                            files_changed(pelican.theme, ['']):
+                            files_changed(pelican.theme, [''], pelican.ignore_files):
+                        if not files_found_error:
+                            files_found_error = True
                         pelican.run()
 
                     # reload also if settings.py changed
@@ -258,11 +308,23 @@ def main():
 
                     time.sleep(.5)  # sleep to avoid cpu load
                 except KeyboardInterrupt:
+                    logger.warning("Keyboard interrupt, quitting.")
                     break
+                except NoFilesError:
+                    if files_found_error:
+                        logger.warning("No valid files found in content. "
+                                       "Nothing to generate.")
+                        files_found_error = False
+                    time.sleep(1)  # sleep to avoid cpu load
+                except Exception as e:
+                    logger.warning(
+                        "Caught exception \"{}\". Reloading.".format(e)
+                    )
+                    continue
         else:
             pelican.run()
-    except Exception, e:
+    except Exception as e:
-        logger.critical(unicode(e))
+        logger.critical(e)
 
         if (args.verbosity == logging.DEBUG):
             raise

pelican/contents.py

@@ -1,19 +1,28 @@
 # -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
+import six
+
+import copy
 import locale
 import logging
 import functools
+import os
+import re
+import sys
 
 from datetime import datetime
-from os import getenv
 from sys import platform, stdin
 
 
 from pelican.settings import _DEFAULT_CONFIG
-from pelican.utils import slugify, truncate_html_words
+from pelican.utils import (slugify, truncate_html_words, memoized,
-
+    python_2_unicode_compatible, deprecated_attribute)
+from pelican import signals
+import pelican.utils
 
 logger = logging.getLogger(__name__)
 
+
 class Page(object):
     """Represents a page
     Given a content, and metadata, create an adequate object.
@@ -21,17 +30,23 @@ class Page(object):
     :param content: the string to parse, containing the original content.
     """
     mandatory_properties = ('title',)
+    default_template = 'page'
+
+    @deprecated_attribute(old='filename', new='source_path', since=(3, 2, 0))
+    def filename():
+        return None
 
     def __init__(self, content, metadata=None, settings=None,
-                 filename=None):
+                 source_path=None, context=None):
         # init parameters
         if not metadata:
             metadata = {}
         if not settings:
-            settings = _DEFAULT_CONFIG
+            settings = copy.deepcopy(_DEFAULT_CONFIG)
 
         self.settings = settings
         self._content = content
+        self._context = context
         self.translations = []
 
         local_metadata = dict(settings.get('DEFAULT_METADATA', ()))
@@ -44,15 +59,13 @@ class Page(object):
         # also keep track of the metadata attributes available
         self.metadata = local_metadata
 
+        #default template if it's not defined in page
+        self.template = self._get_template()
+
         # default author to the one in settings if not defined
         if not hasattr(self, 'author'):
             if 'AUTHOR' in settings:
                 self.author = Author(settings['AUTHOR'], settings)
-            else:
-                title = filename.decode('utf-8') if filename else self.title
-                self.author = Author(getenv('USER', 'John Doe'), settings)
-                logger.warning(u"Author of `{0}' unknown, assuming that his name is "
-                         "`{1}'".format(title, self.author))
 
         # manage languages
         self.in_default_lang = True
@@ -67,8 +80,8 @@ class Page(object):
         if not hasattr(self, 'slug') and hasattr(self, 'title'):
             self.slug = slugify(self.title)
 
-        if filename:
+        if source_path:
-            self.filename = filename
+            self.source_path = source_path
 
         # manage the date format
         if not hasattr(self, 'date_format'):
@@ -78,17 +91,15 @@ class Page(object):
                 self.date_format = settings['DEFAULT_DATE_FORMAT']
 
         if isinstance(self.date_format, tuple):
-            locale.setlocale(locale.LC_ALL, self.date_format[0])
+            locale_string = self.date_format[0]
+            if sys.version_info < (3, ) and isinstance(locale_string, six.text_type):
+                locale_string = locale_string.encode('ascii')
+            locale.setlocale(locale.LC_ALL, locale_string)
             self.date_format = self.date_format[1]
 
         if hasattr(self, 'date'):
-            encoded_date = self.date.strftime(
+            self.locale_date = pelican.utils.strftime(self.date,
-                    self.date_format.encode('ascii', 'xmlcharrefreplace'))
+                self.date_format)
-
-            if platform == 'win32':
-                self.locale_date = encoded_date.decode(stdin.encoding)
-            else:
-                self.locale_date = encoded_date.decode('utf')
 
         # manage status
         if not hasattr(self, 'status'):
@@ -101,6 +112,8 @@ class Page(object):
         if 'summary' in metadata:
             self._summary = metadata['summary']
 
+        signals.content_object_init.send(self.__class__, instance=self)
+
     def check_properties(self):
         """test that each mandatory property is set."""
         for prop in self.mandatory_properties:
@@ -109,13 +122,16 @@ class Page(object):
 
     @property
     def url_format(self):
-        return {
+        metadata = copy.copy(self.metadata)
+        metadata.update({
             'slug': getattr(self, 'slug', ''),
             'lang': getattr(self, 'lang', 'en'),
             'date': getattr(self, 'date', datetime.now()),
-            'author': self.author,
+            'author': getattr(self, 'author', ''),
-            'category': getattr(self, 'category', 'misc'),
+            'category': getattr(self, 'category',
-        }
+                self.settings['DEFAULT_CATEGORY']),
+            })
+        return metadata
 
     def _expand_settings(self, key):
         fq_key = ('%s_%s' % (self.__class__.__name__, key)).upper()
@@ -125,13 +141,60 @@ class Page(object):
         key = key if self.in_default_lang else 'lang_%s' % key
         return self._expand_settings(key)
 
+    def _update_content(self, content, siteurl):
+        """Change all the relative paths of the content to relative paths
+        suitable for the ouput content.
+
+        :param content: content resource that will be passed to the templates.
+        :param siteurl: siteurl which is locally generated by the writer in
+            case of RELATIVE_URLS.
+        """
+        hrefs = re.compile(r"""
+            (?P<markup><\s*[^\>]*  # match tag with src and href attr
+                (?:href|src)\s*=)
+
+            (?P<quote>["\'])      # require value to be quoted
+            (?P<path>\|(?P<what>.*?)\|(?P<value>.*?))  # the url value
+            \2""", re.X)
+
+        def replacer(m):
+            what = m.group('what')
+            value = m.group('value')
+            origin = m.group('path')
+            # we support only filename for now. the plan is to support
+            # categories, tags, etc. in the future, but let's keep things
+            # simple for now.
+            if what == 'filename':
+                if value.startswith('/'):
+                    value = value[1:]
+                else:
+                    # relative to the source path of this content
+                    value = self.get_relative_source_path(
+                        os.path.join(self.relative_dir, value)
+                    )
+
+                if value in self._context['filenames']:
+                    origin = '/'.join((siteurl,
+                             self._context['filenames'][value].url))
+                else:
+                    logger.warning("Unable to find {fn}, skipping url"
+                                    " replacement".format(fn=value))
+
+            return m.group('markup') + m.group('quote') + origin \
+                    + m.group('quote')
+
+        return hrefs.sub(replacer, content)
+
+    @memoized
+    def get_content(self, siteurl):
+        return self._update_content(
+                self._get_content() if hasattr(self, "_get_content")
+                    else self._content,
+                siteurl)
+
     @property
     def content(self):
-        if hasattr(self, "_get_content"):
+        return self.get_content(self._context['localsiteurl'])
-            content = self._get_content()
-        else:
-            content = self._content
-        return content
 
     def _get_summary(self):
         """Returns the summary of an article, based on the summary metadata
@@ -140,7 +203,8 @@ class Page(object):
             return self._summary
         else:
             if self.settings['SUMMARY_MAX_LENGTH']:
-                return truncate_html_words(self.content, self.settings['SUMMARY_MAX_LENGTH'])
+                return truncate_html_words(self.content,
+                        self.settings['SUMMARY_MAX_LENGTH'])
             return self.content
 
     def _set_summary(self, summary):
@@ -153,18 +217,49 @@ class Page(object):
     url = property(functools.partial(get_url_setting, key='url'))
     save_as = property(functools.partial(get_url_setting, key='save_as'))
 
+    def _get_template(self):
+        if hasattr(self, 'template') and self.template is not None:
+            return self.template
+        else:
+            return self.default_template
+
+    def get_relative_source_path(self, source_path=None):
+        """Return the relative path (from the content path) to the given
+        source_path.
+
+        If no source path is specified, use the source path of this
+        content object.
+        """
+        if not source_path:
+            source_path = self.source_path
+
+        return os.path.relpath(
+            os.path.abspath(os.path.join(self.settings['PATH'], source_path)),
+            os.path.abspath(self.settings['PATH'])
+        )
+
+    @property
+    def relative_dir(self):
+        return os.path.dirname(os.path.relpath(
+            os.path.abspath(self.source_path),
+            os.path.abspath(self.settings['PATH']))
+        )
+
 
 class Article(Page):
     mandatory_properties = ('title', 'date', 'category')
+    default_template = 'article'
 
 
 class Quote(Page):
     base_properties = ('author', 'date')
 
 
+@python_2_unicode_compatible
+@functools.total_ordering
 class URLWrapper(object):
     def __init__(self, name, settings):
-        self.name = unicode(name)
+        self.name = name
         self.slug = slugify(self.name)
         self.settings = settings
 
@@ -174,24 +269,41 @@ class URLWrapper(object):
     def __hash__(self):
         return hash(self.name)
 
-    def __eq__(self, other):
+    def _key(self):
-        return self.name == unicode(other)
-
-    def __str__(self):
-        return str(self.name.encode('utf-8', 'replace'))
-
-    def __unicode__(self):
         return self.name
 
-    def _from_settings(self, key):
+    def _normalize_key(self, key):
+        return six.text_type(key)
+
+    def __eq__(self, other):
+        return self._key() == self._normalize_key(other)
+
+    def __ne__(self, other):
+        return self._key() != self._normalize_key(other)
+
+    def __lt__(self, other):
+        return self._key() < self._normalize_key(other)
+
+    def __str__(self):
+        return self.name
+
+    def _from_settings(self, key, get_page_name=False):
+        """Returns URL information as defined in settings. 
+        When get_page_name=True returns URL without anything after {slug}
+        e.g. if in settings: CATEGORY_URL="cat/{slug}.html" this returns "cat/{slug}"
+        Useful for pagination."""
         setting = "%s_%s" % (self.__class__.__name__.upper(), key)
         value = self.settings[setting]
-        if not isinstance(value, basestring):
+        if not isinstance(value, six.string_types):
-            logger.warning(u'%s is set to %s' % (setting, value))
+            logger.warning('%s is set to %s' % (setting, value))
             return value
         else:
-            return unicode(value).format(**self.as_dict())
+            if get_page_name:
+                return os.path.splitext(value)[0].format(**self.as_dict())
+            else:
+                return value.format(**self.as_dict())
 
+    page_name = property(functools.partial(_from_settings, key='URL', get_page_name=True))
     url = property(functools.partial(_from_settings, key='URL'))
     save_as = property(functools.partial(_from_settings, key='SAVE_AS'))
 
@@ -202,18 +314,36 @@ class Category(URLWrapper):
 
 class Tag(URLWrapper):
     def __init__(self, name, *args, **kwargs):
-        super(Tag, self).__init__(unicode.strip(name), *args, **kwargs)
+        super(Tag, self).__init__(name.strip(), *args, **kwargs)
 
 
 class Author(URLWrapper):
     pass
 
 
+@python_2_unicode_compatible
+class StaticContent(object):
+    @deprecated_attribute(old='filepath', new='source_path', since=(3, 2, 0))
+    def filepath():
+        return None
+
+    def __init__(self, src, dst=None, settings=None):
+        if not settings:
+            settings = copy.deepcopy(_DEFAULT_CONFIG)
+        self.src = src
+        self.url = dst or src
+        self.source_path = os.path.join(settings['PATH'], src)
+        self.save_as = os.path.join(settings['OUTPUT_PATH'], self.url)
+
+    def __str__(self):
+        return self.source_path
+
+
 def is_valid_content(content, f):
     try:
         content.check_properties()
         return True
-    except NameError, e:
+    except NameError as e:
-        logger.error(u"Skipping %s: impossible to find informations about '%s'"\
+        logger.error("Skipping %s: impossible to find informations about "
-                % (f, e))
+                      "'%s'" % (f, e))
         return False

pelican/generators.py

@@ -1,22 +1,26 @@
 # -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
+
 import os
 import math
 import random
 import logging
 import datetime
-import subprocess
+import shutil
 
+from codecs import open
 from collections import defaultdict
 from functools import partial
 from itertools import chain
 from operator import attrgetter, itemgetter
 
-from jinja2 import Environment, FileSystemLoader, PrefixLoader, ChoiceLoader
+from jinja2 import (Environment, FileSystemLoader, PrefixLoader, ChoiceLoader,
-from jinja2.exceptions import TemplateNotFound
+                    BaseLoader, TemplateNotFound)
 
-from pelican.contents import Article, Page, Category, is_valid_content
+from pelican.contents import Article, Page, Category, StaticContent, \
+        is_valid_content
 from pelican.readers import read_file
-from pelican.utils import copy, process_translations, open
+from pelican.utils import copy, process_translations, mkdir_p
 from pelican import signals
 
 
@@ -36,14 +40,17 @@ class Generator(object):
 
         # templates cache
         self._templates = {}
-        self._templates_path = os.path.expanduser(
+        self._templates_path = []
-                os.path.join(self.theme, 'templates'))
+        self._templates_path.append(os.path.expanduser(
+                os.path.join(self.theme, 'templates')))
+        self._templates_path += self.settings.get('EXTRA_TEMPLATES_PATHS', [])
 
         theme_path = os.path.dirname(os.path.abspath(__file__))
 
         simple_loader = FileSystemLoader(os.path.join(theme_path,
                                          "themes", "simple", "templates"))
         self.env = Environment(
+            trim_blocks=True,
             loader=ChoiceLoader([
                 FileSystemLoader(self._templates_path),
                 simple_loader,  # implicit inheritance
@@ -58,6 +65,8 @@ class Generator(object):
         custom_filters = self.settings.get('JINJA_FILTERS', {})
         self.env.filters.update(custom_filters)
 
+        signals.generator_init.send(self)
+
     def get_template(self, name):
         """Return the template by name.
         Use self.theme to get the templates to use, and return a list of
@@ -76,8 +85,10 @@ class Generator(object):
 
         :param path: the path to search the file on
         :param exclude: the list of path to exclude
+        :param extensions: the list of allowed extensions (if False, all
+            extensions are allowed)
         """
-        if not extensions:
+        if extensions is None:
             extensions = self.markup
 
         files = []
@@ -91,10 +102,17 @@ class Generator(object):
             for e in exclude:
                 if e in dirs:
                     dirs.remove(e)
-            files.extend([os.sep.join((root, f)) for f in temp_files
+            for f in temp_files:
-                if True in [f.endswith(ext) for ext in extensions]])
+                if extensions is False or \
+                        (True in [f.endswith(ext) for ext in extensions]):
+                    files.append(os.sep.join((root, f)))
         return files
 
+    def add_source_path(self, content):
+        location = os.path.relpath(os.path.abspath(content.source_path),
+                                   os.path.abspath(self.path))
+        self.context['filenames'][location] = content
+
     def _update_context(self, items):
         """Update the context with the given items from the currrent
         processor.
@@ -102,10 +120,39 @@ class Generator(object):
         for item in items:
             value = getattr(self, item)
             if hasattr(value, 'items'):
-                value = value.items()
+                value = list(value.items())  # py3k safeguard for iterators
             self.context[item] = value
 
 
+class _FileLoader(BaseLoader):
+
+    def __init__(self, path, basedir):
+        self.path = path
+        self.fullpath = os.path.join(basedir, path)
+
+    def get_source(self, environment, template):
+        if template != self.path or not os.path.exists(self.fullpath):
+            raise TemplateNotFound(template)
+        mtime = os.path.getmtime(self.fullpath)
+        with open(self.fullpath, 'r', encoding='utf-8') as f:
+            source = f.read()
+        return source, self.fullpath, \
+                lambda: mtime == os.path.getmtime(self.fullpath)
+
+
+class TemplatePagesGenerator(Generator):
+
+    def generate_output(self, writer):
+        for source, dest in self.settings['TEMPLATE_PAGES'].items():
+            self.env.loader.loaders.insert(0, _FileLoader(source, self.path))
+            try:
+                template = self.env.get_template(source)
+                rurls = self.settings.get('RELATIVE_URLS')
+                writer.write_file(dest, template, self.context, rurls)
+            finally:
+                del self.env.loader.loaders[0]
+
+
 class ArticlesGenerator(Generator):
     """Generate blog articles"""
 
@@ -116,6 +163,7 @@ class ArticlesGenerator(Generator):
         self.dates = {}
         self.tags = defaultdict(list)
         self.categories = defaultdict(list)
+        self.related_posts = []
         self.authors = defaultdict(list)
         super(ArticlesGenerator, self).__init__(*args, **kwargs)
         self.drafts = []
@@ -124,54 +172,74 @@ class ArticlesGenerator(Generator):
     def generate_feeds(self, writer):
         """Generate the feeds from the current context, and output files."""
 
-        if self.settings.get('FEED'):
+        if self.settings.get('FEED_ATOM'):
             writer.write_feed(self.articles, self.context,
-                              self.settings['FEED'])
+                              self.settings['FEED_ATOM'])
 
         if self.settings.get('FEED_RSS'):
             writer.write_feed(self.articles, self.context,
                               self.settings['FEED_RSS'], feed_type='rss')
 
+        if self.settings.get('FEED_ALL_ATOM') or \
+                self.settings.get('FEED_ALL_RSS'):
+            all_articles = list(self.articles)
+            for article in self.articles:
+                all_articles.extend(article.translations)
+            all_articles.sort(key=attrgetter('date'), reverse=True)
+
+            if self.settings.get('FEED_ALL_ATOM'):
+                writer.write_feed(all_articles, self.context,
+                                  self.settings['FEED_ALL_ATOM'])
+
+            if self.settings.get('FEED_ALL_RSS'):
+                writer.write_feed(all_articles, self.context,
+                                  self.settings['FEED_ALL_RSS'], feed_type='rss')
+
         for cat, arts in self.categories:
             arts.sort(key=attrgetter('date'), reverse=True)
-            if self.settings.get('CATEGORY_FEED'):
+            if self.settings.get('CATEGORY_FEED_ATOM'):
                 writer.write_feed(arts, self.context,
-                                  self.settings['CATEGORY_FEED'] % cat)
+                                  self.settings['CATEGORY_FEED_ATOM'] % cat)
 
             if self.settings.get('CATEGORY_FEED_RSS'):
                 writer.write_feed(arts, self.context,
                                   self.settings['CATEGORY_FEED_RSS'] % cat,
                                   feed_type='rss')
 
-        if self.settings.get('TAG_FEED') or self.settings.get('TAG_FEED_RSS'):
+        if self.settings.get('TAG_FEED_ATOM') \
+                or self.settings.get('TAG_FEED_RSS'):
             for tag, arts in self.tags.items():
                 arts.sort(key=attrgetter('date'), reverse=True)
-                if self.settings.get('TAG_FEED'):
+                if self.settings.get('TAG_FEED_ATOM'):
                     writer.write_feed(arts, self.context,
-                                      self.settings['TAG_FEED'] % tag)
+                                      self.settings['TAG_FEED_ATOM'] % tag)
 
                 if self.settings.get('TAG_FEED_RSS'):
                     writer.write_feed(arts, self.context,
                                       self.settings['TAG_FEED_RSS'] % tag,
                                       feed_type='rss')
 
-        if self.settings.get('TRANSLATION_FEED'):
+        if self.settings.get('TRANSLATION_FEED_ATOM') or \
+                self.settings.get('TRANSLATION_FEED_RSS'):
             translations_feeds = defaultdict(list)
             for article in chain(self.articles, self.translations):
                 translations_feeds[article.lang].append(article)
 
             for lang, items in translations_feeds.items():
                 items.sort(key=attrgetter('date'), reverse=True)
-                writer.write_feed(items, self.context,
+                if self.settings.get('TRANSLATION_FEED_ATOM'):
-                                  self.settings['TRANSLATION_FEED'] % lang)
+                    writer.write_feed(items, self.context,
+                            self.settings['TRANSLATION_FEED_ATOM'] % lang)
+                if self.settings.get('TRANSLATION_FEED_RSS'):
+                    writer.write_feed(items, self.context,
+                            self.settings['TRANSLATION_FEED_RSS'] % lang,
+                            feed_type='rss')
 
     def generate_articles(self, write):
         """Generate the articles."""
-        article_template = self.get_template('article')
         for article in chain(self.translations, self.articles):
-            write(article.save_as,
+            write(article.save_as, self.get_template(article.template),
-                          article_template, self.context, article=article,
+                self.context, article=article, category=article.category)
-                          category=article.category)
 
     def generate_direct_templates(self, write):
         """Generate direct templates pages"""
@@ -183,7 +251,7 @@ class ArticlesGenerator(Generator):
             save_as = self.settings.get("%s_SAVE_AS" % template.upper(),
                                                         '%s.html' % template)
             if not save_as:
-                 continue
+                continue
 
             write(save_as, self.get_template(template),
                   self.context, blog=True, paginated=paginated,
@@ -198,7 +266,7 @@ class ArticlesGenerator(Generator):
             write(tag.save_as, tag_template, self.context, tag=tag,
                 articles=articles, dates=dates,
                 paginated={'articles': articles, 'dates': dates},
-                page_name=u'tag/%s' % tag)
+                page_name=tag.page_name)
 
     def generate_categories(self, write):
         """Generate category pages."""
@@ -208,7 +276,7 @@ class ArticlesGenerator(Generator):
             write(cat.save_as, category_template, self.context,
                 category=cat, articles=articles, dates=dates,
                 paginated={'articles': articles, 'dates': dates},
-                page_name=u'category/%s' % cat)
+                page_name=cat.page_name)
 
     def generate_authors(self, write):
         """Generate Author pages."""
@@ -218,14 +286,14 @@ class ArticlesGenerator(Generator):
             write(aut.save_as, author_template, self.context,
                 author=aut, articles=articles, dates=dates,
                 paginated={'articles': articles, 'dates': dates},
-                page_name=u'author/%s' % aut)
+                page_name=aut.page_name)
 
     def generate_drafts(self, write):
         """Generate drafts pages."""
-        article_template = self.get_template('article')
         for article in self.drafts:
-            write('drafts/%s.html' % article.slug, article_template,
+            write('drafts/%s.html' % article.slug,
-                  self.context, article=article, category=article.category)
+                self.get_template(article.template), self.context,
+                article=article, category=article.category)
 
     def generate_pages(self, writer):
         """Generate the pages on the disk"""
@@ -237,7 +305,6 @@ class ArticlesGenerator(Generator):
         self.generate_articles(write)
         self.generate_direct_templates(write)
 
-
         # and subfolders after that
         self.generate_tags(write)
         self.generate_categories(write)
@@ -245,7 +312,7 @@ class ArticlesGenerator(Generator):
         self.generate_drafts(write)
 
     def generate_context(self):
-        """change the context"""
+        """Add the articles into the shared context"""
 
         article_path = os.path.normpath(  # we have to remove trailing slashes
             os.path.join(self.path, self.settings['ARTICLE_DIR'])
@@ -255,33 +322,42 @@ class ArticlesGenerator(Generator):
                 article_path,
                 exclude=self.settings['ARTICLE_EXCLUDES']):
             try:
+                signals.article_generate_preread.send(self)
                 content, metadata = read_file(f, settings=self.settings)
-            except Exception, e:
+            except Exception as e:
-                logger.warning(u'Could not process %s\n%s' % (f, str(e)))
+                logger.warning('Could not process %s\n%s' % (f, str(e)))
                 continue
 
             # if no category is set, use the name of the path as a category
             if 'category' not in metadata:
 
-                if os.path.dirname(f) == article_path:  # if the article is not in a subdirectory
+                if (self.settings['USE_FOLDER_AS_CATEGORY']
-                    category = self.settings['DEFAULT_CATEGORY'] 
+                    and os.path.dirname(f) != article_path):
+                    # if the article is in a subdirectory
+                    category = os.path.basename(os.path.dirname(f))
                 else:
-                    category = os.path.basename(os.path.dirname(f))\
+                    # if the article is not in a subdirectory
-                                .decode('utf-8')
+                    category = self.settings['DEFAULT_CATEGORY']
 
                 if category != '':
                     metadata['category'] = Category(category, self.settings)
 
-            if 'date' not in metadata and self.settings['FALLBACK_ON_FS_DATE']:
+            if 'date' not in metadata and self.settings.get('DEFAULT_DATE'):
+                if self.settings['DEFAULT_DATE'] == 'fs':
                     metadata['date'] = datetime.datetime.fromtimestamp(
-                                        os.stat(f).st_ctime)
+                            os.stat(f).st_ctime)
+                else:
+                    metadata['date'] = datetime.datetime(
+                            *self.settings['DEFAULT_DATE'])
 
             signals.article_generate_context.send(self, metadata=metadata)
             article = Article(content, metadata, settings=self.settings,
-                              filename=f)
+                              source_path=f, context=self.context)
             if not is_valid_content(article, f):
                 continue
 
+            self.add_source_path(article)
+
             if article.status == "published":
                 if hasattr(article, 'tags'):
                     for tag in article.tags:
@@ -290,8 +366,8 @@ class ArticlesGenerator(Generator):
             elif article.status == "draft":
                 self.drafts.append(article)
             else:
-                logger.warning(u"Unknown status %s for file %s, skipping it." %
+                logger.warning("Unknown status %s for file %s, skipping it." %
-                               (repr(unicode.encode(article.status, 'utf-8')),
+                               (repr(article.status),
                                 repr(f)))
 
         self.articles, self.translations = process_translations(all_articles)
@@ -299,13 +375,15 @@ class ArticlesGenerator(Generator):
         for article in self.articles:
             # only main articles are listed in categories, not translations
             self.categories[article.category].append(article)
-            self.authors[article.author].append(article)
+            # ignore blank authors as well as undefined
+            if hasattr(article,'author') and article.author.name != '':
+                self.authors[article.author].append(article)
 
         # sort the articles by date
         self.articles.sort(key=attrgetter('date'), reverse=True)
         self.dates = list(self.articles)
         self.dates.sort(key=attrgetter('date'),
-                reverse=self.context['REVERSE_ARCHIVE_ORDER'])
+                reverse=self.context['NEWEST_FIRST_ARCHIVES'])
 
         # create tag cloud
         tag_cloud = defaultdict(int)
@@ -316,7 +394,7 @@ class ArticlesGenerator(Generator):
         tag_cloud = sorted(tag_cloud.items(), key=itemgetter(1), reverse=True)
         tag_cloud = tag_cloud[:self.settings.get('TAG_CLOUD_MAX_ITEMS')]
 
-        tags = map(itemgetter(1), tag_cloud)
+        tags = list(map(itemgetter(1), tag_cloud))
         if tags:
             max_count = max(tags)
         steps = self.settings.get('TAG_CLOUD_STEPS')
@@ -338,14 +416,15 @@ class ArticlesGenerator(Generator):
         # order the categories per name
         self.categories = list(self.categories.items())
         self.categories.sort(
-                key=lambda item: item[0].name,
                 reverse=self.settings['REVERSE_CATEGORY_ORDER'])
 
         self.authors = list(self.authors.items())
-        self.authors.sort(key=lambda item: item[0].name)
+        self.authors.sort()
 
         self._update_context(('articles', 'dates', 'tags', 'categories',
-                              'tag_cloud', 'authors'))
+                              'tag_cloud', 'authors', 'related_posts'))
+
+        signals.article_generator_finalized.send(self)
 
     def generate_output(self, writer):
         self.generate_feeds(writer)
@@ -360,6 +439,7 @@ class PagesGenerator(Generator):
         self.hidden_pages = []
         self.hidden_translations = []
         super(PagesGenerator, self).__init__(*args, **kwargs)
+        signals.pages_generator_init.send(self)
 
     def generate_context(self):
         all_pages = []
@@ -368,24 +448,27 @@ class PagesGenerator(Generator):
                 os.path.join(self.path, self.settings['PAGE_DIR']),
                 exclude=self.settings['PAGE_EXCLUDES']):
             try:
-                content, metadata = read_file(f)
+                content, metadata = read_file(f, settings=self.settings)
-            except Exception, e:
+            except Exception as e:
-                logger.error(u'Could not process %s\n%s' % (f, str(e)))
+                logger.warning('Could not process %s\n%s' % (f, str(e)))
                 continue
+            signals.pages_generate_context.send(self, metadata=metadata)
             page = Page(content, metadata, settings=self.settings,
-                        filename=f)
+                        source_path=f, context=self.context)
             if not is_valid_content(page, f):
                 continue
+
+            self.add_source_path(page)
+
             if page.status == "published":
                 all_pages.append(page)
             elif page.status == "hidden":
                 hidden_pages.append(page)
             else:
-                logger.warning(u"Unknown status %s for file %s, skipping it." %
+                logger.warning("Unknown status %s for file %s, skipping it." %
-                               (repr(unicode.encode(page.status, 'utf-8')),
+                               (repr(page.status),
                                 repr(f)))
 
-
         self.pages, self.translations = process_translations(all_pages)
         self.hidden_pages, self.hidden_translations = process_translations(hidden_pages)
 
@@ -395,7 +478,7 @@ class PagesGenerator(Generator):
     def generate_output(self, writer):
         for page in chain(self.translations, self.pages,
                             self.hidden_translations, self.hidden_pages):
-            writer.write_file(page.save_as, self.get_template('page'),
+            writer.write_file(page.save_as, self.get_template(page.template),
                     self.context, page=page,
                     relative_urls=self.settings.get('RELATIVE_URLS'))
 
@@ -412,53 +495,61 @@ class StaticGenerator(Generator):
                  final_path, overwrite=True)
 
     def generate_context(self):
+        self.staticfiles = []
 
-        if self.settings['WEBASSETS']:
+        # walk static paths
-            from webassets import Environment as AssetsEnvironment
+        for static_path in self.settings['STATIC_PATHS']:
-
+            for f in self.get_files(
-            # Define the assets environment that will be passed to the
+                    os.path.join(self.path, static_path), extensions=False):
-            # generators. The StaticGenerator must then be run first to have
+                f_rel = os.path.relpath(f, self.path)
-            # the assets in the output_path before generating the templates.
+                # TODO remove this hardcoded 'static' subdirectory
-            assets_url = self.settings['SITEURL'] + '/theme/'
+                sc = StaticContent(f_rel, os.path.join('static', f_rel),
-            assets_src = os.path.join(self.output_path, 'theme')
+                        settings=self.settings)
-            self.assets_env = AssetsEnvironment(assets_src, assets_url)
+                self.staticfiles.append(sc)
-
+                self.context['filenames'][f_rel] = sc
-            if logging.getLevelName(logger.getEffectiveLevel()) == "DEBUG":
+        # same thing for FILES_TO_COPY
-                self.assets_env.debug = True
+        for src, dest in self.settings['FILES_TO_COPY']:
+            sc = StaticContent(src, dest, settings=self.settings)
+            self.staticfiles.append(sc)
+            self.context['filenames'][src] = sc
 
     def generate_output(self, writer):
-
-        self._copy_paths(self.settings['STATIC_PATHS'], self.path,
-                         'static', self.output_path)
         self._copy_paths(self.settings['THEME_STATIC_PATHS'], self.theme,
                          'theme', self.output_path, '.')
-
+        # copy all StaticContent files
-        # copy all the files needed
+        for sc in self.staticfiles:
-        for source, destination in self.settings['FILES_TO_COPY']:
+            mkdir_p(os.path.dirname(sc.save_as))
-            copy(source, self.path, self.output_path, destination,
+            shutil.copy(sc.source_path, sc.save_as)
-                 overwrite=True)
+            logger.info('copying {} to {}'.format(sc.source_path, sc.save_as))
 
 
 class PdfGenerator(Generator):
     """Generate PDFs on the output dir, for all articles and pages coming from
     rst"""
     def __init__(self, *args, **kwargs):
+        super(PdfGenerator, self).__init__(*args, **kwargs)
         try:
             from rst2pdf.createpdf import RstToPdf
+            pdf_style_path = os.path.join(self.settings['PDF_STYLE_PATH']) \
+                                if 'PDF_STYLE_PATH' in self.settings.keys() \
+                                else ''
+            pdf_style = self.settings['PDF_STYLE'] if 'PDF_STYLE' \
+                                                    in self.settings.keys() \
+                                                    else 'twelvepoint'
             self.pdfcreator = RstToPdf(breakside=0,
-                                       stylesheets=['twelvepoint'])
+                                       stylesheets=[pdf_style],
+                                       style_path=[pdf_style_path])
         except ImportError:
             raise Exception("unable to find rst2pdf")
-        super(PdfGenerator, self).__init__(*args, **kwargs)
 
     def _create_pdf(self, obj, output_path):
-        if obj.filename.endswith(".rst"):
+        if obj.source_path.endswith('.rst'):
             filename = obj.slug + ".pdf"
             output_pdf = os.path.join(output_path, filename)
-            # print "Generating pdf for", obj.filename, " in ", output_pdf
+            # print('Generating pdf for', obj.source_path, 'in', output_pdf)
-            with open(obj.filename) as f:
+            with open(obj.source_path) as f:
-                self.pdfcreator.createPdf(text=f, output=output_pdf)
+                self.pdfcreator.createPdf(text=f.read(), output=output_pdf)
-            logger.info(u' [ok] writing %s' % output_pdf)
+            logger.info(' [ok] writing %s' % output_pdf)
 
     def generate_context(self):
         pass
@@ -466,14 +557,14 @@ class PdfGenerator(Generator):
     def generate_output(self, writer=None):
         # we don't use the writer passed as argument here
         # since we write our own files
-        logger.info(u' Generating PDF files...')
+        logger.info(' Generating PDF files...')
         pdf_path = os.path.join(self.output_path, 'pdf')
         if not os.path.exists(pdf_path):
             try:
                 os.mkdir(pdf_path)
             except OSError:
-                logger.error("Couldn't create the pdf output folder in " + pdf_path)
+                logger.error("Couldn't create the pdf output folder in " +
-                pass
+                             pdf_path)
 
         for article in self.context['articles']:
             self._create_pdf(article, pdf_path)
@@ -481,49 +572,16 @@ class PdfGenerator(Generator):
         for page in self.context['pages']:
             self._create_pdf(page, pdf_path)
 
+class SourceFileGenerator(Generator):
+    def generate_context(self):
+        self.output_extension = self.settings['OUTPUT_SOURCES_EXTENSION']
 
-class LessCSSGenerator(Generator):
+    def _create_source(self, obj, output_path):
-    """Compile less css files."""
+        output_path = os.path.splitext(obj.save_as)[0]
-
+        dest = os.path.join(output_path, output_path + self.output_extension)
-    def _compile(self, less_file, source_dir, dest_dir):
+        copy('', obj.source_path, dest)
-        base = os.path.relpath(less_file, source_dir)
-        target = os.path.splitext(
-                os.path.join(dest_dir, base))[0] + '.css'
-        target_dir = os.path.dirname(target)
-
-        if not os.path.exists(target_dir):
-            try:
-                os.makedirs(target_dir)
-            except OSError:
-                logger.error("Couldn't create the less css output folder in " +
-                        target_dir)
-
-        subprocess.call([self._lessc, less_file, target])
-        logger.info(u' [ok] compiled %s' % base)
 
     def generate_output(self, writer=None):
-        logger.info(u' Compiling less css')
+        logger.info(' Generating source files...')
-
+        for object in chain(self.context['articles'], self.context['pages']):
-        # store out compiler here, so it won't be evaulted on each run of
+            self._create_source(object, self.output_path)
-        # _compile
-        lg = self.settings['LESS_GENERATOR']
-        self._lessc = lg if isinstance(lg, basestring) else 'lessc'
-
-        # walk static paths
-        for static_path in self.settings['STATIC_PATHS']:
-            for f in self.get_files(
-                    os.path.join(self.path, static_path),
-                    extensions=['less']):
-
-                self._compile(f, self.path, self.output_path)
-
-        # walk theme static paths
-        theme_output_path = os.path.join(self.output_path, 'theme')
-
-        for static_path in self.settings['THEME_STATIC_PATHS']:
-            theme_static_path = os.path.join(self.theme, static_path)
-            for f in self.get_files(
-                    theme_static_path,
-                    extensions=['less']):
-
-                self._compile(f, theme_static_path, theme_output_path)

pelican/log.py

@@ -1,3 +1,6 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
+
 __all__ = [
     'init'
 ]
@@ -9,7 +12,7 @@ import logging
 from logging import Formatter, getLogger, StreamHandler, DEBUG
 
 
-RESET_TERM = u'\033[0;m'
+RESET_TERM = '\033[0;m'
 
 COLOR_CODES = {
     'red': 31,
@@ -24,37 +27,38 @@ COLOR_CODES = {
 def ansi(color, text):
     """Wrap text in an ansi escape sequence"""
     code = COLOR_CODES[color]
-    return u'\033[1;{0}m{1}{2}'.format(code, text, RESET_TERM)
+    return '\033[1;{0}m{1}{2}'.format(code, text, RESET_TERM)
 
 
 class ANSIFormatter(Formatter):
     """
-    Convert a `logging.LogReport' object into colored text, using ANSI escape sequences.
+    Convert a `logging.LogRecord' object into colored text, using ANSI escape sequences.
     """
     ## colors:
 
     def format(self, record):
-        if record.levelname is 'INFO':
+        msg = str(record.msg)
-            return ansi('cyan', '-> ') + unicode(record.msg)
+        if record.levelname == 'INFO':
-        elif record.levelname is 'WARNING':
+            return ansi('cyan', '-> ') + msg
-            return ansi('yellow', record.levelname) + ': ' + unicode(record.msg)
+        elif record.levelname == 'WARNING':
-        elif record.levelname is 'ERROR':
+            return ansi('yellow', record.levelname) + ': ' + msg
-            return ansi('red', record.levelname) + ': ' + unicode(record.msg)
+        elif record.levelname == 'ERROR':
-        elif record.levelname is 'CRITICAL':
+            return ansi('red', record.levelname) + ': ' + msg
-            return ansi('bgred', record.levelname) + ': ' + unicode(record.msg)
+        elif record.levelname == 'CRITICAL':
-        elif record.levelname is 'DEBUG':
+            return ansi('bgred', record.levelname) + ': ' + msg
-            return ansi('bggrey', record.levelname) + ': ' + unicode(record.msg)
+        elif record.levelname == 'DEBUG':
+            return ansi('bggrey', record.levelname) + ': ' + msg
         else:
-            return ansi('white', record.levelname) + ': ' + unicode(record.msg)
+            return ansi('white', record.levelname) + ': ' + msg
 
 
 class TextFormatter(Formatter):
     """
-    Convert a `logging.LogReport' object into text.
+    Convert a `logging.LogRecord' object into text.
     """
 
     def format(self, record):
-        if not record.levelname or record.levelname is 'INFO':
+        if not record.levelname or record.levelname == 'INFO':
             return record.msg
         else:
             return record.levelname + ': ' + record.msg

pelican/paginator.py

@@ -1,3 +1,6 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
+
 # From django.core.paginator
 from math import ceil
 
@@ -37,7 +40,7 @@ class Paginator(object):
         Returns a 1-based range of pages for iterating through within
         a template for loop.
         """
-        return range(1, self.num_pages + 1)
+        return list(range(1, self.num_pages + 1))
     page_range = property(_get_page_range)
 
 

pelican/plugins/assets.py

@@ -0,0 +1,53 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+"""
+Asset management plugin for Pelican
+===================================
+
+This plugin allows you to use the `webassets`_ module to manage assets such as
+CSS and JS files.
+
+The ASSET_URL is set to a relative url to honor Pelican's RELATIVE_URLS
+setting. This requires the use of SITEURL in the templates::
+
+    <link rel="stylesheet" href="{{ SITEURL }}/{{ ASSET_URL }}">
+
+.. _webassets: https://webassets.readthedocs.org/
+
+"""
+
+import os
+import logging
+
+from pelican import signals
+from webassets import Environment
+from webassets.ext.jinja2 import AssetsExtension
+
+
+def add_jinja2_ext(pelican):
+    """Add Webassets to Jinja2 extensions in Pelican settings."""
+
+    pelican.settings['JINJA_EXTENSIONS'].append(AssetsExtension)
+
+
+def create_assets_env(generator):
+    """Define the assets environment and pass it to the generator."""
+
+    assets_url = 'theme/'
+    assets_src = os.path.join(generator.output_path, 'theme')
+    generator.env.assets_environment = Environment(assets_src, assets_url)
+
+    if 'ASSET_CONFIG' in generator.settings:
+        for item in generator.settings['ASSET_CONFIG']:
+            generator.env.assets_environment.config[item[0]] = item[1]
+
+    logger = logging.getLogger(__name__)
+    if logging.getLevelName(logger.getEffectiveLevel()) == "DEBUG":
+        generator.env.assets_environment.debug = True
+
+
+def register():
+    """Plugin registration."""
+
+    signals.initialized.connect(add_jinja2_ext)
+    signals.generator_init.connect(create_assets_env)

pelican/plugins/github_activity.py

@@ -1,4 +1,5 @@
 # -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
 """
     Copyright (c) Marco Milanesi <kpanic@gnufunk.org>
 

pelican/plugins/global_license.py

@@ -4,13 +4,14 @@ from pelican import signals
 License plugin for Pelican
 ==========================
 
-Simply add license variable in article's context, which contain 
+This plugin allows you to define a LICENSE setting and adds the contents of that
-the license text.
+license variable to the article's context, making that variable available to use
+from within your theme's templates.
 
 Settings:
 ---------
 
-Add LICENSE to your settings file to define default license.
+Define LICENSE in your settings file with the contents of your default license.
 
 """
 

pelican/plugins/gravatar.py

@@ -5,20 +5,22 @@ from pelican import signals
 Gravatar plugin for Pelican
 ===========================
 
-Simply add author_gravatar variable in article's context, which contains
+This plugin assigns the ``author_gravatar`` variable to the Gravatar URL and
-the gravatar url.
+makes the variable available within the article's context.
 
 Settings:
 ---------
 
-Add AUTHOR_EMAIL to your settings file to define default author email.
+Add AUTHOR_EMAIL to your settings file to define the default author's email
+address. Obviously, that email address must be associated with a Gravatar
+account.
 
 Article metadata:
 ------------------
 
 :email:  article's author email
 
-If one of them are defined, the author_gravatar variable is added to
+If one of them are defined, the author_gravatar variable is added to the
 article's context.
 """
 

pelican/plugins/gzip_cache.py

@@ -0,0 +1,78 @@
+# Copyright (c) 2012 Matt Layman
+'''A plugin to create .gz cache files for optimization.'''
+
+import gzip
+import logging
+import os
+
+from pelican import signals
+
+logger = logging.getLogger(__name__)
+
+# A list of file types to exclude from possible compression
+EXCLUDE_TYPES = [
+    # Compressed types
+    '.bz2',
+    '.gz',
+
+    # Audio types
+    '.aac',
+    '.flac',
+    '.mp3',
+    '.wma',
+
+    # Image types
+    '.gif',
+    '.jpg',
+    '.jpeg',
+    '.png',
+
+    # Video types
+    '.avi',
+    '.mov',
+    '.mp4',
+]
+
+def create_gzip_cache(pelican):
+    '''Create a gzip cache file for every file that a webserver would
+    reasonably want to cache (e.g., text type files).
+
+    :param pelican: The Pelican instance
+    '''
+    for dirpath, _, filenames in os.walk(pelican.settings['OUTPUT_PATH']):
+        for name in filenames:
+            if should_compress(name):
+                filepath = os.path.join(dirpath, name)
+                create_gzip_file(filepath)
+
+def should_compress(filename):
+    '''Check if the filename is a type of file that should be compressed.
+
+    :param filename: A file name to check against
+    '''
+    for extension in EXCLUDE_TYPES:
+        if filename.endswith(extension):
+            return False
+
+    return True
+
+def create_gzip_file(filepath):
+    '''Create a gzipped file in the same directory with a filepath.gz name.
+
+    :param filepath: A file to compress
+    '''
+    compressed_path = filepath + '.gz'
+
+    with open(filepath, 'rb') as uncompressed:
+        try:
+            logger.debug('Compressing: %s' % filepath)
+            compressed = gzip.open(compressed_path, 'wb')
+            compressed.writelines(uncompressed)
+        except Exception as ex:
+            logger.critical('Gzip compression failed: %s' % ex)
+        finally:
+            compressed.close()
+
+def register():
+    signals.finalized.connect(create_gzip_cache)
+

pelican/plugins/html_rst_directive.py

@@ -1,6 +1,7 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
 from docutils import nodes
 from docutils.parsers.rst import directives, Directive
-from pelican import log
 
 """
 HTML tags for reStructuredText
@@ -52,7 +53,7 @@ class RawHtml(Directive):
     has_content = True
 
     def run(self):
-        html = u' '.join(self.content)
+        html = ' '.join(self.content)
         node = nodes.raw('', html, format='html')
         return [node]
 

pelican/plugins/initialized.py

@@ -1,7 +1,9 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
 from pelican import signals
 
 def test(sender):
-    print "%s initialized !!" % sender
+    print("%s initialized !!" % sender)
 
 def register():
     signals.initialized.connect(test)

pelican/plugins/multi_part.py

@@ -0,0 +1,59 @@
+# -*- coding: utf-8 -*-
+"""
+Copyright (c) FELD Boris <lothiraldan@gmail.com>
+
+Multiple part support
+=====================
+
+Create a navigation menu for multi-part related_posts
+
+Article metadata:
+------------------
+
+:parts:  a unique identifier for multi-part posts, must be the same in each
+post part.
+
+Usage
+-----
+    {% if article.metadata.parts_articles %}
+        <ol>
+        {% for part_article in article.metadata.parts_articles %}
+            {% if part_article == article %}
+                <li>
+                    <a href='{{ SITEURL }}/{{ part_article.url }}'><b>{{ part_article.title }}</b>
+                    </a>
+                </li>
+            {% else %}
+                <li>
+                    <a href='{{ SITEURL }}/{{ part_article.url }}'>{{ part_article.title }}
+                    </a>
+                </li>
+            {% endif %}
+        {% endfor %}
+        </ol>
+    {% endif %}
+"""
+from collections import defaultdict
+
+from pelican import signals
+
+
+def aggregate_multi_part(generator):
+        multi_part = defaultdict(list)
+
+        for article in generator.articles:
+            if 'parts' in article.metadata:
+                multi_part[article.metadata['parts']].append(article)
+
+        for part_id in multi_part:
+            parts = multi_part[part_id]
+
+            # Sort by date
+            parts.sort(key=lambda x: x.metadata['date'])
+
+            for article in parts:
+                article.metadata['parts_articles'] = parts
+
+
+def register():
+    signals.article_generator_finalized.connect(aggregate_multi_part)

pelican/plugins/related_posts.py

@@ -0,0 +1,52 @@
+from pelican import signals
+
+"""
+Related posts plugin for Pelican
+================================
+
+Adds related_posts variable to article's context
+
+Settings
+--------
+To enable, add
+
+    from pelican.plugins import related_posts
+    PLUGINS = [related_posts]
+
+to your settings.py.
+
+Usage
+-----
+    {% if article.related_posts %}
+        <ul>
+        {% for related_post in article.related_posts %}
+            <li>{{ related_post }}</li>
+        {% endfor %}
+        </ul>
+    {% endif %}
+
+
+"""
+
+related_posts = []
+
+
+def add_related_posts(generator, metadata):
+    if 'tags' in metadata:
+        for tag in metadata['tags']:
+            #print tag
+            for related_article in generator.tags[tag]:
+                related_posts.append(related_article)
+
+        if len(related_posts) < 1:
+            return
+
+        relation_score = dict(list(zip(set(related_posts), list(map(related_posts.count,
+                              set(related_posts))))))
+        ranked_related = sorted(relation_score, key=relation_score.get)
+
+        metadata["related_posts"] = ranked_related[:5]
+
+
+def register():
+    signals.article_generate_context.connect(add_related_posts)

pelican/plugins/sitemap.py

@@ -0,0 +1,195 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+
+import collections
+import os.path
+
+from datetime import datetime
+from logging import warning, info
+from codecs import open
+
+from pelican import signals, contents
+
+TXT_HEADER = """{0}/index.html
+{0}/archives.html
+{0}/tags.html
+{0}/categories.html
+"""
+
+XML_HEADER = """<?xml version="1.0" encoding="utf-8"?>
+<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"
+xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
+"""
+
+XML_URL = """
+<url>
+<loc>{0}/{1}</loc>
+<lastmod>{2}</lastmod>
+<changefreq>{3}</changefreq>
+<priority>{4}</priority>
+</url>
+"""
+
+XML_FOOTER = """
+</urlset>
+"""
+
+
+def format_date(date):
+    if date.tzinfo:
+        tz = date.strftime('%s')
+        tz = tz[:-2] + ':' + tz[-2:]
+    else:
+        tz = "-00:00"
+    return date.strftime("%Y-%m-%dT%H:%M:%S") + tz
+
+
+class SitemapGenerator(object):
+
+    def __init__(self, context, settings, path, theme, output_path, *null):
+
+        self.output_path = output_path
+        self.context = context
+        self.now = datetime.now()
+        self.siteurl = settings.get('SITEURL')
+
+        self.format = 'xml'
+
+        self.changefreqs = {
+            'articles': 'monthly',
+            'indexes': 'daily',
+            'pages': 'monthly'
+        }
+
+        self.priorities = {
+            'articles': 0.5,
+            'indexes': 0.5,
+            'pages': 0.5
+        }
+
+        config = settings.get('SITEMAP', {})
+
+        if not isinstance(config, dict):
+            warning("sitemap plugin: the SITEMAP setting must be a dict")
+        else:
+            fmt = config.get('format')
+            pris = config.get('priorities')
+            chfreqs = config.get('changefreqs')
+
+            if fmt not in ('xml', 'txt'):
+                warning("sitemap plugin: SITEMAP['format'] must be `txt' or `xml'")
+                warning("sitemap plugin: Setting SITEMAP['format'] on `xml'")
+            elif fmt == 'txt':
+                self.format = fmt
+                return
+
+            valid_keys = ('articles', 'indexes', 'pages')
+            valid_chfreqs = ('always', 'hourly', 'daily', 'weekly', 'monthly',
+                    'yearly', 'never')
+
+            if isinstance(pris, dict):
+                # We use items for Py3k compat. .iteritems() otherwise
+                for k, v in pris.items():
+                    if k in valid_keys and not isinstance(v, (int, float)):
+                        default = self.priorities[k]
+                        warning("sitemap plugin: priorities must be numbers")
+                        warning("sitemap plugin: setting SITEMAP['priorities']"
+                                "['{0}'] on {1}".format(k, default))
+                        pris[k] = default
+                self.priorities.update(pris)
+            elif pris is not None:
+                warning("sitemap plugin: SITEMAP['priorities'] must be a dict")
+                warning("sitemap plugin: using the default values")
+
+            if isinstance(chfreqs, dict):
+                # .items() for py3k compat.
+                for k, v in chfreqs.items():
+                    if k in valid_keys and v not in valid_chfreqs:
+                        default = self.changefreqs[k]
+                        warning("sitemap plugin: invalid changefreq `{0}'".format(v))
+                        warning("sitemap plugin: setting SITEMAP['changefreqs']"
+                                "['{0}'] on '{1}'".format(k, default))
+                        chfreqs[k] = default
+                self.changefreqs.update(chfreqs)
+            elif chfreqs is not None:
+                warning("sitemap plugin: SITEMAP['changefreqs'] must be a dict")
+                warning("sitemap plugin: using the default values")
+
+
+
+    def write_url(self, page, fd):
+
+        if getattr(page, 'status', 'published') != 'published':
+            return
+
+        page_path = os.path.join(self.output_path, page.url)
+        if not os.path.exists(page_path):
+            return
+
+        lastmod = format_date(getattr(page, 'date', self.now))
+
+        if isinstance(page, contents.Article):
+            pri = self.priorities['articles']
+            chfreq = self.changefreqs['articles']
+        elif isinstance(page, contents.Page):
+            pri = self.priorities['pages']
+            chfreq = self.changefreqs['pages']
+        else:
+            pri = self.priorities['indexes']
+            chfreq = self.changefreqs['indexes']
+
+
+        if self.format == 'xml':
+            fd.write(XML_URL.format(self.siteurl, page.url, lastmod, chfreq, pri))
+        else:
+            fd.write(self.siteurl + '/' + loc + '\n')
+
+
+    def generate_output(self, writer):
+        path = os.path.join(self.output_path, 'sitemap.{0}'.format(self.format))
+
+        pages = self.context['pages'] + self.context['articles'] \
+                + [ c for (c, a) in self.context['categories']] \
+                + [ t for (t, a) in self.context['tags']] \
+                + [ a for (a, b) in self.context['authors']]
+
+        for article in self.context['articles']:
+            pages += article.translations
+
+        info('writing {0}'.format(path))
+
+        with open(path, 'w', encoding='utf-8') as fd:
+
+            if self.format == 'xml':
+                fd.write(XML_HEADER)
+            else:
+                fd.write(TXT_HEADER.format(self.siteurl))
+
+            FakePage = collections.namedtuple('FakePage',
+                                              ['status',
+                                               'date',
+                                               'url'])
+
+            for standard_page_url in ['index.html',
+                                      'archives.html',
+                                      'tags.html',
+                                      'categories.html']:
+                fake = FakePage(status='published',
+                                date=self.now,
+                                url=standard_page_url)
+                self.write_url(fake, fd)
+
+            for page in pages:
+                self.write_url(page, fd)
+
+            if self.format == 'xml':
+                fd.write(XML_FOOTER)
+
+
+def get_generators(generators):
+    return SitemapGenerator
+
+
+def register():
+    signals.get_generators.connect(get_generators)

pelican/readers.py

@@ -1,4 +1,9 @@
 # -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
+import six
+
+import os
+import re
 try:
     import docutils
     import docutils.core
@@ -13,19 +18,24 @@ try:
     from markdown import Markdown
 except ImportError:
     Markdown = False  # NOQA
+try:
+    from asciidocapi import AsciiDocAPI
+    asciidoc = True
+except ImportError:
+    asciidoc = False
 import re
 
 import cgi
 from HTMLParser import HTMLParser
 
 from pelican.contents import Category, Tag, Author
-from pelican.utils import get_date, open
+from pelican.utils import get_date, pelican_open
 
 
 _METADATA_PROCESSORS = {
-    'tags': lambda x, y: [Tag(tag, y) for tag in unicode(x).split(',')],
+    'tags': lambda x, y: [Tag(tag, y) for tag in x.split(',')],
     'date': lambda x, y: get_date(x),
-    'status': lambda x, y: unicode.strip(x),
+    'status': lambda x, y: x.strip(),
     'category': Category,
     'author': Author,
 }
@@ -66,6 +76,18 @@ def render_node_to_html(document, node):
     return visitor.astext()
 
 
+class PelicanHTMLTranslator(HTMLTranslator):
+
+    def visit_abbreviation(self, node):
+        attrs = {}
+        if node.hasattr('explanation'):
+            attrs['title'] = node['explanation']
+        self.body.append(self.starttag(node, 'abbr', '', **attrs))
+
+    def depart_abbreviation(self, node):
+        self.body.append('</abbr>')
+
+
 class RstReader(Reader):
     enabled = bool(docutils)
     file_extensions = ['rst']
@@ -90,19 +112,20 @@ class RstReader(Reader):
                 output[name] = self.process_metadata(name, value)
         return output
 
-    def _get_publisher(self, filename):
+    def _get_publisher(self, source_path):
         extra_params = {'initial_header_level': '2'}
         pub = docutils.core.Publisher(
-                destination_class=docutils.io.StringOutput)
+            destination_class=docutils.io.StringOutput)
         pub.set_components('standalone', 'restructuredtext', 'html')
+        pub.writer.translator_class = PelicanHTMLTranslator
         pub.process_programmatic_settings(None, extra_params, None)
-        pub.set_source(source_path=filename)
+        pub.set_source(source_path=source_path)
         pub.publish()
         return pub
 
-    def read(self, filename):
+    def read(self, source_path):
         """Parses restructured text"""
-        pub = self._get_publisher(filename)
+        pub = self._get_publisher(source_path)
         parts = pub.writer.parts
         content = parts.get('body')
 
@@ -117,16 +140,28 @@ class MarkdownReader(Reader):
     file_extensions = ['md', 'markdown', 'mkd']
     extensions = ['codehilite', 'extra']
 
-    def read(self, filename):
+    def _parse_metadata(self, meta):
+        """Return the dict containing document metadata"""
+        md = Markdown(extensions=set(self.extensions + ['meta']))
+        output = {}
+        for name, value in meta.items():
+            name = name.lower()
+            if name == "summary":
+                summary_values = "\n".join(str(item) for item in value)
+                summary = md.convert(summary_values)
+                output[name] = self.process_metadata(name, summary)
+            else:
+                output[name] = self.process_metadata(name, value[0])
+        return output
+
+    def read(self, source_path):
         """Parse content and metadata of markdown files"""
-        with open(filename) as text:
+
+        with pelican_open(source_path) as text:
             md = Markdown(extensions=set(self.extensions + ['meta']))
             content = md.convert(text)
 
-        metadata = {}
+        metadata = self._parse_metadata(md.Meta)
-        for name, value in md.Meta.items():
-            name = name.lower()
-            metadata[name] = self.process_metadata(name, value[0])
         return content, metadata
 
 class HTMLReader(Reader):
@@ -223,7 +258,7 @@ class HTMLReader(Reader):
 
     def read(self, filename):
         """Parse content and metadata of markdown files"""
-        with open(filename) as content:
+        with pelican_open(filename) as content:
             parser = self._HTMLParser(self.settings)
             parser.feed(content)
             parser.close()
@@ -233,6 +268,37 @@ class HTMLReader(Reader):
             metadata[k] = self.process_metadata(k, parser.metadata[k])
         return parser.body, metadata
 
+class AsciiDocReader(Reader):
+    enabled = bool(asciidoc)
+    file_extensions = ['asc']
+    default_options = ["--no-header-footer", "-a newline=\\n"]
+
+    def read(self, source_path):
+        """Parse content and metadata of asciidoc files"""
+        from cStringIO import StringIO
+        text = StringIO(pelican_open(source_path))
+        content = StringIO()
+        ad = AsciiDocAPI()
+
+        options = self.settings.get('ASCIIDOC_OPTIONS', [])
+        if isinstance(options, (str, unicode)):
+            options = [m.strip() for m in options.split(',')]
+        options = self.default_options + options
+        for o in options:
+            ad.options(*o.split())
+
+        ad.execute(text, content, backend="html4")
+        content = content.getvalue()
+
+        metadata = {}
+        for name, value in ad.asciidoc.document.attributes.items():
+            name = name.lower()
+            metadata[name] = self.process_metadata(name, value)
+        if 'doctitle' in metadata:
+            metadata['title'] = metadata['doctitle']
+        return content, metadata
+
+
 _EXTENSIONS = {}
 
 for cls in Reader.__subclasses__():
@@ -240,13 +306,14 @@ for cls in Reader.__subclasses__():
         _EXTENSIONS[ext] = cls
 
 
-def read_file(filename, fmt=None, settings=None):
+def read_file(path, fmt=None, settings=None):
     """Return a reader object using the given format."""
+    base, ext = os.path.splitext(os.path.basename(path))
     if not fmt:
-        fmt = filename.split('.')[-1]
+        fmt = ext[1:]
 
     if fmt not in _EXTENSIONS:
-        raise TypeError('Pelican does not know how to parse %s' % filename)
+        raise TypeError('Pelican does not know how to parse {}'.format(path))
 
     reader = _EXTENSIONS[fmt](settings)
     settings_key = '%s_EXTENSIONS' % fmt.upper()
@@ -257,12 +324,22 @@ def read_file(filename, fmt=None, settings=None):
     if not reader.enabled:
         raise ValueError("Missing dependencies for %s" % fmt)
 
-    content, metadata = reader.read(filename)
+    content, metadata = reader.read(path)
 
     # eventually filter the content with typogrify if asked so
-    if settings and settings['TYPOGRIFY']:
+    if settings and settings.get('TYPOGRIFY'):
-        from typogrify import Typogrify
+        from typogrify.filters import typogrify
-        content = Typogrify.typogrify(content)
+        content = typogrify(content)
-        metadata['title'] = Typogrify.typogrify(metadata['title'])
+        metadata['title'] = typogrify(metadata['title'])
+
+    file_metadata = settings and settings.get('FILENAME_METADATA')
+    if file_metadata:
+        match = re.match(file_metadata, base)
+        if match:
+            # .items() for py3k compat.
+            for k, v in match.groupdict().items():
+                if k not in metadata:
+                    k = k.lower()  # metadata must be lowercase
+                    metadata[k] = reader.process_metadata(k, v)
 
     return content, metadata

pelican/rstdirectives.py

@@ -1,9 +1,12 @@
 # -*- coding: utf-8 -*-
-from docutils import nodes
+from __future__ import unicode_literals, print_function
-from docutils.parsers.rst import directives, Directive
+
+from docutils import nodes, utils
+from docutils.parsers.rst import directives, roles, Directive
 from pygments.formatters import HtmlFormatter
 from pygments import highlight
 from pygments.lexers import get_lexer_by_name, TextLexer
+import re
 
 INLINESTYLES = False
 DEFAULT = HtmlFormatter(noclasses=INLINESTYLES)
@@ -31,7 +34,7 @@ class Pygments(Directive):
         # take an arbitrary option if more than one is given
         formatter = self.options and VARIANTS[self.options.keys()[0]] \
                     or DEFAULT
-        parsed = highlight(u'\n'.join(self.content), lexer, formatter)
+        parsed = highlight('\n'.join(self.content), lexer, formatter)
         return [nodes.raw('', parsed, format='html')]
 
 directives.register_directive('code-block', Pygments)
@@ -94,3 +97,21 @@ class YouTube(Directive):
             nodes.raw('', '</div>', format='html')]
 
 directives.register_directive('youtube', YouTube)
+
+_abbr_re = re.compile('\((.*)\)$')
+
+
+class abbreviation(nodes.Inline, nodes.TextElement):
+    pass
+
+
+def abbr_role(typ, rawtext, text, lineno, inliner, options={}, content=[]):
+    text = utils.unescape(text)
+    m = _abbr_re.search(text)
+    if m is None:
+        return [abbreviation(text, text)], []
+    abbr = text[:m.start()].strip()
+    expl = m.group(1)
+    return [abbreviation(abbr, abbr, explanation=expl)], []
+
+roles.register_local_role('abbr', abbr_role)

pelican/server.py

@@ -0,0 +1,20 @@
+from __future__ import print_function
+try:
+    import SimpleHTTPServer as srvmod
+except ImportError:
+    import http.server as srvmod
+
+try:
+    import SocketServer as socketserver
+except ImportError:
+    import socketserver
+
+PORT = 8000
+
+Handler = srvmod.SimpleHTTPRequestHandler
+
+httpd = socketserver.TCPServer(("", PORT), Handler)
+
+print("serving at port", PORT)
+httpd.serve_forever()
+

pelican/settings.py

@@ -1,4 +1,10 @@
 # -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
+import six
+
+import copy
+import imp
+import inspect
 import os
 import locale
 import logging
@@ -21,18 +27,21 @@ _DEFAULT_CONFIG = {'PATH': '.',
                    'MARKUP': ('rst', 'md'),
                    'STATIC_PATHS': ['images', ],
                    'THEME_STATIC_PATHS': ['static', ],
-                   'FEED': 'feeds/all.atom.xml',
+                   'FEED_ALL_ATOM': 'feeds/all.atom.xml',
-                   'CATEGORY_FEED': 'feeds/%s.atom.xml',
+                   'CATEGORY_FEED_ATOM': 'feeds/%s.atom.xml',
-                   'TRANSLATION_FEED': 'feeds/all-%s.atom.xml',
+                   'TRANSLATION_FEED_ATOM': 'feeds/all-%s.atom.xml',
                    'FEED_MAX_ITEMS': '',
+                   'SITEURL': '',
                    'SITENAME': 'A Pelican Blog',
                    'DISPLAY_PAGES_ON_MENU': True,
                    'PDF_GENERATOR': False,
+                   'OUTPUT_SOURCES': False,
+                   'OUTPUT_SOURCES_EXTENSION': '.text',
+                   'USE_FOLDER_AS_CATEGORY': True,
                    'DEFAULT_CATEGORY': 'misc',
-                   'FALLBACK_ON_FS_DATE': True,
                    'WITH_FUTURE_DATES': True,
                    'CSS_FILE': 'main.css',
-                   'REVERSE_ARCHIVE_ORDER': False,
+                   'NEWEST_FIRST_ARCHIVES': True,
                    'REVERSE_CATEGORY_ORDER': False,
                    'DELETE_OUTPUT_DIRECTORY': False,
                    'ARTICLE_URL': '{slug}.html',
@@ -47,13 +56,14 @@ _DEFAULT_CONFIG = {'PATH': '.',
                    'CATEGORY_SAVE_AS': 'category/{slug}.html',
                    'TAG_URL': 'tag/{slug}.html',
                    'TAG_SAVE_AS': 'tag/{slug}.html',
-                   'AUTHOR_URL': u'author/{slug}.html',
+                   'AUTHOR_URL': 'author/{slug}.html',
-                   'AUTHOR_SAVE_AS': u'author/{slug}.html',
+                   'AUTHOR_SAVE_AS': 'author/{slug}.html',
                    'RELATIVE_URLS': True,
                    'DEFAULT_LANG': 'en',
                    'TAG_CLOUD_STEPS': 4,
                    'TAG_CLOUD_MAX_ITEMS': 100,
                    'DIRECT_TEMPLATES': ('index', 'tags', 'categories', 'archives'),
+                   'EXTRA_TEMPLATES_PATHS': [],
                    'PAGINATED_DIRECT_TEMPLATES': ('index', ),
                    'PELICAN_CLASS': 'pelican.Pelican',
                    'DEFAULT_DATE_FORMAT': '%a %d %B %Y',
@@ -63,59 +73,83 @@ _DEFAULT_CONFIG = {'PATH': '.',
                    'DEFAULT_PAGINATION': False,
                    'DEFAULT_ORPHANS': 0,
                    'DEFAULT_METADATA': (),
+                   'FILENAME_METADATA': '(?P<date>\d{4}-\d{2}-\d{2}).*',
                    'FILES_TO_COPY': (),
                    'DEFAULT_STATUS': 'published',
                    'ARTICLE_PERMALINK_STRUCTURE': '',
                    'TYPOGRIFY': False,
-                   'LESS_GENERATOR': False,
                    'SUMMARY_MAX_LENGTH': 50,
-                   'WEBASSETS': False,
                    'PLUGINS': [],
+                   'TEMPLATE_PAGES': {},
+                   'IGNORE_FILES': []
                    }
 
 
-def read_settings(filename=None):
+def read_settings(path=None, override=None):
-    if filename:
+    if path:
-        local_settings = get_settings_from_file(filename)
+        local_settings = get_settings_from_file(path)
+        # Make the paths relative to the settings file
+        for p in ['PATH', 'OUTPUT_PATH', 'THEME']:
+            if p in local_settings and local_settings[p] is not None \
+                    and not isabs(local_settings[p]):
+                absp = os.path.abspath(os.path.normpath(os.path.join(
+                            os.path.dirname(path), local_settings[p])))
+                if p != 'THEME' or os.path.exists(absp):
+                    local_settings[p] = absp
     else:
-        local_settings = _DEFAULT_CONFIG
+        local_settings = copy.deepcopy(_DEFAULT_CONFIG)
-    configured_settings = configure_settings(local_settings, None, filename)
+
-    return configured_settings
+    if override:
+        local_settings.update(override)
+
+    return configure_settings(local_settings)
 
 
-def get_settings_from_file(filename, default_settings=None):
+def get_settings_from_module(module=None, default_settings=_DEFAULT_CONFIG):
-    """Load a Python file into a dictionary.
     """
-    if default_settings == None:
+    Load settings from a module, returning a dict.
-        default_settings = _DEFAULT_CONFIG
+    """
-    context = default_settings.copy()
+
-    if filename:
+    context = copy.deepcopy(default_settings)
-        tempdict = {}
+    if module is not None:
-        execfile(filename, tempdict)
+        context.update(
-        for key in tempdict:
+                (k, v) for k, v in inspect.getmembers(module) if k.isupper())
-            if key.isupper():
-                context[key] = tempdict[key]
     return context
 
 
-def configure_settings(settings, default_settings=None, filename=None):
+def get_settings_from_file(path, default_settings=_DEFAULT_CONFIG):
-    """Provide optimizations, error checking, and warnings for loaded settings"""
+    """
-    if default_settings is None:
+    Load settings from a file path, returning a dict.
-        default_settings = _DEFAULT_CONFIG
 
-    # Make the paths relative to the settings file
+    """
-    if filename:
+
-        for path in ['PATH', 'OUTPUT_PATH']:
+    name = os.path.basename(path).rpartition('.')[0]
-            if path in settings:
+    module = imp.load_source(name, path)
-                if settings[path] is not None and not isabs(settings[path]):
+    return get_settings_from_module(module, default_settings=default_settings)
-                    settings[path] = os.path.abspath(os.path.normpath(
+
-                        os.path.join(os.path.dirname(filename), settings[path]))
+
-                    )
+def configure_settings(settings):
+    """
+    Provide optimizations, error checking, and warnings for loaded settings
+    """
+    if not 'PATH' in settings or not os.path.isdir(settings['PATH']):
+        raise Exception('You need to specify a path containing the content'
+                        ' (see pelican --help for more information)')
+
+    # find the theme in pelican.theme if the given one does not exists
+    if not os.path.isdir(settings['THEME']):
+        theme_path = os.sep.join([os.path.dirname(
+            os.path.abspath(__file__)), "themes/%s" % settings['THEME']])
+        if os.path.exists(theme_path):
+            settings['THEME'] = theme_path
+        else:
+            raise Exception("Impossible to find the theme %s"
+                            % settings['THEME'])
 
     # if locales is not a list, make it one
     locales = settings['LOCALE']
 
-    if isinstance(locales, basestring):
+    if isinstance(locales, six.string_types):
         locales = [locales]
 
     # try to set the different locales, fallback on the default.
@@ -124,8 +158,8 @@ def configure_settings(settings, default_settings=None, filename=None):
 
     for locale_ in locales:
         try:
-            locale.setlocale(locale.LC_ALL, locale_)
+            locale.setlocale(locale.LC_ALL, str(locale_))
-            break  # break if it is successfull
+            break  # break if it is successful
         except locale.Error:
             pass
     else:
@@ -142,10 +176,21 @@ def configure_settings(settings, default_settings=None, filename=None):
             settings['FEED_DOMAIN'] = settings['SITEURL']
 
     # Warn if feeds are generated with both SITEURL & FEED_DOMAIN undefined
-    if (('FEED' in settings) or ('FEED_RSS' in settings)) and (not 'FEED_DOMAIN' in settings):
+    feed_keys = ['FEED_ATOM', 'FEED_RSS',
-        logger.warn("Since feed URLs should always be absolute, you should specify "
+                 'FEED_ALL_ATOM', 'FEED_ALL_RSS',
-                 "FEED_DOMAIN in your settings. (e.g., 'FEED_DOMAIN = "
+                 'CATEGORY_FEED_ATOM', 'CATEGORY_FEED_RSS',
-                 "http://www.example.com')")
+                 'TAG_FEED_ATOM', 'TAG_FEED_RSS',
+                 'TRANSLATION_FEED_ATOM', 'TRANSLATION_FEED_RSS',
+                ]
+
+    if any(settings.get(k) for k in feed_keys):
+        if not settings.get('FEED_DOMAIN'):
+            logger.warn("Since feed URLs should always be absolute, you should specify "
+                     "FEED_DOMAIN in your settings. (e.g., 'FEED_DOMAIN = "
+                     "http://www.example.com')")
+
+        if not settings.get('SITEURL'):
+            logger.warn("Feeds generated without SITEURL set properly may not be valid")
 
     if not 'TIMEZONE' in settings:
         logger.warn("No timezone information specified in the settings. Assuming"
@@ -153,12 +198,23 @@ def configure_settings(settings, default_settings=None, filename=None):
                  "http://docs.notmyidea.org/alexis/pelican/settings.html#timezone "
                  "for more information")
 
-    if 'WEBASSETS' in settings and settings['WEBASSETS'] is not False:
+    if 'LESS_GENERATOR' in settings:
-        try:
+        logger.warn("The LESS_GENERATOR setting has been removed in favor "
-            from webassets.ext.jinja2 import AssetsExtension
+                    "of the Webassets plugin")
-            settings['JINJA_EXTENSIONS'].append(AssetsExtension)
+
-        except ImportError:
+    if 'OUTPUT_SOURCES_EXTENSION' in settings:
-            logger.warn("You must install the webassets module to use WEBASSETS.")
+        if not isinstance(settings['OUTPUT_SOURCES_EXTENSION'], six.string_types):
-            settings['WEBASSETS'] = False
+            settings['OUTPUT_SOURCES_EXTENSION'] = _DEFAULT_CONFIG['OUTPUT_SOURCES_EXTENSION']
+            logger.warn("Detected misconfiguration with OUTPUT_SOURCES_EXTENSION."
+                       " falling back to the default extension " +
+                       _DEFAULT_CONFIG['OUTPUT_SOURCES_EXTENSION'])
+
+    filename_metadata = settings.get('FILENAME_METADATA')
+    if filename_metadata and not isinstance(filename_metadata, six.string_types):
+        logger.error("Detected misconfiguration with FILENAME_METADATA"
+                " setting (must be string or compiled pattern), falling"
+                "back to the default")
+        settings['FILENAME_METADATA'] = \
+                _DEFAULT_CONFIG['FILENAME_METADATA']
 
     return settings

pelican/signals.py

@@ -1,5 +1,15 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
 from blinker import signal
 
 initialized = signal('pelican_initialized')
+finalized = signal('pelican_finalized')
+article_generate_preread = signal('article_generate_preread')
+generator_init = signal('generator_init')
 article_generate_context = signal('article_generate_context')
 article_generator_init = signal('article_generator_init')
+article_generator_finalized = signal('article_generate_finalized')
+get_generators = signal('get_generators')
+pages_generate_context = signal('pages_generate_context')
+pages_generator_init = signal('pages_generator_init')
+content_object_init = signal('content_object_init')

pelican/themes/notmyidea/static/css/main.css

@@ -70,9 +70,6 @@ p {margin-bottom: 1.143em;}
 strong, b {font-weight: bold;}
 em, i {font-style: italic;}
 
-::-moz-selection {background: #F6CF74; color: #fff;}
-::selection {background: #F6CF74; color: #fff;}
-
 /* Lists */
 ul {
 	list-style: outside disc;
@@ -100,7 +97,7 @@ dl {margin: 0 0 1.5em 0;}
 dt {font-weight: bold;}
 dd {margin-left: 1.5em;}
 
-pre{background-color: #000; padding: 10px; color: #fff; margin: 10px; overflow: auto;}
+pre{background-color:  rgb(238, 238, 238); padding: 10px; margin: 10px; overflow: auto;}
 
 /* Quotes */
 blockquote {
@@ -144,8 +141,8 @@ aside, nav, article, figure {
 
 /***** Layout *****/
 .body {clear: both; margin: 0 auto; width: 800px;}
-img.right figure.right {float: right; margin: 0 0 2em 2em;}
+img.right, figure.right {float: right; margin: 0 0 2em 2em;}
-img.left, figure.left {float: right; margin: 0 0 2em 2em;}
+img.left, figure.left {float: left; margin: 0 2em 2em 0;}
 
 /*
 	Header
@@ -163,7 +160,6 @@ img.left, figure.left {float: right; margin: 0 0 2em 2em;}
 		font-weight: bold;
 		margin: 0 0 .6em .2em;
 		text-decoration: none;
-		width: 427px;
 	}
 	#banner h1 a:hover, #banner h1 a:active {
 		background: none;
@@ -312,7 +308,11 @@ img.left, figure.left {float: right; margin: 0 0 2em 2em;}
 		.social a[type$='atom+xml'], .social a[type$='rss+xml'] {background-image: url('../images/icons/rss.png');}
 		.social a[href*='twitter.com'] {background-image: url('../images/icons/twitter.png');}
 		.social a[href*='linkedin.com'] {background-image: url('../images/icons/linkedin.png');}
-		.social a[href*='gitorious.org'] {background-image: url('../images/icons/gitorious.org');}
+		.social a[href*='gitorious.org'] {background-image: url('../images/icons/gitorious.png');}
+		.social a[href*='github.com'],
+		.social a[href*='git.io'] {background-image: url('../images/icons/github.png');}
+		.social a[href*='gittip.com'] {background-image: url('../images/icons/gittip.png');}
+		.social a[href*='plus.google.com'] {background-image: url('../images/icons/google-plus.png');}
 
 /*
 	About

pelican/themes/notmyidea/static/css/pygment.css

@@ -1,5 +1,5 @@
 .hll {
-background-color:#FFFFCC;
+background-color:#eee;
 }
 .c {
 color:#408090;

pelican/themes/notmyidea/templates/analytics.html

@@ -1,11 +1,12 @@
 {% if GOOGLE_ANALYTICS %}
     <script type="text/javascript">
-    var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
+    var _gaq = _gaq || [];
-    document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
+    _gaq.push(['_setAccount', '{{GOOGLE_ANALYTICS}}']);
+    _gaq.push(['_trackPageview']);
+    (function() {
+        var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+        ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+        var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+    })();
     </script>
-    <script type="text/javascript">
-    try {
-        var pageTracker = _gat._getTracker("{{GOOGLE_ANALYTICS}}");
-    pageTracker._trackPageview();
-    } catch(err) {}</script>
 {% endif %}

pelican/themes/notmyidea/templates/article.html

@@ -5,7 +5,7 @@
   <article>
     <header>
       <h1 class="entry-title">
-        <a href="{{ article.url }}" rel="bookmark"
+        <a href="{{ SITEURL }}/{{ article.url }}" rel="bookmark"
            title="Permalink to {{ article.title|striptags }}">{{ article.title}}</a></h1>
       {% include 'twitter.html' %}
     </header>

pelican/themes/notmyidea/templates/article_infos.html

@@ -10,5 +10,6 @@
         {% endif %}
 <p>In <a href="{{ SITEURL }}/{{ article.category.url }}">{{ article.category }}</a>. {% if PDF_PROCESSOR %}<a href="{{ SITEURL }}/pdf/{{ article.slug }}.pdf">get the pdf</a>{% endif %}</p>
 {% include 'taglist.html' %}
-{% include 'translations.html' %}
+{% import 'translations.html' as translations with context %}
+{{ translations.translations_for(article) }}
 </footer><!-- /.post-info -->

pelican/themes/notmyidea/templates/base.html

@@ -4,9 +4,11 @@
         <title>{% block title %}{{ SITENAME }}{%endblock%}</title>
         <meta charset="utf-8" />
         <link rel="stylesheet" href="{{ SITEURL }}/theme/css/{{ CSS_FILE }}" type="text/css" />
-        <link href="{{ FEED_DOMAIN }}/{{ FEED }}" type="application/atom+xml" rel="alternate" title="{{ SITENAME }} Atom Feed" />
+        {% if FEED_ALL_ATOM %}
-        {% if FEED_RSS %}
+        <link href="{{ FEED_DOMAIN }}/{{ FEED_ALL_ATOM }}" type="application/atom+xml" rel="alternate" title="{{ SITENAME }} Atom Feed" />
-        <link href="{{ FEED_DOMAIN }}/{{ FEED_RSS }}" type="application/rss+xml" rel="alternate" title="{{ SITENAME }} RSS Feed" />
+        {% endif %}
+        {% if FEED_ALL_RSS %}
+        <link href="{{ FEED_DOMAIN }}/{{ FEED_ALL_RSS }}" type="application/rss+xml" rel="alternate" title="{{ SITENAME }} RSS Feed" />
         {% endif %}
 
         <!--[if IE]>
@@ -24,7 +26,7 @@
 <body id="index" class="home">
 {% include 'github.html' %}
         <header id="banner" class="body">
-                <h1><a href="{{ SITEURL }}">{{ SITENAME }} {% if SITESUBTITLE %} <strong>{{ SITESUBTITLE }}</strong>{% endif %}</a></h1>
+                <h1><a href="{{ SITEURL }}/">{{ SITENAME }} {% if SITESUBTITLE %} <strong>{{ SITESUBTITLE }}</strong>{% endif %}</a></h1>
                 <nav><ul>
                 {% for title, link in MENUITEMS %}
                     <li><a href="{{ link }}">{{ title }}</a></li>
@@ -56,9 +58,9 @@
                 <div class="social">
                         <h2>social</h2>
                         <ul>
-                            <li><a href="{{ FEED_DOMAIN }}/{{ FEED }}" type="application/atom+xml" rel="alternate">atom feed</a></li>
+                            <li><a href="{{ FEED_DOMAIN }}/{{ FEED_ALL_ATOM }}" type="application/atom+xml" rel="alternate">atom feed</a></li>
-                            {% if FEED_RSS %}
+                            {% if FEED_ALL_RSS %}
-                            <li><a href="{{ FEED_DOMAIN }}/{{ FEED_RSS }}" type="application/rss+xml" rel="alternate">rss feed</a></li>
+                            <li><a href="{{ FEED_DOMAIN }}/{{ FEED_ALL_RSS }}" type="application/rss+xml" rel="alternate">rss feed</a></li>
                             {% endif %}
 
                         {% for name, link in SOCIAL %}
@@ -71,7 +73,7 @@
 
         <footer id="contentinfo" class="body">
                 <address id="about" class="vcard body">
-                Proudly powered by <a href="http://pelican.notmyidea.org/">Pelican</a>, which takes great advantage of <a href="http://python.org">Python</a>.
+                Proudly powered by <a href="http://getpelican.com/">Pelican</a>, which takes great advantage of <a href="http://python.org">Python</a>.
                 </address><!-- /#about -->
 
                 <p>The theme is by <a href="http://coding.smashingmagazine.com/2009/08/04/designing-a-html-5-layout-from-scratch/">Smashing Magazine</a>, thanks!</p>

pelican/themes/notmyidea/templates/page.html

@@ -3,6 +3,8 @@
 {% block content %}        
 <section id="content" class="body">    
     <h1 class="entry-title">{{ page.title }}</h1>
+    {% import 'translations.html' as translations with context %}
+    {{ translations.translations_for(page) }}
     {% if PDF_PROCESSOR %}<a href="{{ SITEURL }}/pdf/{{ page.slug }}.pdf">get
     the pdf</a>{% endif %}
     {{ page.content }}

pelican/themes/notmyidea/templates/translations.html

@@ -1,6 +1,8 @@
+{% macro translations_for(article) %}
 {% if article.translations %}
 Translations: 
     {% for translation in article.translations %}
         <a href="{{ SITEURL }}/{{ translation.url }}">{{ translation.lang }}</a>
     {% endfor %}
 {% endif %}
+{% endmacro %}

pelican/themes/simple/templates/article.html

@@ -5,6 +5,8 @@
     <h2 class="entry-title">
       <a href="{{ article.url }}" rel="bookmark"
          title="Permalink to {{ article.title|striptags }}">{{ article.title }}</a></h2>
+ {% import 'translations.html' as translations with context %}
+ {{ translations.translations_for(article) }}
   </header>
   <footer class="post-info">
     <abbr class="published" title="{{ article.date.isoformat() }}">

pelican/themes/simple/templates/base.html

@@ -4,6 +4,30 @@
         {% block head %}
         <title>{% block title %}{{ SITENAME }}{% endblock title %}</title>
         <meta charset="utf-8" />
+        {% if FEED_ALL_ATOM %}
+        <link href="{{ FEED_DOMAIN }}/{{ FEED_ALL_ATOM }}" type="application/atom+xml" rel="alternate" title="{{ SITENAME }} Full Atom Feed" />
+        {% endif %}
+        {% if FEED_ALL_RSS %}
+        <link href="{{ FEED_DOMAIN }}/{{ FEED_ALL_RSS }}" type="application/rss+xml" rel="alternate" title="{{ SITENAME }} Full RSS Feed" />
+        {% endif %}
+        {% if FEED_ATOM %}
+        <link href="{{ FEED_DOMAIN }}/{{ FEED_ATOM }}" type="application/atom+xml" rel="alternate" title="{{ SITENAME }} Atom Feed" />
+        {% endif %}
+        {% if FEED_RSS %}
+        <link href="{{ FEED_DOMAIN }}/{{ FEED_RSS }}" type="application/rss+xml" rel="alternate" title="{{ SITENAME }} RSS Feed" />
+        {% endif %}
+        {% if CATEGORY_FEED_ATOM %}
+        <link href="{{ FEED_DOMAIN }}/{{ CATEGORY_FEED_ATOM|format(category) }}" type="application/atom+xml" rel="alternate" title="{{ SITENAME }} Categories Atom Feed" />
+        {% endif %}
+        {% if CATEGORY_FEED_RSS %}
+        <link href="{{ FEED_DOMAIN }}/{{ CATEGORY_FEED_RSS|format(category) }}" type="application/rss+xml" rel="alternate" title="{{ SITENAME }} Categories RSS Feed" />
+        {% endif %}
+        {% if TAG_FEED_ATOM %}
+        <link href="{{ FEED_DOMAIN }}/{{ TAG_FEED_ATOM|format(tag) }}" type="application/atom+xml" rel="alternate" title="{{ SITENAME }} Tags Atom Feed" />
+        {% endif %}
+        {% if TAG_FEED_RSS %}
+        <link href="{{ FEED_DOMAIN }}/{{ TAG_FEED_RSS|format(tag) }}" type="application/rss+xml" rel="alternate" title="{{ SITENAME }} Tags RSS Feed" />
+        {% endif %}
         {% endblock head %}
 </head>
 
@@ -29,7 +53,7 @@
         {% endblock %}
         <footer id="contentinfo" class="body">
                 <address id="about" class="vcard body">
-                Proudly powered by <a href="http://pelican.notmyidea.org/">Pelican</a>,
+                Proudly powered by <a href="http://getpelican.com/">Pelican</a>,
                 which takes great advantage of <a href="http://python.org">Python</a>.
                 </address><!-- /#about -->
         </footer><!-- /#contentinfo -->

pelican/themes/simple/templates/index.html

@@ -1,14 +1,14 @@
 {% extends "base.html" %}
-{% block content %}        
+{% block content %}
 <section id="content">
 {% block content_title %}
 <h2>All articles</h2>
 {% endblock %}
 
 <ol id="post-list">
-{% for article in articles_page.object_list %}        
+{% for article in articles_page.object_list %}
-        <li><article class="hentry">    
+        <li><article class="hentry">
-                <header> <h2 class="entry-title"><a href="{{ SITEURL }}/{{ article.url }}" rel="bookmark" title="Permalink to {{ article.title}}">{{ article.title }}</a></h2> </header>
+                <header> <h2 class="entry-title"><a href="{{ SITEURL }}/{{ article.url }}" rel="bookmark" title="Permalink to {{ article.title|striptags }}">{{ article.title }}</a></h2> </header>
                 <footer class="post-info">
                     <abbr class="published" title="{{ article.date.isoformat() }}"> {{ article.locale_date }} </abbr>
                     {% if article.author %}<address class="vcard author">By <a class="url fn" href="{{ SITEURL }}/{{ article.author.url }}">{{ article.author }}</a></address>{% endif %}

pelican/themes/simple/templates/page.html

@@ -2,5 +2,8 @@
 {% block title %}{{ page.title }}{%endblock%}
 {% block content %}
     <h1>{{ page.title }}</h1>
+    {% import 'translations.html' as translations with context %}
+    {{ translations.translations_for(page) }}
+
     {{ page.content }}
 {% endblock %}

pelican/themes/simple/templates/translations.html

@@ -0,0 +1,9 @@
+{% macro translations_for(article) %}
+{% if article.translations %}
+Translations: 
+{% for translation in article.translations %}
+<a href="{{ SITEURL }}/{{ translation.url }}">{{ translation.lang }}</a>
+{% endfor %}
+{% endif %}
+{% endmacro %}
+

pelican/tools/pelican_import.py

@@ -1,6 +1,14 @@
 #!/usr/bin/env python
 
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
 import argparse
+try:
+    # py3k import
+    from html.parser import HTMLParser
+except ImportError:
+    # py2 import
+    from HTMLParser import HTMLParser  # NOQA
 import os
 import subprocess
 import sys
@@ -14,14 +22,14 @@ from pelican.utils import slugify
 def wp2fields(xml):
     """Opens a wordpress XML file, and yield pelican fields"""
     try:
-        from BeautifulSoup import BeautifulStoneSoup
+        from bs4 import BeautifulSoup
     except ImportError:
         error = ('Missing dependency '
-                 '"BeautifulSoup" required to import Wordpress XML files.')
+                 '"BeautifulSoup4" and "lxml" required to import Wordpress XML files.')
         sys.exit(error)
 
     xmlfile = open(xml, encoding='utf-8').read()
-    soup = BeautifulStoneSoup(xmlfile)
+    soup = BeautifulSoup(xmlfile, "xml")
     items = soup.rss.channel.findAll('item')
 
     for item in items:
@@ -29,7 +37,8 @@ def wp2fields(xml):
         if item.fetch('wp:status')[0].contents[0] == "publish":
 
             try:
-                title = item.title.contents[0]
+                # Use HTMLParser due to issues with BeautifulSoup 3
+                title = HTMLParser().unescape(item.title.contents[0])
             except IndexError:
                 continue
 
@@ -52,10 +61,10 @@ def wp2fields(xml):
 def dc2fields(file):
     """Opens a Dotclear export file, and yield pelican fields"""
     try:
-        from BeautifulSoup import BeautifulStoneSoup
+        from bs4 import BeautifulSoup
     except ImportError:
         error = ('Missing dependency '
-                 '"BeautifulSoup" required to import Dotclear files.')
+                 '"BeautifulSoup4" and "lxml" required to import Dotclear files.')
         sys.exit(error)
 
 
@@ -140,13 +149,27 @@ def dc2fields(file):
         if len(tag) > 1:
             if int(tag[:1]) == 1:
                 newtag = tag.split('"')[1]
-                tags.append(unicode(BeautifulStoneSoup(newtag,convertEntities=BeautifulStoneSoup.HTML_ENTITIES )))
+                tags.append(
+                    BeautifulSoup(
+                        newtag
+                        , "xml"
+                    )
+                    # bs4 always outputs UTF-8
+                    .decode('utf-8')
+                )
             else:
                 i=1
                 j=1
                 while(i <= int(tag[:1])):
                     newtag = tag.split('"')[j].replace('\\','')
-                    tags.append(unicode(BeautifulStoneSoup(newtag,convertEntities=BeautifulStoneSoup.HTML_ENTITIES )))
+                    tags.append(
+                        BeautifulSoup(
+                            newtag
+                            , "xml"
+                        )
+                        # bs4 always outputs UTF-8
+                        .decode('utf-8')
+                    )
                     i=i+1
                     if j < int(tag[:1])*2:
                         j=j+2
@@ -179,44 +202,53 @@ def feed2fields(file):
         yield (entry.title, entry.description, slug, date, author, [], tags, "html")
 
 
-def build_header(title, date, author, categories, tags):
+def build_header(title, date, author, categories, tags, slug):
     """Build a header from a list of fields"""
     header = '%s\n%s\n' % (title, '#' * len(title))
     if date:
         header += ':date: %s\n' % date
+    if author:
+        header += ':author: %s\n' % author
     if categories:
         header += ':category: %s\n' % ', '.join(categories)
     if tags:
         header += ':tags: %s\n' % ', '.join(tags)
+    if slug:
+        header += ':slug: %s\n' % slug
     header += '\n'
     return header
 
-def build_markdown_header(title, date, author, categories, tags):
+def build_markdown_header(title, date, author, categories, tags, slug):
     """Build a header from a list of fields"""
     header = 'Title: %s\n' % title
     if date:
         header += 'Date: %s\n' % date
+    if author:
+        header += 'Author: %s\n' % author
     if categories:
         header += 'Category: %s\n' % ', '.join(categories)
     if tags:
         header += 'Tags: %s\n' % ', '.join(tags)
+    if slug:
+        header += 'Slug: %s\n' % slug
     header += '\n'
     return header
 
-def fields2pelican(fields, out_markup, output_path, dircat=False, strip_raw=False):
+def fields2pelican(fields, out_markup, output_path, dircat=False, strip_raw=False, disable_slugs=False):
     for title, content, filename, date, author, categories, tags, in_markup in fields:
+        slug = not disable_slugs and filename or None
         if (in_markup == "markdown") or (out_markup == "markdown") :
             ext = '.md'
-            header = build_markdown_header(title, date, author, categories, tags)
+            header = build_markdown_header(title, date, author, categories, tags, slug)
         else:
             out_markup = "rst"
             ext = '.rst'
-            header = build_header(title, date, author, categories, tags)
+            header = build_header(title, date, author, categories, tags, slug)
 
         filename = os.path.basename(filename)
 
         # option to put files in directories with categories names
-        if dircat and (len(categories) == 1):
+        if dircat and (len(categories) > 0):
             catname = slugify(categories[0])
             out_filename = os.path.join(output_path, catname, filename+ext)
             if not os.path.isdir(os.path.join(output_path, catname)):
@@ -232,8 +264,8 @@ def fields2pelican(fields, out_markup, output_path, dircat=False, strip_raw=Fals
             with open(html_filename, 'w', encoding='utf-8') as fp:
                 # Replace newlines with paragraphs wrapped with <p> so
                 # HTML is valid before conversion
-                paragraphs = content.split('\n\n')
+                paragraphs = content.splitlines()
-                paragraphs = [u'<p>{}</p>'.format(p) for p in paragraphs]
+                paragraphs = ['<p>{0}</p>'.format(p) for p in paragraphs]
                 new_content = ''.join(paragraphs)
 
                 fp.write(new_content)
@@ -253,7 +285,7 @@ def fields2pelican(fields, out_markup, output_path, dircat=False, strip_raw=Fals
                 elif rc > 0:
                     error = "Please, check your Pandoc installation."
                     exit(error)
-            except OSError, e:
+            except OSError as e:
                 error = "Pandoc execution failed: %s" % e
                 exit(error)
 
@@ -272,8 +304,8 @@ def fields2pelican(fields, out_markup, output_path, dircat=False, strip_raw=Fals
 
 def main():
     parser = argparse.ArgumentParser(
-        description="Transform feed, Wordpress or Dotclear files to rst files."
+        description="Transform feed, Wordpress or Dotclear files to reST (rst) "
-            "Be sure to have pandoc installed",
+                    "or Markdown (md) files. Be sure to have pandoc installed.",
         formatter_class=argparse.ArgumentDefaultsHelpFormatter)
 
     parser.add_argument(dest='input', help='The input file to read')
@@ -292,6 +324,11 @@ def main():
     parser.add_argument('--strip-raw', action='store_true', dest='strip_raw',
         help="Strip raw HTML code that can't be converted to "
              "markup such as flash embeds or iframes (wordpress import only)")
+    parser.add_argument('--disable-slugs', action='store_true',
+        dest='disable_slugs',
+        help='Disable storing slugs from imported posts within output. '
+             'With this disabled, your Pelican URLs may not be consistent '
+             'with your original posts.')
 
     args = parser.parse_args()
 
@@ -322,4 +359,5 @@ def main():
 
     fields2pelican(fields, args.markup, args.output,
                    dircat=args.dircat or False,
-                   strip_raw=args.strip_raw or False)
+                   strip_raw=args.strip_raw or False,
+                   disable_slugs=args.disable_slugs or False)

pelican/tools/pelican_quickstart.py

@@ -1,19 +1,23 @@
 #!/usr/bin/env python
-# -*- coding: utf-8 -*- #
+
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
+import six
 
 import os
 import string
 import argparse
+import sys
+import codecs
 
 from pelican import __version__
 
-_TEMPLATES_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), \
+_TEMPLATES_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)),
                               "templates")
 
-
 CONF = {
-    'pelican' : 'pelican',
+    'pelican': 'pelican',
-    'pelicanopts' : '',
+    'pelicanopts': '',
     'basedir': '.',
     'ftp_host': 'localhost',
     'ftp_user': 'anonymous',
@@ -22,20 +26,41 @@ CONF = {
     'ssh_port': 22,
     'ssh_user': 'root',
     'ssh_target_dir': '/var/www',
-    'dropbox_dir' : '~/Dropbox/Public/',
+    'dropbox_dir': '~/Dropbox/Public/',
-    'default_pagination' : 10,
+    'default_pagination': 10,
     'siteurl': '',
     'lang': 'en'
 }
 
+def _input_compat(prompt):
+    if six.PY3:
+        r = input(prompt)
+    else:
+        r = raw_input(prompt).decode('utf-8')
+    return r
 
-def get_template(name):
+if six.PY3:
+    str_compat = str
+else:
+    str_compat = unicode
+
+def decoding_strings(f):
+    def wrapper(*args, **kwargs):
+        out = f(*args, **kwargs)
+        if isinstance(out, six.string_types):
+            # todo: make encoding configurable?
+            return out.decode(sys.stdin.encoding)
+        return out
+    return wrapper
+
+
+def get_template(name, as_encoding='utf-8'):
     template = os.path.join(_TEMPLATES_DIR, "{0}.in".format(name))
 
     if not os.path.isfile(template):
         raise RuntimeError("Cannot open {0}".format(template))
 
-    with open(template, 'r') as fd:
+    with codecs.open(template, 'r', as_encoding) as fd:
         line = fd.readline()
         while line:
             yield line
@@ -43,14 +68,15 @@ def get_template(name):
         fd.close()
 
 
-def ask(question, answer=str, default=None, l=None):
+@decoding_strings
-    if answer == str:
+def ask(question, answer=str_compat, default=None, l=None):
+    if answer == str_compat:
         r = ''
         while True:
             if default:
-                r = raw_input('> {0} [{1}] '.format(question, default))
+                r = _input_compat('> {0} [{1}] '.format(question, default))
             else:
-                r = raw_input('> {0} '.format(question, default))
+                r = _input_compat('> {0} '.format(question, default))
 
             r = r.strip()
 
@@ -64,7 +90,7 @@ def ask(question, answer=str, default=None, l=None):
                 if l and len(r) != l:
                     print('You must enter a {0} letters long string'.format(l))
                 else:
-                   break
+                    break
 
         return r
 
@@ -72,11 +98,11 @@ def ask(question, answer=str, default=None, l=None):
         r = None
         while True:
             if default is True:
-                r = raw_input('> {0} (Y/n) '.format(question))
+                r = _input_compat('> {0} (Y/n) '.format(question))
             elif default is False:
-                r = raw_input('> {0} (y/N) '.format(question))
+                r = _input_compat('> {0} (y/N) '.format(question))
             else:
-                r = raw_input('> {0} (y/n) '.format(question))
+                r = _input_compat('> {0} (y/n) '.format(question))
 
             r = r.strip().lower()
 
@@ -96,9 +122,9 @@ def ask(question, answer=str, default=None, l=None):
         r = None
         while True:
             if default:
-                r = raw_input('> {0} [{1}] '.format(question, default))
+                r = _input_compat('> {0} [{1}] '.format(question, default))
             else:
-                r = raw_input('> {0} '.format(question))
+                r = _input_compat('> {0} '.format(question))
 
             r = r.strip()
 
@@ -113,7 +139,7 @@ def ask(question, answer=str, default=None, l=None):
                 print('You must enter an integer')
         return r
     else:
-        raise NotImplemented('Argument `answer` must be str, bool, or integer')
+        raise NotImplemented('Argument `answer` must be str_compat, bool, or integer')
 
 
 def main():
@@ -135,23 +161,25 @@ def main():
 
 This script will help you create a new Pelican-based website.
 
-Please answer the following questions so this script can generate the files needed by Pelican.
+Please answer the following questions so this script can generate the files
+needed by Pelican.
 
     '''.format(v=__version__))
 
-    project = os.path.join(os.environ['VIRTUAL_ENV'], '.project')
+    project = os.path.join(os.environ.get('VIRTUAL_ENV', '.'), '.project')
     if os.path.isfile(project):
         CONF['basedir'] = open(project, 'r').read().rstrip("\n")
-        print('Using project associated with current virtual environment. Will save to:\n%s\n' % CONF['basedir'])
+        print('Using project associated with current virtual environment.'
+              'Will save to:\n%s\n' % CONF['basedir'])
     else:
-        CONF['basedir'] = os.path.abspath(ask('Where do you want to create your new web site?', answer=str, default=args.path))
+        CONF['basedir'] = os.path.abspath(ask('Where do you want to create your new web site?', answer=str_compat, default=args.path))
 
-    CONF['sitename'] = ask('What will be the title of this web site?', answer=str, default=args.title)
+    CONF['sitename'] = ask('What will be the title of this web site?', answer=str_compat, default=args.title)
-    CONF['author'] = ask('Who will be the author of this web site?', answer=str, default=args.author)
+    CONF['author'] = ask('Who will be the author of this web site?', answer=str_compat, default=args.author)
-    CONF['lang'] = ask('What will be the default language of this web site?', str, args.lang or CONF['lang'], 2)
+    CONF['lang'] = ask('What will be the default language of this web site?', str_compat, args.lang or CONF['lang'], 2)
 
     if ask('Do you want to specify a URL prefix? e.g., http://example.com  ', answer=bool, default=True):
-        CONF['siteurl'] = ask('What is your URL prefix? (see above example; no trailing slash)', str, CONF['siteurl'])
+        CONF['siteurl'] = ask('What is your URL prefix? (see above example; no trailing slash)', str_compat, CONF['siteurl'])
 
     CONF['with_pagination'] = ask('Do you want to enable article pagination?', bool, bool(CONF['default_pagination']))
 
@@ -161,57 +189,77 @@ Please answer the following questions so this script can generate the files need
         CONF['default_pagination'] = False
 
     mkfile = ask('Do you want to generate a Makefile to easily manage your website?', bool, True)
+    develop = ask('Do you want an auto-reload & simpleHTTP script to assist with theme and site development?', bool, True)
 
     if mkfile:
         if ask('Do you want to upload your website using FTP?', answer=bool, default=False):
-            CONF['ftp_host'] = ask('What is the hostname of your FTP server?', str, CONF['ftp_host'])
+            CONF['ftp_host'] = ask('What is the hostname of your FTP server?', str_compat, CONF['ftp_host'])
-            CONF['ftp_user'] = ask('What is your username on that server?', str, CONF['ftp_user'])
+            CONF['ftp_user'] = ask('What is your username on that server?', str_compat, CONF['ftp_user'])
-            CONF['ftp_target_dir'] = ask('Where do you want to put your web site on that server?', str, CONF['ftp_target_dir']) 
+            CONF['ftp_target_dir'] = ask('Where do you want to put your web site on that server?', str_compat, CONF['ftp_target_dir'])
         if ask('Do you want to upload your website using SSH?', answer=bool, default=False):
-            CONF['ssh_host'] = ask('What is the hostname of your SSH server?', str, CONF['ssh_host'])
+            CONF['ssh_host'] = ask('What is the hostname of your SSH server?', str_compat, CONF['ssh_host'])
             CONF['ssh_port'] = ask('What is the port of your SSH server?', int, CONF['ssh_port'])
-            CONF['ssh_user'] = ask('What is your username on that server?', str, CONF['ssh_user'])
+            CONF['ssh_user'] = ask('What is your username on that server?', str_compat, CONF['ssh_user'])
-            CONF['ssh_target_dir'] = ask('Where do you want to put your web site on that server?', str, CONF['ssh_target_dir'])
+            CONF['ssh_target_dir'] = ask('Where do you want to put your web site on that server?', str_compat, CONF['ssh_target_dir'])
         if ask('Do you want to upload your website using Dropbox?', answer=bool, default=False):
-            CONF['dropbox_dir'] = ask('Where is your Dropbox directory?', str, CONF['dropbox_dir'])
+            CONF['dropbox_dir'] = ask('Where is your Dropbox directory?', str_compat, CONF['dropbox_dir'])
 
     try:
         os.makedirs(os.path.join(CONF['basedir'], 'content'))
-    except OSError, e:
+    except OSError as e:
         print('Error: {0}'.format(e))
 
     try:
         os.makedirs(os.path.join(CONF['basedir'], 'output'))
-    except OSError, e:
+    except OSError as e:
         print('Error: {0}'.format(e))
 
     try:
-        with open(os.path.join(CONF['basedir'], 'pelicanconf.py'), 'w') as fd:
+        with codecs.open(os.path.join(CONF['basedir'], 'pelicanconf.py'), 'w', 'utf-8') as fd:
+            conf_python = dict()
+            for key, value in CONF.items():
+                conf_python[key] = repr(value)
+
             for line in get_template('pelicanconf.py'):
                 template = string.Template(line)
-                fd.write(template.safe_substitute(CONF))
+                fd.write(template.safe_substitute(conf_python))
             fd.close()
-    except OSError, e:
+    except OSError as e:
         print('Error: {0}'.format(e))
 
     try:
-        with open(os.path.join(CONF['basedir'], 'publishconf.py'), 'w') as fd:
+        with codecs.open(os.path.join(CONF['basedir'], 'publishconf.py'), 'w', 'utf-8') as fd:
             for line in get_template('publishconf.py'):
                 template = string.Template(line)
                 fd.write(template.safe_substitute(CONF))
             fd.close()
-    except OSError, e:
+    except OSError as e:
         print('Error: {0}'.format(e))
 
     if mkfile:
-
         try:
-            with open(os.path.join(CONF['basedir'], 'Makefile'), 'w') as fd:
+            with codecs.open(os.path.join(CONF['basedir'], 'Makefile'), 'w', 'utf-8') as fd:
                 for line in get_template('Makefile'):
                     template = string.Template(line)
                     fd.write(template.safe_substitute(CONF))
                 fd.close()
-        except OSError, e:
+        except OSError as e:
+            print('Error: {0}'.format(e))
+
+    if develop:
+        conf_shell = dict()
+        for key, value in CONF.items():
+            if isinstance(value, six.string_types) and ' ' in value:
+                value = '"' + value.replace('"', '\\"') + '"'
+            conf_shell[key] = value
+        try:
+            with codecs.open(os.path.join(CONF['basedir'], 'develop_server.sh'), 'w', 'utf-8') as fd:
+                for line in get_template('develop_server.sh'):
+                    template = string.Template(line)
+                    fd.write(template.safe_substitute(conf_shell))
+                fd.close()
+                os.chmod((os.path.join(CONF['basedir'], 'develop_server.sh')), 493) # mode 0o755
+        except OSError as e:
             print('Error: {0}'.format(e))
 
     print('Done. Your new project is available at %s' % CONF['basedir'])

pelican/tools/pelican_themes.py

@@ -1,5 +1,8 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
+
+import six
 
 import argparse
 import os
@@ -28,7 +31,7 @@ _BUILTIN_THEMES = ['simple', 'notmyidea']
 
 def err(msg, die=None):
     """Print an error message and exits if an exit code is given"""
-    sys.stderr.write(str(msg) + '\n')
+    sys.stderr.write(msg + '\n')
     if die:
         sys.exit((die if type(die) is int else 1))
 
@@ -180,7 +183,19 @@ def install(path, v=False, u=False):
                 print("Copying `{p}' to `{t}' ...".format(p=path, t=theme_path))
             try:
                 shutil.copytree(path, theme_path)
-            except Exception, e:
+
+                try:
+                    if os.name == 'posix':
+                        for root, dirs, files in os.walk(theme_path):
+                            for d in dirs:
+                                dname = os.path.join(root, d)
+                                os.chmod(dname, 493) # 0o755
+                            for f in files:
+                                fname = os.path.join(root, f)
+                                os.chmod(fname, 420) # 0o644
+                except OSError as e:
+                    err("Cannot change permissions of files or directory in `{r}':\n{e}".format(r=theme_path, e=str(e)), die=False)
+            except Exception as e:
                 err("Cannot copy `{p}' to `{t}':\n{e}".format(p=path, t=theme_path, e=str(e)))
 
 
@@ -200,7 +215,7 @@ def symlink(path, v=False):
                 print("Linking `{p}' to `{t}' ...".format(p=path, t=theme_path))
             try:
                 os.symlink(path, theme_path)
-            except Exception, e:
+            except Exception as e:
                 err("Cannot link `{p}' to `{t}':\n{e}".format(p=path, t=theme_path, e=str(e)))
 
 
@@ -221,7 +236,7 @@ def clean(v=False):
                     print('Removing {0}'.format(path))
                 try:
                     os.remove(path)
-                except OSError, e:
+                except OSError as e:
                     print('Error: cannot remove {0}'.format(path))
                 else:
                     c+=1

pelican/tools/templates/Makefile.in

@@ -1,7 +1,7 @@
 PELICAN=$pelican
 PELICANOPTS=$pelicanopts
 
-BASEDIR=$$(PWD)
+BASEDIR=$$(CURDIR)
 INPUTDIR=$$(BASEDIR)/content
 OUTPUTDIR=$$(BASEDIR)/output
 CONFFILE=$$(BASEDIR)/pelicanconf.py
@@ -24,11 +24,15 @@ help:
 	@echo 'Usage:                                                                 '
 	@echo '   make html                        (re)generate the web site          '
 	@echo '   make clean                       remove the generated files         '
+	@echo '   make regenerate                  regenerate files upon modification '
 	@echo '   make publish                     generate using production settings '
-	@echo '   ftp_upload                       upload the web site via FTP        '
+	@echo '   make serve                       serve site at http://localhost:8000'
+	@echo '   make devserver                   start/restart develop_server.sh    '
 	@echo '   ssh_upload                       upload the web site via SSH        '
+	@echo '   rsync_upload                     upload the web site via rsync+ssh  '
 	@echo '   dropbox_upload                   upload the web site via Dropbox    '
-	@echo '   rsync_upload                     upload the web site via rsync/ssh  '
+	@echo '   ftp_upload                       upload the web site via FTP        '
+	@echo '   github                           upload the web site via gh-pages   '
 	@echo '                                                                       '
 
 
@@ -45,19 +49,22 @@ regenerate: clean
 	$$(PELICAN) -r $$(INPUTDIR) -o $$(OUTPUTDIR) -s $$(CONFFILE) $$(PELICANOPTS)
 
 serve:
-	cd $$(OUTPUTDIR) && python -m SimpleHTTPServer
+	cd $$(OUTPUTDIR) && python -m pelican.server
+
+devserver:
+	$$(BASEDIR)/develop_server.sh restart
 
 publish:
 	$$(PELICAN) $$(INPUTDIR) -o $$(OUTPUTDIR) -s $$(PUBLISHCONF) $$(PELICANOPTS)
 
-dropbox_upload: publish
-	cp -r $$(OUTPUTDIR)/* $$(DROPBOX_DIR)
-
 ssh_upload: publish
 	scp -P $$(SSH_PORT) -r $$(OUTPUTDIR)/* $$(SSH_USER)@$$(SSH_HOST):$$(SSH_TARGET_DIR)
 
 rsync_upload: publish
-	rsync -e "ssh -p $(SSH_PORT)" -P -rvz --delete $(OUTPUTDIR)/* $(SSH_USER)@$(SSH_HOST):$(SSH_TARGET_DIR)
+	rsync -e "ssh -p $(SSH_PORT)" -P -rvz --delete $(OUTPUTDIR) $(SSH_USER)@$(SSH_HOST):$(SSH_TARGET_DIR)
+
+dropbox_upload: publish
+	cp -r $$(OUTPUTDIR)/* $$(DROPBOX_DIR)
 
 ftp_upload: publish
 	lftp ftp://$$(FTP_USER)@$$(FTP_HOST) -e "mirror -R $$(OUTPUTDIR) $$(FTP_TARGET_DIR) ; quit"
@@ -66,4 +73,4 @@ github: publish
 	ghp-import $$(OUTPUTDIR)
 	git push origin gh-pages
 
-.PHONY: html help clean regenerate serve publish ftp_upload ssh_upload rsync_upload dropbox_upload github
+.PHONY: html help clean regenerate serve devserver publish ssh_upload rsync_upload dropbox_upload ftp_upload github

pelican/tools/templates/develop_server.sh.in

@@ -0,0 +1,84 @@
+#!/usr/bin/env bash
+##
+# This section should match your Makefile
+##
+PELICAN=$pelican
+PELICANOPTS=$pelicanopts
+
+BASEDIR=$$(pwd)
+INPUTDIR=$$BASEDIR/content
+OUTPUTDIR=$$BASEDIR/output
+CONFFILE=$$BASEDIR/pelicanconf.py
+
+###
+# Don't change stuff below here unless you are sure
+###
+
+SRV_PID=$$BASEDIR/srv.pid
+PELICAN_PID=$$BASEDIR/pelican.pid
+
+function usage(){
+  echo "usage: $$0 (stop) (start) (restart)"
+  echo "This starts pelican in debug and reload mode and then launches"
+  echo "A pelican.server to help site development. It doesn't read"
+  echo "your pelican options so you edit any paths in your Makefile"
+  echo "you will need to edit it as well"
+  exit 3
+}
+
+function shut_down(){
+  if [[ -f $$SRV_PID ]]; then
+    PID=$$(cat $$SRV_PID)
+    PROCESS=$$(ps -p $$PID | tail -n 1 | awk '{print $$4}')
+    if [[ $$PROCESS != "" ]]; then
+      echo "Killing pelican.server"
+      kill $$PID
+    else
+      echo "Stale PID, deleting"
+    fi
+    rm $$SRV_PID
+  else
+    echo "pelican.server PIDFile not found"
+  fi
+
+  if [[ -f $$PELICAN_PID ]]; then
+    PID=$$(cat $$PELICAN_PID)
+    PROCESS=$$(ps -p $$PID | tail -n 1 | awk '{print $$4}')
+    if [[ $$PROCESS != "" ]]; then
+      echo "Killing Pelican"
+      kill $$PID
+    else
+      echo "Stale PID, deleting"
+    fi
+    rm $$PELICAN_PID
+  else
+    echo "Pelican PIDFile not found"
+  fi
+}
+
+function start_up(){
+  echo "Starting up Pelican and pelican.server"
+  shift
+  $$PELICAN --debug --autoreload -r $$INPUTDIR -o $$OUTPUTDIR -s $$CONFFILE $$PELICANOPTS &
+  echo $$! > $$PELICAN_PID
+  cd $$OUTPUTDIR
+  python -m pelican.server &
+  echo $$! > $$SRV_PID
+  cd $$BASEDIR
+  sleep 1 && echo 'Pelican and pelican.server processes now running in background.'
+}
+
+###
+#  MAIN
+###
+[[ $$# -ne 1 ]] && usage
+if [[ $$1 == "stop" ]]; then
+  shut_down
+elif [[ $$1 == "restart" ]]; then
+  shut_down
+  start_up
+elif [[ $$1 == "start" ]]; then
+  start_up
+else
+  usage
+fi

pelican/tools/templates/pelicanconf.py.in

@@ -1,13 +1,13 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*- #
 
-AUTHOR = u"$author"
+AUTHOR = $author
-SITENAME = u"$sitename"
+SITENAME = $sitename
 SITEURL = ''
 
 TIMEZONE = 'Europe/Paris'
 
-DEFAULT_LANG = '$lang'
+DEFAULT_LANG = $lang
 
 # Blogroll
 LINKS =  (('Pelican', 'http://docs.notmyidea.org/alexis/pelican/'),

pelican/tools/templates/publishconf.py.in

@@ -1,6 +1,8 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*- #
 
+import sys
+sys.path.append('.')
 from pelicanconf import *
 
 SITEURL = '$siteurl'

pelican/utils.py

@@ -1,12 +1,20 @@
 # -*- coding: utf-8 -*-
+from __future__ import unicode_literals, print_function
+import six
+
 import os
 import re
 import pytz
 import shutil
+import traceback
 import logging
-from collections import defaultdict
+import errno
+import locale
+import fnmatch
+from collections import defaultdict, Hashable
+from functools import partial
 
-from codecs import open as _open
+from codecs import open
 from datetime import datetime
 from itertools import groupby
 from jinja2 import Markup
@@ -15,6 +23,149 @@ from operator import attrgetter
 logger = logging.getLogger(__name__)
 
 
+def strftime(date, date_format):
+    """
+    Replacement for the builtin strftime().
+
+    This :func:`strftime()` is compatible to Python 2 and 3. In both cases,
+    input and output is always unicode.
+        
+    Still, Python 3's :func:`strftime()` seems to somehow "normalize" unicode
+    chars in the format string. So if e.g. your format string contains 'ø' or
+    'ä', the result will be 'o' and 'a'.
+
+    See here for an `extensive testcase <https://github.com/dmdm/test_strftime>`_.
+
+    :param date: Any object that sports a :meth:`strftime()` method.
+    :param date_format: Format string, can always be unicode.
+    :returns: Unicode string with formatted date.
+    """
+    # As tehkonst confirmed, above mentioned testcase runs correctly on
+    # Python 2 and 3 on Windows as well. Thanks.
+    if six.PY3:
+        # It could be so easy... *sigh*
+        return date.strftime(date_format)
+        # TODO Perhaps we should refactor again, so that the
+        # xmlcharrefreplace-regex-dance is always done, regardless
+        # of the Python version.
+    else:
+        # We must ensure that the format string is an encoded byte
+        # string, ASCII only WTF!!!
+        # But with "xmlcharrefreplace" our formatted date will produce
+        # *yuck* like this:
+        #        "Øl trinken beim Besäufnis"
+        #    --> "&#216;l trinken beim Bes&#228;ufnis"
+        date_format = date_format.encode('ascii',
+            errors="xmlcharrefreplace")
+        result = date.strftime(date_format)
+        # strftime() returns an encoded byte string
+        # which we must decode into unicode.
+        lang_code, enc = locale.getlocale(locale.LC_ALL)
+        if enc:
+            result = result.decode(enc)
+        else:
+            result = unicode(result)
+        # Convert XML character references back to unicode characters.
+        if "&#" in result:
+            result = re.sub(r'&#(?P<num>\d+);'
+                , lambda m: unichr(int(m.group('num')))
+                , result
+            )
+        return result
+
+
+
+#----------------------------------------------------------------------------
+# Stolen from Django: django.utils.encoding
+#
+
+def python_2_unicode_compatible(klass):
+    """
+    A decorator that defines __unicode__ and __str__ methods under Python 2.
+    Under Python 3 it does nothing.
+
+    To support Python 2 and 3 with a single code base, define a __str__ method
+    returning text and apply this decorator to the class.
+    """
+    if not six.PY3:
+        klass.__unicode__ = klass.__str__
+        klass.__str__ = lambda self: self.__unicode__().encode('utf-8')
+    return klass
+
+#----------------------------------------------------------------------------
+
+class NoFilesError(Exception):
+    pass
+
+
+class memoized(object):
+   '''Decorator. Caches a function's return value each time it is called.
+   If called later with the same arguments, the cached value is returned
+   (not reevaluated).
+   '''
+   def __init__(self, func):
+      self.func = func
+      self.cache = {}
+   def __call__(self, *args):
+      if not isinstance(args, Hashable):
+         # uncacheable. a list, for instance.
+         # better to not cache than blow up.
+         return self.func(*args)
+      if args in self.cache:
+         return self.cache[args]
+      else:
+         value = self.func(*args)
+         self.cache[args] = value
+         return value
+   def __repr__(self):
+      '''Return the function's docstring.'''
+      return self.func.__doc__
+   def __get__(self, obj, objtype):
+      '''Support instance methods.'''
+      return partial(self.__call__, obj)
+
+
+def deprecated_attribute(old, new, since=None, remove=None, doc=None):
+    """Attribute deprecation decorator for gentle upgrades
+
+    For example:
+
+        class MyClass (object):
+            @deprecated_attribute(
+                old='abc', new='xyz', since=(3, 2, 0), remove=(4, 1, 3))
+            def abc(): return None
+
+            def __init__(self):
+                xyz = 5
+
+    Note that the decorator needs a dummy method to attach to, but the
+    content of the dummy method is ignored.
+    """
+    def _warn():
+        version = '.'.join(six.text_type(x) for x in since)
+        message = ['{} has been deprecated since {}'.format(old, version)]
+        if remove:
+            version = '.'.join(six.text_type(x) for x in remove)
+            message.append(
+                ' and will be removed by version {}'.format(version))
+        message.append('.  Use {} instead.'.format(new))
+        logger.warning(''.join(message))
+        logger.debug(''.join(
+                six.text_type(x) for x in traceback.format_stack()))
+
+    def fget(self):
+        _warn()
+        return getattr(self, new)
+
+    def fset(self, value):
+        _warn()
+        setattr(self, new, value)
+
+    def decorator(dummy):
+        return property(fget=fget, fset=fset, doc=doc)
+
+    return decorator
+
 def get_date(string):
     """Return a datetime object from a string.
 
@@ -34,12 +185,13 @@ def get_date(string):
     raise ValueError("'%s' is not a valid date" % string)
 
 
-class open(object):
+class pelican_open(object):
     """Open a file and return it's content"""
     def __init__(self, filename):
         self.filename = filename
+
     def __enter__(self):
-        return _open(self.filename, encoding='utf-8').read()
+        return open(self.filename, encoding='utf-8').read()
 
     def __exit__(self, exc_type, exc_value, traceback):
         pass
@@ -51,12 +203,24 @@ def slugify(value):
 
     Took from django sources.
     """
+    # TODO Maybe steal again from current Django 1.5dev
     value = Markup(value).striptags()
-    if type(value) == unicode:
+    # value must be unicode per se
-        import unicodedata
+    import unicodedata
-        value = unicodedata.normalize('NFKD', value).encode('ascii', 'ignore')
+    from unidecode import unidecode
-    value = unicode(re.sub('[^\w\s-]', '', value).strip().lower())
+    # unidecode returns str in Py2 and 3, so in Py2 we have to make
-    return re.sub('[-\s]+', '-', value)
+    # it unicode again
+    value = unidecode(value)
+    if isinstance(value, six.binary_type):
+        value = value.decode('ascii')
+    # still unicode
+    value = unicodedata.normalize('NFKD', value)
+    value = re.sub('[^\w\s-]', '', value).strip().lower()
+    value = re.sub('[-\s]+', '-', value)
+    # we want only ASCII chars
+    value = value.encode('ascii', 'ignore')
+    # but Pelican should generally use only unicode
+    return value.decode('ascii')
 
 
 def copy(path, source, destination, destination_path=None, overwrite=False):
@@ -86,16 +250,32 @@ def copy(path, source, destination, destination_path=None, overwrite=False):
             if overwrite:
                 shutil.rmtree(destination_)
                 shutil.copytree(source_, destination_)
-                logger.info('replacement of %s with %s' % (source_, destination_))
+                logger.info('replacement of %s with %s' % (source_,
+                    destination_))
 
     elif os.path.isfile(source_):
+        dest_dir = os.path.dirname(destination_)
+        if not os.path.exists(dest_dir):
+            os.makedirs(dest_dir)
         shutil.copy(source_, destination_)
         logger.info('copying %s to %s' % (source_, destination_))
-
+    else:
+        logger.warning('skipped copy %s to %s' % (source_, destination_))
 
 def clean_output_dir(path):
     """Remove all the files from the output directory"""
 
+    if not os.path.exists(path):
+        logger.debug("Directory already removed: %s" % path)
+        return
+
+    if not os.path.isdir(path):
+        try:
+            os.remove(path)
+        except Exception as e:
+            logger.error("Unable to delete file %s; %e" % path, e)
+        return
+
     # remove all the existing content from the output folder
     for filename in os.listdir(path):
         file = os.path.join(path, filename)
@@ -103,21 +283,25 @@ def clean_output_dir(path):
             try:
                 shutil.rmtree(file)
                 logger.debug("Deleted directory %s" % file)
-            except Exception, e:
+            except Exception as e:
                 logger.error("Unable to delete directory %s; %e" % file, e)
         elif os.path.isfile(file) or os.path.islink(file):
             try:
                 os.remove(file)
                 logger.debug("Deleted file/link %s" % file)
-            except Exception, e:
+            except Exception as e:
                 logger.error("Unable to delete file %s; %e" % file, e)
         else:
             logger.error("Unable to delete %s, file type unknown" % file)
 
 
-def get_relative_path(filename):
+def get_relative_path(path):
-    """Return the relative path to the given filename"""
+    """Return the relative path from the given path to the root path."""
-    return '../' * filename.count('/') + '.'
+    nslashes = path.count('/')
+    if nslashes == 0:
+        return '.'
+    else:
+        return '/'.join(['..'] * nslashes)
 
 
 def truncate_html_words(s, num, end_text='...'):
@@ -131,7 +315,7 @@ def truncate_html_words(s, num, end_text='...'):
     """
     length = int(num)
     if length <= 0:
-        return u''
+        return ''
     html4_singlets = ('br', 'col', 'link', 'base', 'img', 'param', 'area',
                       'hr', 'input')
 
@@ -205,34 +389,31 @@ def process_translations(content_list):
     for slug, items in grouped_by_slugs:
         items = list(items)
         # find items with default language
-        default_lang_items = filter(attrgetter('in_default_lang'), items)
+        default_lang_items = list(filter(attrgetter('in_default_lang'), items))
         len_ = len(default_lang_items)
         if len_ > 1:
-            logger.warning(u'there are %s variants of "%s"' % (len_, slug))
+            logger.warning('there are %s variants of "%s"' % (len_, slug))
             for x in default_lang_items:
-                logger.warning('    %s' % x.filename)
+                logger.warning('    {}'.format(x.source_path))
         elif len_ == 0:
             default_lang_items = items[:1]
 
         if not slug:
-            msg = 'empty slug for %r. ' % default_lang_items[0].filename\
+            logger.warning((
-                + 'You can fix this by adding a title or a slug to your '\
+                    'empty slug for {!r}. '
-                + 'content'
+                    'You can fix this by adding a title or a slug to your '
-            logger.warning(msg)
+                    'content'
+                    ).format(default_lang_items[0].source_path))
         index.extend(default_lang_items)
-        translations.extend(filter(
+        translations.extend([x for x in items if x not in default_lang_items])
-            lambda x: x not in default_lang_items,
-            items
-        ))
         for a in items:
-            a.translations = filter(lambda x: x != a, items)
+            a.translations = [x for x in items if x != a]
     return index, translations
 
 
 LAST_MTIME = 0
 
-
+def files_changed(path, extensions, ignores=[]):
-def files_changed(path, extensions):
     """Return True if the files have changed since the last check"""
 
     def file_times(path):
@@ -240,28 +421,32 @@ def files_changed(path, extensions):
         for root, dirs, files in os.walk(path):
             dirs[:] = [x for x in dirs if x[0] != '.']
             for f in files:
-                if any(f.endswith(ext) for ext in extensions):
+                if any(f.endswith(ext) for ext in extensions) \
+                        and not any(fnmatch.fnmatch(f, ignore) for ignore in ignores):
                     yield os.stat(os.path.join(root, f)).st_mtime
 
     global LAST_MTIME
-    mtime = max(file_times(path))
+    try:
-    if mtime > LAST_MTIME:
+        mtime = max(file_times(path))
-        LAST_MTIME = mtime
+        if mtime > LAST_MTIME:
-        return True
+            LAST_MTIME = mtime
+            return True
+    except ValueError:
+        raise NoFilesError("No files with the given extension(s) found.")
     return False
 
 
 FILENAMES_MTIMES = defaultdict(int)
 
 
-def file_changed(filename):
+def file_changed(path):
-    mtime = os.stat(filename).st_mtime
+    mtime = os.stat(path).st_mtime
-    if FILENAMES_MTIMES[filename] == 0:
+    if FILENAMES_MTIMES[path] == 0:
-        FILENAMES_MTIMES[filename] = mtime
+        FILENAMES_MTIMES[path] = mtime
         return False
     else:
-        if mtime > FILENAMES_MTIMES[filename]:
+        if mtime > FILENAMES_MTIMES[path]:
-            FILENAMES_MTIMES[filename] = mtime
+            FILENAMES_MTIMES[path] = mtime
             return True
         return False
 
@@ -276,3 +461,11 @@ def set_date_tzinfo(d, tz_name=None):
         return tz.localize(d)
     else:
         return d
+
+
+def mkdir_p(path):
+    try:
+        os.makedirs(path)
+    except OSError as e:
+        if e.errno != errno.EEXIST or not os.path.isdir(path):
+            raise

pelican/writers.py

@@ -1,13 +1,12 @@
 # -*- coding: utf-8 -*-
-from __future__ import with_statement
+from __future__ import with_statement, unicode_literals, print_function
+import six
 
 import os
-import re
 import locale
 import logging
 
 from codecs import open
-from functools import partial
 from feedgenerator import Atom1Feed, Rss201rev2Feed
 from jinja2 import Markup
 from pelican.paginator import Paginator
@@ -41,45 +40,45 @@ class Writer(object):
             link='%s/%s' % (self.site_url, item.url),
             unique_id='tag:%s,%s:%s' % (self.site_url.replace('http://', ''),
                                         item.date.date(), item.url),
-            description=item.content,
+            description=item.get_content(self.site_url),
             categories=item.tags if hasattr(item, 'tags') else None,
-            author_name=getattr(item, 'author', 'John Doe'),
+            author_name=getattr(item, 'author', ''),
             pubdate=set_date_tzinfo(item.date,
                 self.settings.get('TIMEZONE', None)))
 
-    def write_feed(self, elements, context, filename=None, feed_type='atom'):
+    def write_feed(self, elements, context, path=None, feed_type='atom'):
         """Generate a feed with the list of articles provided
 
-        Return the feed. If no output_path or filename is specified, just
+        Return the feed. If no path or output_path is specified, just
         return the feed object.
 
         :param elements: the articles to put on the feed.
         :param context: the context to get the feed metadata.
-        :param filename: the filename to output.
+        :param path: the path to output.
         :param feed_type: the feed type to use (atom or rss)
         """
         old_locale = locale.setlocale(locale.LC_ALL)
-        locale.setlocale(locale.LC_ALL, 'C')
+        locale.setlocale(locale.LC_ALL, str('C'))
         try:
-            self.site_url = context.get('SITEURL', get_relative_path(filename))
+            self.site_url = context.get('SITEURL', get_relative_path(path))
             self.feed_domain = context.get('FEED_DOMAIN')
-            self.feed_url = '%s/%s' % (self.feed_domain, filename)
+            self.feed_url = '{}/{}'.format(self.feed_domain, path)
 
             feed = self._create_new_feed(feed_type, context)
 
             max_items = len(elements)
             if self.settings['FEED_MAX_ITEMS']:
                 max_items = min(self.settings['FEED_MAX_ITEMS'], max_items)
-            for i in xrange(max_items):
+            for i in range(max_items):
                 self._add_item_to_the_feed(feed, elements[i])
 
-            if filename:
+            if path:
-                complete_path = os.path.join(self.output_path, filename)
+                complete_path = os.path.join(self.output_path, path)
                 try:
                     os.makedirs(os.path.dirname(complete_path))
                 except Exception:
                     pass
-                fp = open(complete_path, 'w')
+                fp = open(complete_path, 'w', encoding='utf-8' if six.PY3 else None)
                 feed.write(fp, 'utf-8')
                 logger.info('writing %s' % complete_path)
 
@@ -110,34 +109,34 @@ class Writer(object):
         def _write_file(template, localcontext, output_path, name):
             """Render the template write the file."""
             old_locale = locale.setlocale(locale.LC_ALL)
-            locale.setlocale(locale.LC_ALL, 'C')
+            locale.setlocale(locale.LC_ALL, str('C'))
             try:
                 output = template.render(localcontext)
             finally:
                 locale.setlocale(locale.LC_ALL, old_locale)
-            filename = os.sep.join((output_path, name))
+            path = os.path.join(output_path, name)
             try:
-                os.makedirs(os.path.dirname(filename))
+                os.makedirs(os.path.dirname(path))
             except Exception:
                 pass
-            with open(filename, 'w', encoding='utf-8') as f:
+            with open(path, 'w', encoding='utf-8') as f:
                 f.write(output)
-            logger.info(u'writing %s' % filename)
+            logger.info('writing {}'.format(path))
 
         localcontext = context.copy()
         if relative_urls:
-            localcontext['SITEURL'] = get_relative_path(name)
+            relative_path = get_relative_path(name)
+            context['localsiteurl'] = relative_path
+            localcontext['SITEURL'] = relative_path
 
         localcontext.update(kwargs)
-        if relative_urls:
-            self.update_context_contents(name, localcontext)
 
         # check paginated
         paginated = paginated or {}
         if paginated:
             # pagination needed, init paginators
             paginators = {}
-            for key in paginated.iterkeys():
+            for key in paginated.keys():
                 object_list = paginated[key]
 
                 if self.settings.get('DEFAULT_PAGINATION'):
@@ -148,81 +147,23 @@ class Writer(object):
                     paginators[key] = Paginator(object_list, len(object_list))
 
             # generated pages, and write
-            for page_num in range(paginators.values()[0].num_pages):
+            name_root, ext = os.path.splitext(name)
+            for page_num in range(list(paginators.values())[0].num_pages):
                 paginated_localcontext = localcontext.copy()
-                paginated_name = name
+                for key in paginators.keys():
-                for key in paginators.iterkeys():
                     paginator = paginators[key]
                     page = paginator.page(page_num + 1)
                     paginated_localcontext.update(
                             {'%s_paginator' % key: paginator,
                              '%s_page' % key: page})
                 if page_num > 0:
-                    ext = '.' + paginated_name.rsplit('.')[-1]
+                    paginated_name = '%s%s%s' % (
-                    paginated_name = paginated_name.replace(ext,
+                        name_root, page_num + 1, ext)
-                        '%s%s' % (page_num + 1, ext))
+                else:
+                    paginated_name = name
 
                 _write_file(template, paginated_localcontext, self.output_path,
                     paginated_name)
         else:
             # no pagination
             _write_file(template, localcontext, self.output_path, name)
-
-    def update_context_contents(self, name, context):
-        """Recursively run the context to find elements (articles, pages, etc)
-        whose content getter needs to be modified in order to deal with
-        relative paths.
-
-        :param name: name of the file to output.
-        :param context: dict that will be passed to the templates, which need
-                        to be updated.
-        """
-        def _update_content(name, input):
-            """Change all the relatives paths of the input content to relatives
-            paths suitable fot the ouput content
-
-            :param name: path of the output.
-            :param input: input resource that will be passed to the templates.
-            """
-            content = input._content
-
-            hrefs = re.compile(r"""
-                (?P<markup><\s*[^\>]*  # match tag with src and href attr
-                    (?:href|src)\s*=\s*
-                )
-                (?P<quote>["\'])       # require value to be quoted
-                (?![#?])               # don't match fragment or query URLs
-                (?![a-z]+:)            # don't match protocol URLS
-                (?P<path>.*?)          # the url value
-                \2""", re.X)
-
-            def replacer(m):
-                relative_path = m.group('path')
-                dest_path = os.path.normpath(
-                                os.sep.join((get_relative_path(name), "static",
-                                relative_path)))
-
-                return m.group('markup') + m.group('quote') + dest_path \
-                        + m.group('quote')
-
-            return hrefs.sub(replacer, content)
-
-        if context is None:
-            return
-        if hasattr(context, 'values'):
-            context = context.values()
-
-        for item in context:
-            # run recursively on iterables
-            if hasattr(item, '__iter__'):
-                self.update_context_contents(name, item)
-
-            # if it is a content, patch it
-            elif hasattr(item, '_content'):
-                relative_path = get_relative_path(name)
-
-                paths = self.reminder.setdefault(item, [])
-                if relative_path not in paths:
-                    paths.append(relative_path)
-                    setattr(item, "_get_content",
-                        partial(_update_content, name, item))

samples/content/2012-11-30_filename-metadata.rst

@@ -0,0 +1,4 @@
+FILENAME_METADATA example
+#########################
+
+Some cool stuff!

samples/content/another_super_article.rst

@@ -14,7 +14,7 @@ Why not ?
 After all, why not ? It's pretty simple to do it, and it will allow me to write my blogposts in rst !
 YEAH !
 
-.. image:: pictures/Sushi.jpg
+.. image:: |filename|/pictures/Sushi.jpg
    :height: 450 px
    :width: 600 px
    :alt: alternate text

samples/content/cat1/markdown-article.md

@@ -2,3 +2,6 @@ Title: A markdown powered article
 Date: 2011-04-20
 
 You're mutually oblivious.
+
+[a root-relative link to unbelievable](|filename|/unbelievable.rst)
+[a file-relative link to unbelievable](|filename|../unbelievable.rst)

samples/content/pages/jinja2_template.html

@@ -0,0 +1,6 @@
+{% extends "base.html" %}
+{% block content %}
+
+Some text
+
+{% endblock %}

samples/content/pages/test_page.rst

@@ -5,7 +5,7 @@ This is a test page
 
 Just an image.
 
-.. image:: pictures/Fat_Cat.jpg
+.. image:: |filename|/pictures/Fat_Cat.jpg
    :height: 450 px
    :width: 600 px
    :alt: alternate text

samples/content/super_article.rst

@@ -16,12 +16,12 @@ This is a simple title
 
 And here comes the cool stuff_.
 
-.. image:: pictures/Sushi.jpg
+.. image:: |filename|/pictures/Sushi.jpg
    :height: 450 px
    :width: 600 px
    :alt: alternate text
 
-.. image:: pictures/Sushi_Macro.jpg
+.. image:: |filename|/pictures/Sushi_Macro.jpg
    :height: 450 px
    :width: 600 px
    :alt: alternate text

samples/content/unbelievable.rst

@@ -4,3 +4,6 @@ Unbelievable !
 :date: 2010-10-15 20:30
 
 Or completely awesome. Depends the needs.
+
+`a root-relative link to markdown-article <|filename|/cat1/markdown-article.md>`_
+`a file-relative link to markdown-article <|filename|cat1/markdown-article.md>`_

samples/pelican.conf.py

@@ -1,6 +1,8 @@
 # -*- coding: utf-8 -*-
-AUTHOR = u'Alexis Métaireau'
+from __future__ import unicode_literals
-SITENAME = u"Alexis' log"
+
+AUTHOR = 'Alexis Métaireau'
+SITENAME = "Alexis' log"
 SITEURL = 'http://blog.notmyidea.org'
 TIMEZONE = "Europe/Paris"
 
@@ -8,17 +10,18 @@ GITHUB_URL = 'http://github.com/ametaireau/'
 DISQUS_SITENAME = "blog-notmyidea"
 PDF_GENERATOR = False
 REVERSE_CATEGORY_ORDER = True
-LOCALE = ""
+LOCALE = "C"
 DEFAULT_PAGINATION = 4
+DEFAULT_DATE = (2012, 3, 2, 14, 1, 1)
 
-FEED_RSS = 'feeds/all.rss.xml'
+FEED_ALL_RSS = 'feeds/all.rss.xml'
 CATEGORY_FEED_RSS = 'feeds/%s.rss.xml'
 
 LINKS = (('Biologeek', 'http://biologeek.org'),
          ('Filyb', "http://filyb.info/"),
          ('Libert-fr', "http://www.libert-fr.com"),
          ('N1k0', "http://prendreuncafe.com/blog/"),
-         (u'Tarek Ziadé', "http://ziade.org/blog"),
+         ('Tarek Ziadé', "http://ziade.org/blog"),
          ('Zubin Mithra', "http://zubin71.wordpress.com/"),)
 
 SOCIAL = (('twitter', 'http://twitter.com/ametaireau'),
@@ -34,6 +37,9 @@ STATIC_PATHS = ["pictures", ]
 # A list of files to copy from the source to the destination
 FILES_TO_COPY = (('extra/robots.txt', 'robots.txt'),)
 
+# custom page generated with a jinja2 template
+TEMPLATE_PAGES = {'pages/jinja2_template.html': 'jinja2_template.html'}
+
 # foobar will not be used, because it's not in caps. All configuration keys
 # have to be in caps
 foobar = "barbaz"

setup.py

@@ -1,10 +1,11 @@
 #!/usr/bin/env python
 from setuptools import setup
 
-requires = ['feedgenerator', 'jinja2 >= 2.4', 'pygments', 'docutils', 'pytz', 'blinker']
+requires = ['feedgenerator>=1.5', 'jinja2 >= 2.6', 'pygments', 'docutils', 'pytz',
+            'blinker', 'unidecode', 'six']
 
 try:
-    import argparse
+    import argparse  # NOQA
 except ImportError:
     requires.append('argparse')
 
@@ -17,25 +18,35 @@ entry_points = {
    ]
 }
 
+
+README = open('README.rst').read()
+CHANGELOG = open('docs/changelog.rst').read()
+
+
 setup(
-    name = "pelican",
+    name="pelican",
-    version = "3.0",
+    version="3.2",
-    url = 'http://pelican.notmyidea.org/',
+    url='http://getpelican.com/',
-    author = 'Alexis Metaireau',
+    author='Alexis Metaireau',
-    author_email = 'alexis@notmyidea.org',
+    author_email='authors@getpelican.com',
-    description = "A tool to generate a static blog from reStructuredText or Markdown input files.",
+    description="A tool to generate a static blog from reStructuredText or "
-    long_description=open('README.rst').read(),
+                "Markdown input files.",
-    packages = ['pelican', 'pelican.tools', 'pelican.plugins'],
+    long_description=README + '\n' + CHANGELOG,
-    include_package_data = True,
+    packages=['pelican', 'pelican.tools', 'pelican.plugins'],
-    install_requires = requires,
+    include_package_data=True,
-    entry_points = entry_points,
+    install_requires=requires,
-    classifiers = ['Development Status :: 5 - Production/Stable',
+    entry_points=entry_points,
-                   'Environment :: Console',
+    classifiers=[
-                   'License :: OSI Approved :: GNU Affero General Public License v3',
+        'Development Status :: 5 - Production/Stable',
-                   'Operating System :: OS Independent',
+         'Environment :: Console',
-                   'Programming Language :: Python :: 2.6',
+         'License :: OSI Approved :: GNU Affero General Public License v3',
-                   'Programming Language :: Python :: 2.7',
+         'Operating System :: OS Independent',
-                   'Topic :: Internet :: WWW/HTTP',
+         'Programming Language :: Python :: 2.7',
-                   'Topic :: Software Development :: Libraries :: Python Modules',
+         'Programming Language :: Python :: 3',
-                   ],
+         'Programming Language :: Python :: 3.2',
+         'Programming Language :: Python :: 3.3',
+         'Topic :: Internet :: WWW/HTTP',
+         'Topic :: Software Development :: Libraries :: Python Modules',
+    ],
+    test_suite='tests',
 )

tests/TestPages/hidden_page_with_template.rst

@@ -0,0 +1,11 @@
+This is a test hidden page with a custom template
+#################################################
+
+:status: hidden
+:template: custom
+
+The quick brown fox jumped over the lazy dog's back.
+
+This page is hidden
+
+This page has a custom template to be called when rendered

tests/TestPages/page_with_template.rst

@@ -0,0 +1,8 @@
+This is a test page with a preset template
+##########################################
+
+:template: custom
+
+The quick brown fox jumped over the lazy dog's back.
+
+This article has a custom template to be called when rendered

tests/content/2012-11-29_rst_w_filename_meta#foo-bar.rst

@@ -0,0 +1,6 @@
+
+Rst with filename metadata
+##########################
+
+:category: yeah
+:author: Alexis Métaireau

tests/content/2012-11-30_md_w_filename_meta#foo-bar.md

@@ -0,0 +1,6 @@
+category: yeah
+author: Alexis Métaireau
+
+Markdown with filename metadata
+===============================
+

tests/content/TestCategory/article_with_category.rst

@@ -2,5 +2,6 @@ This is an article with category !
 ##################################
 
 :category: yeah
+:date: 1970-01-01
 
 This article should be in 'yeah' category.

tests/content/article.rst

@@ -2,3 +2,5 @@ Article title
 #############
 
 This is some content. With some stuff to "typogrify".
+
+Now with added support for :abbr:`TLA (three letter acronym)`.

tests/content/article_with_asc_extension.asc

@@ -0,0 +1,12 @@
+Test AsciiDoc File Header
+=========================
+:Author: Author O. Article
+:Email: <author@nowhere.com>
+:Date: 2011-09-15 09:05
+:Category: Blog
+:Tags: Linux, Python, Pelican
+
+Used for pelican test
+---------------------
+
+The quick brown fox jumped over the lazy dog's back.

tests/content/article_with_asc_options.asc

@@ -0,0 +1,9 @@
+Test AsciiDoc File Header
+=========================
+
+Used for pelican test
+---------------------
+
+version {revision}
+
+The quick brown fox jumped over the lazy dog's back.

tests/content/article_with_markdown_and_summary_metadata_multi.md

@@ -0,0 +1,7 @@
+Title: Article with markdown and summary metadata multi
+Date: 2012-10-31
+Summary:
+    A multi-line summary should be supported
+    as well as **inline markup**.
+
+This is some content.

tests/content/article_with_markdown_and_summary_metadata_single.md

@@ -0,0 +1,5 @@
+Title: Article with markdown and summary metadata single
+Date: 2012-10-30
+Summary: A single-line summary should be supported as well as **inline markup**.
+
+This is some content.

tests/content/article_with_markdown_extension.markdown

@@ -0,0 +1,10 @@
+title: Test markdown File
+category: test
+
+Test Markdown File Header
+=========================
+
+Used for pelican test
+---------------------
+
+This is another markdown test file.  Uses the markdown extension.

tests/content/article_with_markdown_markup_extensions.md

@@ -0,0 +1,8 @@
+Title: Test Markdown extensions
+
+[TOC]
+
+## Level1
+
+### Level2
+