merge upstream

MANIFEST.in

@@ -1,4 +1,4 @@
 include *.rst
 global-include *.py
-recursive-include pelican *.html *.css *png
+recursive-include pelican *.html *.css *png *.in
 include LICENSE

dev_requirements.txt

@@ -6,3 +6,5 @@ unittest2
 pytz
 mock
 Markdown
+blinker
+BeautifulSoup

docs/getting_started.rst

@@ -4,17 +4,48 @@ Getting started
 Installing
 ==========
 
-You're ready? Let's go! You can install Pelican via several different methods. The simplest is via `pip <http://pip.openplans.org/>`_::
+You're ready? Let's go! You can install Pelican via several different methods.
+The simplest is via `pip <http://www.pip-installer.org/>`_::
 
     $ pip install pelican
 
-If you have the project source, you can install Pelican using the distutils 
+If you don't have pip installed, an alternative method is easy_install::
-method. I recommend doing so in a virtualenv::
 
-    $ virtualenv pelican_venv
+    $ easy_install pelican
-    $ source bin/activate
+
+While the above is the simplest method, the recommended approach is to create
+a virtual environment for Pelican via `virtualenv <http://www.virtualenv.org/>`_
+and `virtualenvwrapper <http://www.doughellmann.com/projects/virtualenvwrapper/>`_
+before installing Pelican::
+
+    $ pip install virtualenvwrapper
+    $ mkvirtualenv pelican
+
+Once the virtual environment has been created and activated, Pelican can be
+be installed via pip or easy_install as noted above. Alternatively, if you
+have the project source, you can install Pelican using the distutils 
+method::
+
+    $ cd path-to-Pelican-source
     $ python setup.py install
 
+If you have Git installed and prefer to install the latest bleeding-edge
+version of Pelican rather than a stable release, use the following command::
+
+    $ pip install -e git://github.com/ametaireau/pelican#egg=pelican
+
+Upgrading
+---------
+
+If you installed a stable Pelican release via pip or easy_install and wish to
+upgrade to the latest stable release, you can do so by adding `--upgrade` to
+the relevant command. For pip, that would be::
+
+    $ pip install --upgrade pelican
+
+If you installed Pelican via distutils or the bleeding-edge method, simply
+perform the same step to install the most recent version.
+
 Dependencies
 ------------
 
@@ -54,10 +85,9 @@ following syntax (give your file the `.rst` extension)::
 
 
 You can also use Markdown syntax (with a file ending in `.md`).
-Markdown generation will not work until you explicitly install the `markdown`
+Markdown generation will not work until you explicitly install the `Markdown`
-distribution. You can do so on a normal system using `pip install markdown`
+package, which can be done via `pip install Markdown`. Metadata syntax for
-
+Markdown posts should follow this pattern::
-::
 
     Date: 2010-12-03
     Title: My super title
@@ -94,7 +124,7 @@ Kickstart a blog
 ----------------
 
 You also can use the `pelican-quickstart` script to start a new blog in
-seconds, by just answering few questions. Just run `pelican-quickstart` and
+seconds by just answering a few questions. Just run `pelican-quickstart` and
 you're done! (Added in Pelican 3.0)
 
 Pages
@@ -175,12 +205,12 @@ For Markdown, format your code blocks thusly::
 The specified identifier should be one that appears on the 
 `list of available lexers <http://pygments.org/docs/lexers/>`_.
 
-Autoreload
+Auto-reload
-----------
+-----------
 
 It's possible to tell Pelican to watch for your modifications, instead of
-manually launching it every time you want to see your changes. To enable this,
+manually re-running it every time you want to see your changes. To enable this,
-run the `pelican` command with the `-r` or `--autoreload` options.
+run the `pelican` command with the `-r` or `--autoreload` option.
 
 Publishing drafts
 -----------------
@@ -203,3 +233,6 @@ You can either use your browser to open the files on your disk::
 Or run a simple web server using Python::
 
     cd output && python -m SimpleHTTPServer
+
+(Tip: If using the latter method in conjunction with the auto-reload feature,
+ensure that `DELETE_OUTPUT_DIRECTORY` is set to `False` in your settings file.)

docs/index.rst

@@ -61,6 +61,7 @@ A French version of the documentation is available at :doc:`fr/index`.
    getting_started
    settings
    themes
+   plugins
    internals
    pelican-themes
    importer

docs/plugins.rst

@@ -0,0 +1,108 @@
+.. _plugins:
+
+Plugins
+#######
+
+Since version 3.0, pelican manages plugins. Plugins are a way to add features
+to pelican without having to directly hack pelican code.
+
+Pelican is shipped with a set of core plugins, but you can easily implement
+your own (and this page describes how).
+
+How to use plugins?
+====================
+
+To load plugins, you have to specify them in your settings file. You have two
+ways to do so.
+Either by specifying strings with the path to the callables::
+
+    PLUGINS = ['pelican.plugins.gravatar',] 
+
+Or by importing them and adding them to the list::
+
+    from pelican.plugins import gravatar
+    PLUGINS = [gravatar, ]
+
+If your plugins are not in an importable path, you can specify a `PLUGIN_PATH`
+in the settings::
+
+    PLUGIN_PATH = "plugins"
+    PLUGINS = ["list", "of", "plugins"]
+
+How to create plugins?
+======================
+
+Plugins are based on the concept of signals. Pelican sends signals and plugins
+subscribe to those signals. The list of signals are defined in a following
+section.
+
+The only rule to follow for plugins is to define a `register` callable, in
+which you map the signals to your plugin logic. Let's take a simple exemple::
+
+    from pelican import signals
+
+    def test(sender):
+        print "%s initialized !!" % sender
+
+    def register():
+        signals.initialized.connect(test)
+
+
+List of signals
+===============
+
+Here is the list of currently implemented signals:
+
+=========================   ============================   =========================================
+Signal                      Arguments                      Description
+=========================   ============================   =========================================
+initialized                 pelican object
+article_generate_context    article_generator, metadata
+article_generator_init      article_generator              invoked in the ArticlesGenerator.__init__
+=========================   ============================   =========================================
+
+The list is currently small, don't hesitate to add signals and make a pull
+request if you need them!
+
+List of plugins
+===============
+
+Not all the list are described here, but a few of them have been extracted from
+pelican core and provided in pelican.plugins. They are described here:
+
+Tag cloud
+---------
+
+Translation
+-----------
+
+Github Activity
+---------------
+
+This plugin makes use of the ``feedparser`` library that you'll need to
+install.
+
+Set the GITHUB_ACTIVITY_FEED parameter to your github activity feed.
+For example, my setting would look like::
+
+     GITHUB_ACTIVITY_FEED = 'https://github.com/kpanic.atom'
+
+On the templates side, you just have to iterate over the ``github_activity``
+variable, as in the example::
+
+     {% if GITHUB_ACTIVITY_FEED %}
+        <div class="social">
+                <h2>Github Activity</h2>
+                <ul>
+
+                {% for entry in github_activity %}
+                    <li><b>{{ entry[0] }}</b><br /> {{ entry[1] }}</li>
+                {% endfor %}
+                </ul>
+        </div><!-- /.github_activity -->
+     {% endif %}
+
+
+
+``github_activity`` is a list of lists. The first element is the title
+and the second element is the raw html from github.

docs/settings.rst

@@ -61,6 +61,7 @@ Setting name (default value)                                            What doe
                                                                         `rst2pdf`.
 `RELATIVE_URLS` (``True``)                                              Defines whether Pelican should use relative URLs or
                                                                         not.
+`PLUGINS` (``[]``)                                                      The list of plugins to load. See :ref:`plugins`.
 `SITENAME` (``'A Pelican Blog'``)                                       Your site name
 `SITEURL`                                                               Base URL of your website. Not defined by default,
                                                                         which means the base URL is assumed to be "/" with a
@@ -369,6 +370,7 @@ Setting name (default value)                        What does it do?
                                                     value is `static`, but if your theme has
                                                     other static paths, you can put them here.
 `CSS_FILE` (``'main.css'``)                         Specify the CSS file you want to load.
+`WEBASSETS` (``False``)                             Asset management with `webassets` (see below)
 ================================================    =====================================================
 
 By default, two themes are available. You can specify them using the `-t` option:
@@ -418,7 +420,58 @@ adding the following to your configuration::
 
     CSS_FILE = "wide.css"
 
-.. _pelican-themes: :doc:`pelican-themes`
+Asset management
+----------------
+
+The `WEBASSETS` setting allows to use the `webassets`_ module to manage assets
+(css, js). The module must first be installed::
+
+    pip install webassets
+
+`webassets` allows to concatenate your assets and to use almost all of the
+hype tools of the moment (see the `documentation`_):
+
+* css minifier (`cssmin`, `yuicompressor`, ...)
+* css compiler (`less`, `sass`, ...)
+* js minifier (`uglifyjs`, `yuicompressor`, `closure`, ...)
+
+Others filters include gzip compression, integration of images in css with
+`datauri` and more. Webassets also append a version identifier to your asset
+url to convince browsers to download new versions of your assets when you use
+far future expires headers.
+
+When using it with Pelican, `webassets` is configured to process assets in the
+``OUTPUT_PATH/theme`` directory. You can use it in your templates with a
+template tag, for example:
+
+.. code-block:: jinja
+
+    {% assets filters="cssmin", output="css/style.min.css", "css/inuit.css", "css/pygment-monokai.css", "css/main.css" %}
+        <link rel="stylesheet" href="{{ ASSET_URL }}">
+    {% endassets %}
+
+will produce a minified css file with the version identifier:
+
+.. code-block:: html
+
+    <link href="http://{SITEURL}/theme/css/style.min.css?b3a7c807" rel="stylesheet">
+
+Another example for javascript:
+
+.. code-block:: jinja
+
+    {% assets filters="uglifyjs,gzip", output="js/packed.js", "js/jquery.js", "js/base.js", "js/widgets.js" %}
+        <script src="{{ ASSETS_URL }}"></script>
+    {% endassets %}
+
+will produce a minified and gzipped js file:
+
+.. code-block:: html
+
+    <script src="http://{SITEURL}/theme/js/packed.js?00703b9d"></script>
+
+.. _webassets: https://github.com/miracle2k/webassets
+.. _documentation: http://webassets.readthedocs.org/en/latest/builtin_filters.html
 
 Example settings
 ================

docs/tips.rst

@@ -10,6 +10,26 @@ GitHub comes with an interesting "pages" feature: you can upload things there
 and it will be available directly from their servers. As Pelican is a static
 file generator, we can take advantage of this.
 
+User Pages
+----------
+Github allows you to create user pages in the form of ``username.github.com``. 
+Whatever is created in master branch will be published. For this purposes just 
+the output generated by pelican needs to pushed at github.
+
+So given a repository containing your articles, just run pelican over the posts 
+and deploy the master branch at github::
+
+    $ pelican -s pelican.conf.py ./path/to/posts -o /path/to/output
+
+Now add all the files in the output directory generated by pelican::
+
+    $ git add /path/to/output/*
+    $ git commit -am "Your Message"
+    $ git push origin master
+
+Project Pages
+-------------
+For creating Project pages, a branch called ``gh-pages`` is used for publishing.
 The excellent `ghp-import <https://github.com/davisp/ghp-import>`_ makes this
 really easy. You will have to install it::
 
@@ -31,3 +51,4 @@ Put the following into `.git/hooks/post-commit`::
 
     pelican -s pelican.conf.py . && ghp-import output && git push origin
     gh-pages
+

pelican/__init__.py

@@ -5,11 +5,13 @@ import time
 import logging
 import argparse
 
+from pelican import signals
+
 from pelican.generators import (ArticlesGenerator, PagesGenerator,
         StaticGenerator, PdfGenerator, LessCSSGenerator)
 from pelican.log import init
 from pelican.settings import read_settings, _DEFAULT_CONFIG
-from pelican.utils import clean_output_dir, files_changed
+from pelican.utils import clean_output_dir, files_changed, file_changed
 from pelican.writers import Writer
 
 __major__ = 3
@@ -22,7 +24,7 @@ logger = logging.getLogger(__name__)
 
 class Pelican(object):
     def __init__(self, settings=None, path=None, theme=None, output_path=None,
-            markup=None, delete_outputdir=False):
+            markup=None, delete_outputdir=False, plugin_path=None):
         """Read the settings, and performs some checks on the environment
         before doing anything else.
         """
@@ -58,6 +60,20 @@ class Pelican(object):
             else:
                 raise Exception("Impossible to find the theme %s" % theme)
 
+        self.init_plugins()
+        signals.initialized.send(self)
+
+    def init_plugins(self):
+        self.plugins = self.settings['PLUGINS']
+        for plugin in self.plugins:
+            # if it's a string, then import it
+            if isinstance(plugin, basestring):
+                log.debug("Loading plugin `{0}' ...".format(plugin))
+                plugin = __import__(plugin, globals(), locals(), 'module')
+
+            log.debug("Registering plugin `{0}' ...".format(plugin.__name__))
+            plugin.register()
+
     def _handle_deprecation(self):
 
         if self.settings.get('CLEAN_URLS', False):
@@ -126,15 +142,20 @@ class Pelican(object):
 
         writer = self.get_writer()
 
+        # pass the assets environment to the generators
+        if self.settings['WEBASSETS']:
+            generators[1].env.assets_environment = generators[0].assets_env
+            generators[2].env.assets_environment = generators[0].assets_env
+
         for p in generators:
             if hasattr(p, 'generate_output'):
                 p.generate_output(writer)
 
     def get_generator_classes(self):
-        generators = [ArticlesGenerator, PagesGenerator, StaticGenerator]
+        generators = [StaticGenerator, ArticlesGenerator, PagesGenerator]
         if self.settings['PDF_GENERATOR']:
             generators.append(PdfGenerator)
-        if self.settings['LESS_GENERATOR']: # can be True or PATH to lessc
+        if self.settings['LESS_GENERATOR']:  # can be True or PATH to lessc
             generators.append(LessCSSGenerator)
         return generators
 
@@ -192,11 +213,7 @@ def parse_arguments():
     return parser.parse_args()
 
 
-def main():
+def get_instance(args):
-    args = parse_arguments()
-    init(args.verbosity)
-    # Split the markup languages only if some have been given. Otherwise,
-    # populate the variable with None.
     markup = [a.strip().lower() for a in args.markup.split(',')]\
               if args.markup else None
 
@@ -208,9 +225,18 @@ def main():
         module = __import__(module)
         cls = getattr(module, cls_name)
 
+    return cls(settings, args.path, args.theme, args.output, markup,
+               args.delete_outputdir)
+
+
+def main():
+    args = parse_arguments()
+    init(args.verbosity)
+    # Split the markup languages only if some have been given. Otherwise,
+    # populate the variable with None.
+    pelican = get_instance(args)
+
     try:
-        pelican = cls(settings, args.path, args.theme, args.output, markup,
-                args.delete_outputdir)
         if args.autoreload:
             while True:
                 try:
@@ -222,6 +248,14 @@ def main():
                     if files_changed(pelican.path, pelican.markup) or \
                             files_changed(pelican.theme, ['']):
                         pelican.run()
+
+                    # reload also if settings.py changed
+                    if file_changed(args.settings):
+                        logger.info('%s changed, re-generating' %
+                                    args.settings)
+                        pelican = get_instance(args)
+                        pelican.run()
+
                     time.sleep(.5)  # sleep to avoid cpu load
                 except KeyboardInterrupt:
                     break

pelican/generators.py

@@ -17,6 +17,7 @@ from jinja2.exceptions import TemplateNotFound
 from pelican.contents import Article, Page, Category, is_valid_content
 from pelican.readers import read_file
 from pelican.utils import copy, process_translations, open
+from pelican import signals
 
 
 logger = logging.getLogger(__name__)
@@ -42,7 +43,7 @@ class Generator(object):
 
         simple_loader = FileSystemLoader(os.path.join(theme_path,
                                          "themes", "simple", "templates"))
-        self._env = Environment(
+        self.env = Environment(
             loader=ChoiceLoader([
                 FileSystemLoader(self._templates_path),
                 simple_loader,  # implicit inheritance
@@ -51,11 +52,11 @@ class Generator(object):
             extensions=self.settings.get('JINJA_EXTENSIONS', []),
         )
 
-        logger.debug('template list: {0}'.format(self._env.list_templates()))
+        logger.debug('template list: {0}'.format(self.env.list_templates()))
 
         # get custom Jinja filters from user settings
         custom_filters = self.settings.get('JINJA_FILTERS', {})
-        self._env.filters.update(custom_filters)
+        self.env.filters.update(custom_filters)
 
     def get_template(self, name):
         """Return the template by name.
@@ -64,7 +65,7 @@ class Generator(object):
         """
         if name not in self._templates:
             try:
-                self._templates[name] = self._env.get_template(name + '.html')
+                self._templates[name] = self.env.get_template(name + '.html')
             except TemplateNotFound:
                 raise Exception('[templates] unable to load %s.html from %s' \
                         % (name, self._templates_path))
@@ -118,6 +119,7 @@ class ArticlesGenerator(Generator):
         self.authors = defaultdict(list)
         super(ArticlesGenerator, self).__init__(*args, **kwargs)
         self.drafts = []
+        signals.article_generator_init.send(self)
 
     def generate_feeds(self, writer):
         """Generate the feeds from the current context, and output files."""
@@ -245,7 +247,9 @@ class ArticlesGenerator(Generator):
     def generate_context(self):
         """change the context"""
 
-        article_path = os.path.join(self.path, self.settings['ARTICLE_DIR'])
+        article_path = os.path.normpath(  # we have to remove trailing slashes
+            os.path.join(self.path, self.settings['ARTICLE_DIR'])
+        )
         all_articles = []
         for f in self.get_files(
                 article_path,
@@ -259,8 +263,8 @@ class ArticlesGenerator(Generator):
             # if no category is set, use the name of the path as a category
             if 'category' not in metadata:
 
-                if os.path.dirname(f) == article_path:
+                if os.path.dirname(f) == article_path:  # if the article is not in a subdirectory
-                    category = self.settings['DEFAULT_CATEGORY']
+                    category = self.settings['DEFAULT_CATEGORY'] 
                 else:
                     category = os.path.basename(os.path.dirname(f))\
                                 .decode('utf-8')
@@ -272,6 +276,7 @@ class ArticlesGenerator(Generator):
                     metadata['date'] = datetime.datetime.fromtimestamp(
                                         os.stat(f).st_ctime)
 
+            signals.article_generate_context.send(self, metadata=metadata)
             article = Article(content, metadata, settings=self.settings,
                               filename=f)
             if not is_valid_content(article, f):
@@ -284,6 +289,10 @@ class ArticlesGenerator(Generator):
                 all_articles.append(article)
             elif article.status == "draft":
                 self.drafts.append(article)
+            else:
+                logger.warning(u"Unknown status %s for file %s, skipping it." %
+                               (repr(unicode.encode(article.status, 'utf-8')),
+                                repr(f)))
 
         self.articles, self.translations = process_translations(all_articles)
 
@@ -389,7 +398,23 @@ class StaticGenerator(Generator):
             copy(path, source, os.path.join(output_path, destination),
                  final_path, overwrite=True)
 
+    def generate_context(self):
+
+        if self.settings['WEBASSETS']:
+            from webassets import Environment as AssetsEnvironment
+
+            # Define the assets environment that will be passed to the
+            # generators. The StaticGenerator must then be run first to have
+            # the assets in the output_path before generating the templates.
+            assets_url = self.settings['SITEURL'] + '/theme/'
+            assets_src = os.path.join(self.output_path, 'theme')
+            self.assets_env = AssetsEnvironment(assets_src, assets_url)
+
+            if logging.getLevelName(logger.getEffectiveLevel()) == "DEBUG":
+                self.assets_env.debug = True
+
     def generate_output(self, writer):
+
         self._copy_paths(self.settings['STATIC_PATHS'], self.path,
                          'static', self.output_path)
         self._copy_paths(self.settings['THEME_STATIC_PATHS'], self.theme,

pelican/plugins/github_activity.py

@@ -0,0 +1,85 @@
+# -*- coding: utf-8 -*-
+"""
+    Copyright (c) Marco Milanesi <kpanic@gnufunk.org>
+
+    A plugin to list your Github Activity
+    To enable it set in your pelican config file the GITHUB_ACTIVITY_FEED
+    parameter pointing to your github activity feed.
+
+    for example my personal activity feed is:
+
+        https://github.com/kpanic.atom
+
+    in your template just write a for in jinja2 syntax against the
+    github_activity variable.
+
+    i.e.
+
+        <div class="social">
+                <h2>Github Activity</h2>
+                <ul>
+
+                {% for entry in github_activity %}
+                    <li><b>{{ entry[0] }}</b><br /> {{ entry[1] }}</li>
+                {% endfor %}
+                </ul>
+        </div><!-- /.github_activity -->
+
+    github_activity is a list containing a list. The first element is the title
+    and the second element is the raw html from github
+"""
+
+from pelican import signals
+
+
+class GitHubActivity():
+    """
+        A class created to fetch github activity with feedparser
+    """
+    def __init__(self, generator):
+        try:
+            import feedparser
+            self.activities = feedparser.parse(
+                generator.settings['GITHUB_ACTIVITY_FEED'])
+        except ImportError:
+            raise Exception("Unable to find feedparser")
+
+    def fetch(self):
+        """
+            returns a list of html snippets fetched from github actitivy feed
+        """
+
+        entries = []
+        for activity in self.activities['entries']:
+            entries.append(
+                    [element for element in [activity['title'],
+                        activity['content'][0]['value']]])
+
+        return entries
+
+
+def fetch_github_activity(gen, metadata):
+    """
+        registered handler for the github activity plugin
+        it puts in generator.context the html needed to be displayed on a
+        template
+    """
+
+    if 'GITHUB_ACTIVITY_FEED' in gen.settings.keys():
+        gen.context['github_activity'] = gen.plugin_instance.fetch()
+
+
+def feed_parser_initialization(generator):
+    """
+        Initialization of feed parser
+    """
+
+    generator.plugin_instance = GitHubActivity(generator)
+
+
+def register():
+    """
+        Plugin registration
+    """
+    signals.article_generator_init.connect(feed_parser_initialization)
+    signals.article_generate_context.connect(fetch_github_activity)

pelican/plugins/global_license.py

@@ -0,0 +1,23 @@
+from pelican import signals
+
+"""
+License plugin for Pelican
+==========================
+
+Simply add license variable in article's context, which contain 
+the license text.
+
+Settings:
+---------
+
+Add LICENSE to your settings file to define default license.
+
+"""
+
+def add_license(generator, metadata):
+    if 'license' not in metadata.keys()\
+        and 'LICENSE' in generator.settings.keys():
+            metadata['license'] = generator.settings['LICENSE']
+
+def register():
+    signals.article_generate_context.connect(add_license)

pelican/plugins/gravatar.py

@@ -0,0 +1,40 @@
+import hashlib
+
+from pelican import signals
+"""
+Gravatar plugin for Pelican
+===========================
+
+Simply add author_gravatar variable in article's context, which contains
+the gravatar url.
+
+Settings:
+---------
+
+Add AUTHOR_EMAIL to your settings file to define default author email.
+
+Article metadata:
+------------------
+
+:email:  article's author email
+
+If one of them are defined, the author_gravatar variable is added to
+article's context.
+"""
+
+def add_gravatar(generator, metadata):
+
+    #first check email
+    if 'email' not in metadata.keys()\
+        and 'AUTHOR_EMAIL' in generator.settings.keys():
+            metadata['email'] = generator.settings['AUTHOR_EMAIL']
+
+    #then add gravatar url
+    if 'email' in metadata.keys():
+        gravatar_url = "http://www.gravatar.com/avatar/" + \
+                        hashlib.md5(metadata['email'].lower()).hexdigest()
+        metadata["author_gravatar"] = gravatar_url
+
+
+def register():
+    signals.article_generate_context.connect(add_gravatar)

pelican/plugins/html_rst_directive.py

@@ -0,0 +1,63 @@
+from docutils import nodes
+from docutils.parsers.rst import directives, Directive
+from pelican import log
+
+"""
+HTML tags for reStructuredText
+==============================
+
+Directives
+----------
+
+.. html::
+
+    (HTML code)
+
+
+Example
+-------
+
+A search engine:
+
+.. html::
+   <form action="http://seeks.fr/search" method="GET">
+     <input type="text" value="Pelican v2" title="Search" maxlength="2048" name="q" autocomplete="on" />
+     <input type="hidden" name="lang" value="en" />
+     <input type="submit" value="Seeks !" id="search_button" />
+   </form>
+
+
+A contact form:
+
+.. html::
+
+    <form method="GET" action="mailto:some email">
+      <p>
+        <input type="text" placeholder="Subject" name="subject">
+        <br />
+        <textarea name="body" placeholder="Message">
+        </textarea>
+        <br />
+        <input type="reset"><input type="submit">
+      </p>
+    </form>
+
+"""
+
+
+class RawHtml(Directive):
+    required_arguments = 0
+    optional_arguments = 0
+    final_argument_whitespace = True
+    has_content = True
+
+    def run(self):
+        html = u' '.join(self.content)
+        node = nodes.raw('', html, format='html')
+        return [node]
+
+
+
+def register():
+    directives.register_directive('html', RawHtml)
+

pelican/plugins/initialized.py

@@ -0,0 +1,7 @@
+from pelican import signals
+
+def test(sender):
+    print "%s initialized !!" % sender
+
+def register():
+    signals.initialized.connect(test)

pelican/settings.py

@@ -68,7 +68,9 @@ _DEFAULT_CONFIG = {'PATH': '.',
                    'ARTICLE_PERMALINK_STRUCTURE': '',
                    'TYPOGRIFY': False,
                    'LESS_GENERATOR': False,
-                   'SUMARY_MAX_LENGTH': 50,
+                   'SUMMARY_MAX_LENGTH': 50,
+                   'WEBASSETS': False,
+                   'PLUGINS': [],
                    }
 
 
@@ -151,4 +153,12 @@ def configure_settings(settings, default_settings=None, filename=None):
                  "http://docs.notmyidea.org/alexis/pelican/settings.html#timezone "
                  "for more information")
 
+    if 'WEBASSETS' in settings and settings['WEBASSETS'] is not False:
+        try:
+            from webassets.ext.jinja2 import AssetsExtension
+            settings['JINJA_EXTENSIONS'].append(AssetsExtension)
+        except ImportError:
+            logger.warn("You must install the webassets module to use WEBASSETS.")
+            settings['WEBASSETS'] = False
+
     return settings

pelican/signals.py

@@ -0,0 +1,5 @@
+from blinker import signal
+
+initialized = signal('pelican_initialized')
+article_generate_context = signal('article_generate_context')
+article_generator_init = signal('article_generator_init')

pelican/tools/pelican_import.py

@@ -25,8 +25,14 @@ def wp2fields(xml):
     items = soup.rss.channel.findAll('item')
 
     for item in items:
+
         if item.fetch('wp:status')[0].contents[0] == "publish":
-            title = item.title.contents[0]
+
+            try:
+                title = item.title.contents[0]
+            except IndexError:
+                continue
+
             content = item.fetch('content:encoded')[0].contents[0]
             filename = item.fetch('wp:post_name')[0].contents[0]
 
@@ -197,7 +203,7 @@ def build_markdown_header(title, date, author, categories, tags):
     header += '\n'
     return header
 
-def fields2pelican(fields, out_markup, output_path, dircat=False):
+def fields2pelican(fields, out_markup, output_path, dircat=False, strip_raw=False):
     for title, content, filename, date, author, categories, tags, in_markup in fields:
         if (in_markup == "markdown") or (out_markup == "markdown") :
             ext = '.md'
@@ -230,22 +236,26 @@ def fields2pelican(fields, out_markup, output_path, dircat=False):
                 paragraphs = [u'<p>{}</p>'.format(p) for p in paragraphs]
                 new_content = ''.join(paragraphs)
 
-                fp.write(content)
+                fp.write(new_content)
 
-            cmd = 'pandoc --normalize --reference-links --from=html --to={0} -o "{1}" "{2}"'.format(
+
-                out_markup, out_filename, html_filename)
+            parse_raw = '--parse-raw' if not strip_raw else ''
+            cmd = ('pandoc --normalize --reference-links {0} --from=html'
+                   ' --to={1} -o "{2}" "{3}"').format(
+                    parse_raw, out_markup, out_filename, html_filename)
 
             try:
                 rc = subprocess.call(cmd, shell=True)
                 if rc < 0:
-                    print("Child was terminated by signal %d" % -rc)
+                    error = "Child was terminated by signal %d" % -rc
-                    exit()
+                    exit(error)
+
                 elif rc > 0:
-                    print("Please, check your Pandoc installation.")
+                    error = "Please, check your Pandoc installation."
-                    exit()
+                    exit(error)
             except OSError, e:
-                print("Pandoc execution failed: %s" % e)
+                error = "Pandoc execution failed: %s" % e
-                exit()
+                exit(error)
 
             os.remove(html_filename)
 
@@ -279,6 +289,10 @@ def main():
         help='Output markup format (supports rst & markdown)')
     parser.add_argument('--dir-cat', action='store_true', dest='dircat',
         help='Put files in directories with categories name')
+    parser.add_argument('--strip-raw', action='store_true', dest='strip_raw',
+        help="Strip raw HTML code that can't be converted to "
+             "markup such as flash embeds or iframes (wordpress import only)")
+
     args = parser.parse_args()
 
     input_type = None
@@ -289,15 +303,15 @@ def main():
     elif args.feed:
         input_type = 'feed'
     else:
-        print("You must provide either --wpfile, --dotclear or --feed options")
+        error = "You must provide either --wpfile, --dotclear or --feed options"
-        exit()
+        exit(error)
 
     if not os.path.exists(args.output):
         try:
             os.mkdir(args.output)
         except OSError:
-            print("Unable to create the output folder: " + args.output)
+            error = "Unable to create the output folder: " + args.output
-            exit()
+            exit(error)
 
     if input_type == 'wordpress':
         fields = wp2fields(args.input)
@@ -306,4 +320,6 @@ def main():
     elif input_type == 'feed':
         fields = feed2fields(args.input)
 
-    fields2pelican(fields, args.markup, args.output, dircat=args.dircat or False)
+    fields2pelican(fields, args.markup, args.output,
+                   dircat=args.dircat or False,
+                   strip_raw=args.strip_raw or False)

pelican/utils.py

@@ -5,6 +5,7 @@ import re
 import pytz
 import shutil
 import logging
+from collections import defaultdict
 
 from codecs import open as _open
 from datetime import datetime
@@ -222,9 +223,9 @@ def files_changed(path, extensions):
         """Return the last time files have been modified"""
         for root, dirs, files in os.walk(path):
             dirs[:] = [x for x in dirs if x[0] != '.']
-            for file in files:
+            for f in files:
-                if any(file.endswith(ext) for ext in extensions):
+                if any(f.endswith(ext) for ext in extensions):
-                    yield os.stat(os.path.join(root, file)).st_mtime
+                    yield os.stat(os.path.join(root, f)).st_mtime
 
     global LAST_MTIME
     mtime = max(file_times(path))
@@ -234,6 +235,21 @@ def files_changed(path, extensions):
     return False
 
 
+FILENAMES_MTIMES = defaultdict(int)
+
+
+def file_changed(filename):
+    mtime = os.stat(filename).st_mtime
+    if FILENAMES_MTIMES[filename] == 0:
+        FILENAMES_MTIMES[filename] = mtime
+        return False
+    else:
+        if mtime > FILENAMES_MTIMES[filename]:
+            FILENAMES_MTIMES[filename] = mtime
+            return True
+        return False
+
+
 def set_date_tzinfo(d, tz_name=None):
     """ Date without tzinfo shoudbe utc.
     This function set the right tz to date that aren't utc and don't have

setup.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 from setuptools import setup
 
-requires = ['feedgenerator', 'jinja2', 'pygments', 'docutils', 'pytz']
+requires = ['feedgenerator', 'jinja2', 'pygments', 'docutils', 'pytz', 'blinker']
 
 try:
     import argparse
@@ -25,7 +25,7 @@ setup(
     author_email = 'alexis@notmyidea.org',
     description = "A tool to generate a static blog from reStructuredText or Markdown input files.",
     long_description=open('README.rst').read(),
-    packages = ['pelican', 'pelican.tools'],
+    packages = ['pelican', 'pelican.tools', 'pelican.plugins'],
     include_package_data = True,
     install_requires = requires,
     entry_points = entry_points,

tests/content/wordpressexport.xml

@@ -0,0 +1,578 @@
+<?xml version="1.0" encoding="UTF-8" ?>
+<!-- This is a WordPress eXtended RSS file generated by WordPress as an export of your site. -->
+<!-- It contains information about your site's posts, pages, comments, categories, and other content. -->
+<!-- You may use this file to transfer that content from one site to another. -->
+<!-- This file is not intended to serve as a complete backup of your site. -->
+
+<!-- To import this information into a WordPress site follow these steps: -->
+<!-- 1. Log in to that site as an administrator. -->
+<!-- 2. Go to Tools: Import in the WordPress admin panel. -->
+<!-- 3. Install the "WordPress" importer from the list. -->
+<!-- 4. Activate & Run Importer. -->
+<!-- 5. Upload this file using the form provided on that page. -->
+<!-- 6. You will first be asked to map the authors in this export file to users -->
+<!--    on the site. For each author, you may choose to map to an -->
+<!--    existing user on the site or to create a new user. -->
+<!-- 7. WordPress will then import each of the posts, pages, comments, categories, etc. -->
+<!--    contained in this file into your site. -->
+
+<!-- generator="WordPress/3.3.1" created="2012-05-13 01:13" -->
+<rss version="2.0"
+    xmlns:excerpt="http://wordpress.org/export/1.1/excerpt/"
+    xmlns:content="http://purl.org/rss/1.0/modules/content/"
+    xmlns:wfw="http://wellformedweb.org/CommentAPI/"
+    xmlns:dc="http://purl.org/dc/elements/1.1/"
+    xmlns:wp="http://wordpress.org/export/1.1/"
+>
+
+<channel>
+    <title>Pelican test channel</title>
+    <link>http://thisisa.test</link>
+    <description>Not a real feed, just for test</description>
+    <pubDate>Sun, 13 May 2012 01:13:52 +0000</pubDate>
+    <language>en</language>
+    <wp:wxr_version>1.1</wp:wxr_version>
+    <wp:base_site_url>http://thisisa.test</wp:base_site_url>
+    <wp:base_blog_url>http://thisisa.test</wp:base_blog_url>
+
+    <wp:author><wp:author_id>2</wp:author_id><wp:author_login>Bob</wp:author_login><wp:author_email>bob@thisisa.test</wp:author_email><wp:author_display_name><![CDATA[Bob]]></wp:author_display_name><wp:author_first_name><![CDATA[Bob]]></wp:author_first_name><wp:author_last_name><![CDATA[]]></wp:author_last_name></wp:author>
+    <wp:author><wp:author_id>3</wp:author_id><wp:author_login>Jonh</wp:author_login><wp:author_email>jonh@thisisa.test</wp:author_email><wp:author_display_name><![CDATA[Jonh]]></wp:author_display_name><wp:author_first_name><![CDATA[Jonh]]></wp:author_first_name><wp:author_last_name><![CDATA[]]></wp:author_last_name></wp:author>
+
+    <wp:category><wp:term_id>7</wp:term_id><wp:category_nicename>categ-1</wp:category_nicename><wp:category_parent></wp:category_parent><wp:cat_name><![CDATA[Category 1]]></wp:cat_name></wp:category>
+    <wp:category><wp:term_id>11</wp:term_id><wp:category_nicename>categ-2</wp:category_nicename><wp:category_parent></wp:category_parent><wp:cat_name><![CDATA[Category 2]]></wp:cat_name></wp:category>
+    <wp:category><wp:term_id>1</wp:term_id><wp:category_nicename>uncategorized</wp:category_nicename><wp:category_parent></wp:category_parent><wp:cat_name><![CDATA[Uncategorized]]></wp:cat_name></wp:category>
+    <wp:category><wp:term_id>15</wp:term_id><wp:category_nicename>categ-3</wp:category_nicename><wp:category_parent></wp:category_parent><wp:cat_name><![CDATA[Category 3]]></wp:cat_name></wp:category>
+    <wp:tag><wp:term_id>25</wp:term_id><wp:tag_slug>tag-1</wp:tag_slug><wp:tag_name><![CDATA[tag 1]]></wp:tag_name></wp:tag>
+    <wp:tag><wp:term_id>122</wp:term_id><wp:tag_slug>tag2</wp:tag_slug><wp:tag_name><![CDATA[Tag2]]></wp:tag_name></wp:tag>
+    <wp:tag><wp:term_id>68</wp:term_id><wp:tag_slug>tag-3</wp:tag_slug><wp:tag_name><![CDATA[Tag 3]]></wp:tag_name></wp:tag>
+
+    <generator>http://wordpress.org/?v=3.3.1</generator>
+
+    <item>
+        <title>Empty post</title>
+        <link>http://thisisa.test/?attachment_id=24</link>
+        <pubDate>Sat, 04 Feb 2012 03:17:33 +0000</pubDate>
+        <dc:creator>bob</dc:creator>
+        <guid isPermaLink="false">https://upload.wikimedia.org/wikipedia/commons/thumb/2/2c/Pelican_lakes_entrance02.jpg/240px-Pelican_lakes_entrance02.jpg</guid>
+        <description></description>
+        <content:encoded><![CDATA[]]></content:encoded>
+        <excerpt:encoded><![CDATA[]]></excerpt:encoded>
+        <wp:post_id>24</wp:post_id>
+        <wp:post_date>2012-02-04 03:17:33</wp:post_date>
+        <wp:post_date_gmt>2012-02-04 03:17:33</wp:post_date_gmt>
+        <wp:comment_status>open</wp:comment_status>
+        <wp:ping_status>open</wp:ping_status>
+        <wp:post_name>empty-post</wp:post_name>
+        <wp:status>inherit</wp:status>
+        <wp:post_parent>0</wp:post_parent>
+        <wp:menu_order>0</wp:menu_order>
+        <wp:post_type>attachment</wp:post_type>
+        <wp:post_password></wp:post_password>
+        <wp:is_sticky>0</wp:is_sticky>
+        <wp:attachment_url>https://upload.wikimedia.org/wikipedia/commons/thumb/2/2c/Pelican_lakes_entrance02.jpg/240px-Pelican_lakes_entrance02.jpg</wp:attachment_url>
+        <wp:postmeta>
+            <wp:meta_key>_wp_attachment_metadata</wp:meta_key>
+            <wp:meta_value><![CDATA[a:5:{s:5:"width";s:3:"150";s:6:"height";s:3:"186";s:14:"hwstring_small";s:22:"height='96' width='77'";s:4:"file";s:20:"2012/02/pelican.png";s:10:"image_meta";a:10:{s:8:"aperture";s:1:"0";s:6:"credit";s:0:"";s:6:"camera";s:0:"";s:7:"caption";s:0:"";s:17:"created_timestamp";s:1:"0";s:9:"copyright";s:0:"";s:12:"focal_length";s:1:"0";s:3:"iso";s:1:"0";s:13:"shutter_speed";s:1:"0";s:5:"title";s:0:"";}}]]></wp:meta_value>
+        </wp:postmeta>
+        <wp:postmeta>
+            <wp:meta_key>_wp_attached_file</wp:meta_key>
+            <wp:meta_value><![CDATA[2012/02/stuff.png]]></wp:meta_value>
+        </wp:postmeta>
+        <wp:postmeta>
+            <wp:meta_key>_wp_attachment_image_alt</wp:meta_key>
+            <wp:meta_value><![CDATA[Stuff]]></wp:meta_value>
+        </wp:postmeta>
+    </item>
+    <item>
+        <title></title>
+        <link>http://thisisa.test/?p=168</link>
+        <pubDate>Thu, 01 Jan 1970 00:00:00 +0000</pubDate>
+        <dc:creator>bob</dc:creator>
+        <guid isPermaLink="false">http://thisisa.test/?p=168</guid>
+        <description></description>
+        <content:encoded><![CDATA[This is a draft with no title]]></content:encoded>
+        <excerpt:encoded><![CDATA[]]></excerpt:encoded>
+        <wp:post_id>168</wp:post_id>
+        <wp:post_date>2012-02-15 21:23:57</wp:post_date>
+        <wp:post_date_gmt>0000-00-00 00:00:00</wp:post_date_gmt>
+        <wp:comment_status>open</wp:comment_status>
+        <wp:ping_status>open</wp:ping_status>
+        <wp:post_name></wp:post_name>
+        <wp:status>draft</wp:status>
+        <wp:post_parent>0</wp:post_parent>
+        <wp:menu_order>0</wp:menu_order>
+        <wp:post_type>post</wp:post_type>
+        <wp:post_password></wp:post_password>
+        <wp:is_sticky>0</wp:is_sticky>
+        <category domain="category" nicename="categ-1"><![CDATA[Category 1]]></category>
+        <wp:postmeta>
+            <wp:meta_key>_edit_last</wp:meta_key>
+            <wp:meta_value><![CDATA[3]]></wp:meta_value>
+        </wp:postmeta>
+    </item>
+    <item>
+        <title>A normal post</title>
+        <link>http://thisisa.test/?p=173</link>
+        <pubDate>Thu, 01 Jan 1970 00:00:00 +0000</pubDate>
+        <dc:creator>bob</dc:creator>
+        <guid isPermaLink="false">http://thisisa.test/?p=173</guid>
+        <description></description>
+        <content:encoded><![CDATA[Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
+
+<ul>
+    <li>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</li>
+    <li>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</li>
+</ul>
+
+Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.]]></content:encoded>
+        <excerpt:encoded><![CDATA[]]></excerpt:encoded>
+        <wp:post_id>173</wp:post_id>
+        <wp:post_date>2012-02-16 15:52:55</wp:post_date>
+        <wp:post_date_gmt>0000-00-00 00:00:00</wp:post_date_gmt>
+        <wp:comment_status>open</wp:comment_status>
+        <wp:ping_status>open</wp:ping_status>
+        <wp:post_name></wp:post_name>
+        <wp:status>draft</wp:status>
+        <wp:post_parent>0</wp:post_parent>
+        <wp:menu_order>0</wp:menu_order>
+        <wp:post_type>post</wp:post_type>
+        <wp:post_password></wp:post_password>
+        <wp:is_sticky>0</wp:is_sticky>
+        <category domain="category" nicename="category-2"><![CDATA[Category 2]]></category>
+        <wp:postmeta>
+            <wp:meta_key>_edit_last</wp:meta_key>
+            <wp:meta_value><![CDATA[3]]></wp:meta_value>
+        </wp:postmeta>
+    </item>
+    <item>
+        <title>Complete draft</title>
+        <link>http://thisisa.test/?p=176</link>
+        <pubDate>Thu, 01 Jan 1970 00:00:00 +0000</pubDate>
+        <dc:creator>bob</dc:creator>
+        <guid isPermaLink="false">http://thisisa.test/?p=176</guid>
+        <description></description>
+        <content:encoded><![CDATA[Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.]]></content:encoded>
+        <excerpt:encoded><![CDATA[]]></excerpt:encoded>
+        <wp:post_id>176</wp:post_id>
+        <wp:post_date>2012-02-17 15:11:55</wp:post_date>
+        <wp:post_date_gmt>0000-00-00 00:00:00</wp:post_date_gmt>
+        <wp:comment_status>open</wp:comment_status>
+        <wp:ping_status>open</wp:ping_status>
+        <wp:post_name></wp:post_name>
+        <wp:status>draft</wp:status>
+        <wp:post_parent>0</wp:post_parent>
+        <wp:menu_order>0</wp:menu_order>
+        <wp:post_type>post</wp:post_type>
+        <wp:post_password></wp:post_password>
+        <wp:is_sticky>0</wp:is_sticky>
+        <category domain="category" nicename="category-3"><![CDATA[Category 3]]></category>
+        <wp:postmeta>
+            <wp:meta_key>_edit_last</wp:meta_key>
+            <wp:meta_value><![CDATA[3]]></wp:meta_value>
+        </wp:postmeta>
+    </item>
+    <item>
+        <title>Page</title>
+        <link>http://thisisa.test/contact/</link>
+        <pubDate>Wed, 11 Apr 2012 11:38:08 +0000</pubDate>
+        <dc:creator>bob</dc:creator>
+        <guid isPermaLink="false">http://thisisa.test/?page_id=334</guid>
+        <description></description>
+        <content:encoded><![CDATA[Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.]]></content:encoded>
+        <excerpt:encoded><![CDATA[]]></excerpt:encoded>
+        <wp:post_id>334</wp:post_id>
+        <wp:post_date>2012-04-11 06:38:08</wp:post_date>
+        <wp:post_date_gmt>2012-04-11 11:38:08</wp:post_date_gmt>
+        <wp:comment_status>open</wp:comment_status>
+        <wp:ping_status>open</wp:ping_status>
+        <wp:post_name>contact</wp:post_name>
+        <wp:status>publish</wp:status>
+        <wp:post_parent>0</wp:post_parent>
+        <wp:menu_order>0</wp:menu_order>
+        <wp:post_type>page</wp:post_type>
+        <wp:post_password></wp:post_password>
+        <wp:is_sticky>0</wp:is_sticky>
+        <wp:postmeta>
+            <wp:meta_key>sharing_disabled</wp:meta_key>
+            <wp:meta_value><![CDATA[1]]></wp:meta_value>
+        </wp:postmeta>
+        <wp:postmeta>
+            <wp:meta_key>_wp_page_template</wp:meta_key>
+            <wp:meta_value><![CDATA[default]]></wp:meta_value>
+        </wp:postmeta>
+        <wp:postmeta>
+            <wp:meta_key>_edit_last</wp:meta_key>
+            <wp:meta_value><![CDATA[3]]></wp:meta_value>
+        </wp:postmeta>
+    </item>
+    <item>
+        <title>Empty Page</title>
+        <link>http://thisisa.test/empty/</link>
+        <pubDate>Wed, 11 Apr 2012 11:38:08 +0000</pubDate>
+        <dc:creator>bob</dc:creator>
+        <guid isPermaLink="false">http://thisisa.test/?page_id=334</guid>
+        <description></description>
+        <content:encoded><![CDATA[]]></content:encoded>
+        <excerpt:encoded><![CDATA[]]></excerpt:encoded>
+        <wp:post_id>334</wp:post_id>
+        <wp:post_date>2012-04-11 06:38:08</wp:post_date>
+        <wp:post_date_gmt>2012-04-11 11:38:08</wp:post_date_gmt>
+        <wp:comment_status>open</wp:comment_status>
+        <wp:ping_status>open</wp:ping_status>
+        <wp:post_name>empty</wp:post_name>
+        <wp:status>publish</wp:status>
+        <wp:post_parent>0</wp:post_parent>
+        <wp:menu_order>0</wp:menu_order>
+        <wp:post_type>page</wp:post_type>
+        <wp:post_password></wp:post_password>
+        <wp:is_sticky>0</wp:is_sticky>
+        <wp:postmeta>
+            <wp:meta_key>sharing_disabled</wp:meta_key>
+            <wp:meta_value><![CDATA[1]]></wp:meta_value>
+        </wp:postmeta>
+        <wp:postmeta>
+            <wp:meta_key>_wp_page_template</wp:meta_key>
+            <wp:meta_value><![CDATA[default]]></wp:meta_value>
+        </wp:postmeta>
+        <wp:postmeta>
+            <wp:meta_key>_edit_last</wp:meta_key>
+            <wp:meta_value><![CDATA[3]]></wp:meta_value>
+        </wp:postmeta>
+    </item>
+    <item>
+        <title>Special chars: l&#039;é</title>
+        <link>http://thisisa.test/?p=471</link>
+        <pubDate>Thu, 01 Jan 1970 00:00:00 +0000</pubDate>
+        <dc:creator>bob</dc:creator>
+        <guid isPermaLink="false">http://thisisa.test/?p=471</guid>
+        <description></description>
+        <content:encoded><![CDATA[l&#039;é]]></content:encoded>
+        <excerpt:encoded><![CDATA[]]></excerpt:encoded>
+        <wp:post_id>471</wp:post_id>
+        <wp:post_date>2012-04-29 09:44:27</wp:post_date>
+        <wp:post_date_gmt>0000-00-00 00:00:00</wp:post_date_gmt>
+        <wp:comment_status>open</wp:comment_status>
+        <wp:ping_status>open</wp:ping_status>
+        <wp:post_name></wp:post_name>
+        <wp:status>draft</wp:status>
+        <wp:post_parent>0</wp:post_parent>
+        <wp:menu_order>0</wp:menu_order>
+        <wp:post_type>post</wp:post_type>
+        <wp:post_password></wp:post_password>
+        <wp:is_sticky>0</wp:is_sticky>
+        <category domain="category" nicename="category-1"><![CDATA[Category 1]]></category>
+        <wp:postmeta>
+            <wp:meta_key>_edit_last</wp:meta_key>
+            <wp:meta_value><![CDATA[3]]></wp:meta_value>
+        </wp:postmeta>
+    </item>
+
+    <item>
+        <title>With excerpt</title>
+        <link>http://thisisa.test/with-excerpt/</link>
+        <pubDate>Sat, 04 Feb 2012 02:03:06 +0000</pubDate>
+        <dc:creator>bob</dc:creator>
+        <guid isPermaLink="false">http://thisisa.test/?p=8</guid>
+        <description></description>
+        <content:encoded><![CDATA[Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
+
+Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
+
+Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.]]></content:encoded>
+        <excerpt:encoded><![CDATA[Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.]]></excerpt:encoded>
+        <wp:post_id>8</wp:post_id>
+        <wp:post_date>2012-02-04 02:03:06</wp:post_date>
+        <wp:post_date_gmt>2012-02-04 02:03:06</wp:post_date_gmt>
+        <wp:comment_status>open</wp:comment_status>
+        <wp:ping_status>open</wp:ping_status>
+        <wp:post_name>with-excerpt</wp:post_name>
+        <wp:status>publish</wp:status>
+        <wp:post_parent>0</wp:post_parent>
+        <wp:menu_order>0</wp:menu_order>
+        <wp:post_type>post</wp:post_type>
+        <wp:post_password></wp:post_password>
+        <wp:is_sticky>0</wp:is_sticky>
+        <category domain="category" nicename="Category 1"><![CDATA[Category 1]]></category>
+        <category domain="post_tag" nicename="tag-1"><![CDATA[tag 1]]></category>
+        <category domain="post_tag" nicename="tag2"><![CDATA[Tag2]]></category>
+        <wp:postmeta>
+            <wp:meta_key>_edit_last</wp:meta_key>
+            <wp:meta_value><![CDATA[3]]></wp:meta_value>
+        </wp:postmeta>
+        <wp:postmeta>
+            <wp:meta_key>et_bigpost</wp:meta_key>
+            <wp:meta_value><![CDATA[0]]></wp:meta_value>
+        </wp:postmeta>
+        <wp:postmeta>
+            <wp:meta_key>_thumbnail_id</wp:meta_key>
+            <wp:meta_value><![CDATA[32]]></wp:meta_value>
+        </wp:postmeta>
+    </item>
+    <item>
+        <title>With tags</title>
+        <link>http://thisisa.test/tags/</link>
+        <pubDate>Sat, 04 Feb 2012 21:05:25 +0000</pubDate>
+        <dc:creator>bob</dc:creator>
+        <guid isPermaLink="false">http://thisisa.test/?p=25</guid>
+        <description></description>
+        <content:encoded><![CDATA[Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.]]></content:encoded>
+        <excerpt:encoded><![CDATA[]]></excerpt:encoded>
+        <wp:post_id>25</wp:post_id>
+        <wp:post_date>2012-02-04 21:05:25</wp:post_date>
+        <wp:post_date_gmt>2012-02-04 21:05:25</wp:post_date_gmt>
+        <wp:comment_status>open</wp:comment_status>
+        <wp:ping_status>open</wp:ping_status>
+        <wp:post_name>with-tags</wp:post_name>
+        <wp:status>publish</wp:status>
+        <wp:post_parent>0</wp:post_parent>
+        <wp:menu_order>0</wp:menu_order>
+        <wp:post_type>post</wp:post_type>
+        <wp:post_password></wp:post_password>
+        <wp:is_sticky>0</wp:is_sticky>
+        <category domain="category" nicename="category-3"><![CDATA[Category 3]]></category>
+        <category domain="post_tag" nicename="tag-1"><![CDATA[tag 1]]></category>
+        <category domain="post_tag" nicename="tag-2"><![CDATA[Tag2]]></category>
+        <category domain="post_tag" nicename="tag-3"><![CDATA[Tag 3]]></category>
+        <wp:postmeta>
+            <wp:meta_key>_edit_last</wp:meta_key>
+            <wp:meta_value><![CDATA[3]]></wp:meta_value>
+        </wp:postmeta>
+        <wp:postmeta>
+            <wp:meta_key>et_bigpost</wp:meta_key>
+            <wp:meta_value><![CDATA[0]]></wp:meta_value>
+        </wp:postmeta>
+        <wp:postmeta>
+            <wp:meta_key>_thumbnail_id</wp:meta_key>
+            <wp:meta_value><![CDATA[29]]></wp:meta_value>
+        </wp:postmeta>
+    </item>
+	<item>
+        <title>With comments</title>
+        <link>http://thisisa.test/with-comments/</link>
+        <pubDate>Wed, 18 Apr 2012 08:36:26 +0000</pubDate>
+        <dc:creator>john</dc:creator>
+        <guid isPermaLink="false">http://thisisa.test/?p=422</guid>
+        <description></description>
+        <content:encoded><![CDATA[Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.]]></content:encoded>
+        <excerpt:encoded><![CDATA[]]></excerpt:encoded>
+        <wp:post_id>422</wp:post_id>
+        <wp:post_date>2012-04-18 03:36:26</wp:post_date>
+        <wp:post_date_gmt>2012-04-18 08:36:26</wp:post_date_gmt>
+        <wp:comment_status>open</wp:comment_status>
+        <wp:ping_status>open</wp:ping_status>
+        <wp:post_name>with-comments</wp:post_name>
+        <wp:status>publish</wp:status>
+        <wp:post_parent>0</wp:post_parent>
+        <wp:menu_order>0</wp:menu_order>
+        <wp:post_type>post</wp:post_type>
+        <wp:post_password></wp:post_password>
+        <wp:is_sticky>0</wp:is_sticky>
+        <category domain="category" nicename="category-1"><![CDATA[Category 1]]></category>
+        <wp:postmeta>
+            <wp:meta_key>_edit_last</wp:meta_key>
+            <wp:meta_value><![CDATA[2]]></wp:meta_value>
+        </wp:postmeta>
+        <wp:postmeta>
+            <wp:meta_key>_thumbnail_id</wp:meta_key>
+            <wp:meta_value><![CDATA[423]]></wp:meta_value>
+        </wp:postmeta>
+        <wp:comment>
+            <wp:comment_id>116</wp:comment_id>
+            <wp:comment_author><![CDATA[User2]]></wp:comment_author>
+            <wp:comment_author_email>User2@mail.test</wp:comment_author_email>
+            <wp:comment_author_url></wp:comment_author_url>
+            <wp:comment_author_IP>127.0.0.1</wp:comment_author_IP>
+            <wp:comment_date>2012-05-06 15:46:06</wp:comment_date>
+            <wp:comment_date_gmt>2012-05-06 20:46:06</wp:comment_date_gmt>
+            <wp:comment_content><![CDATA[Comment content.]]></wp:comment_content>
+            <wp:comment_approved>1</wp:comment_approved>
+            <wp:comment_type></wp:comment_type>
+            <wp:comment_parent>0</wp:comment_parent>
+            <wp:comment_user_id>0</wp:comment_user_id>
+            <wp:commentmeta>
+                <wp:meta_key>akismet_result</wp:meta_key>
+                <wp:meta_value><![CDATA[false]]></wp:meta_value>
+            </wp:commentmeta>
+            <wp:commentmeta>
+                <wp:meta_key>akismet_history</wp:meta_key>
+                <wp:meta_value><![CDATA[a:4:{s:4:"time";s:15:"1336337189.7974";s:7:"message";s:28:"Akismet cleared this comment";s:5:"event";s:9:"check-ham";s:4:"user";s:0:"";}]]></wp:meta_value>
+            </wp:commentmeta>
+            <wp:commentmeta>
+                <wp:meta_key>akismet_as_submitted</wp:meta_key>
+                <wp:meta_value><![CDATA[a:64:{s:15:"comment_post_ID";s:3:"422";s:14:"comment_author";s:8:"Baronsed";s:20:"comment_author_email";s:15:"User2@mail.test";s:18:"comment_author_url";s:0:"";s:15:"comment_content";s:118:"Dans les listes d'articles où celui-ci apparaît, l'image est trop grande et ralentit le chargement de toute la page.";s:12:"comment_type";s:0:"";s:14:"comment_parent";s:1:"0";s:7:"user_ID";s:1:"0";s:7:"user_ip";s:14:"127.0.0.1";s:10:"user_agent";s:74:"Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:12.0) Gecko/20100101 Firefox/12.0";s:8:"referrer";s:0:"";s:4:"blog";s:19:"http://thisisa.test";s:9:"blog_lang";s:5:"en_US";s:12:"blog_charset";s:5:"UTF-8";s:9:"permalink";s:62:"http://thisisa.test/htop-un-visualiseur-interactif-de-process/";s:9:"user_role";s:0:"";s:21:"akismet_comment_nonce";s:6:"passed";s:11:"POST_author";s:8:"Baronsed";s:10:"POST_email";s:15:"User2@mail.test";s:8:"POST_url";s:0:"";s:12:"POST_comment";s:118:"Dans les listes d'articles où celui-ci apparaît, l'image est trop grande et ralentit le chargement de toute la page.";s:11:"POST_submit";s:14:"Submit Comment";s:20:"POST_comment_post_ID";s:3:"422";s:19:"POST_comment_parent";s:1:"0";s:26:"POST_akismet_comment_nonce";s:10:"96b66769dc";s:15:"SERVER_SOFTWARE";s:12:"nginx/0.8.55";s:11:"REQUEST_URI";s:21:"/wp-comments-post.php";s:4:"TERM";s:5:"xterm";s:17:"PHP_FCGI_CHILDREN";s:1:"4";s:4:"PATH";s:29:"/sbin:/usr/sbin:/bin:/usr/bin";s:1:"_";s:25:"/usr/local/bin/spawn-fcgi";s:3:"PWD";s:1:"/";s:4:"LANG";s:11:"en_US.UTF-8";s:5:"SHLVL";s:1:"2";s:21:"PHP_FCGI_MAX_REQUESTS";s:4:"1000";s:9:"FCGI_ROLE";s:9:"RESPONDER";s:12:"QUERY_STRING";s:0:"";s:14:"REQUEST_METHOD";s:4:"POST";s:12:"CONTENT_TYPE";s:33:"application/x-www-form-urlencoded";s:14:"CONTENT_LENGTH";s:3:"277";s:11:"SCRIPT_NAME";s:21:"/wp-comments-post.php";s:12:"DOCUMENT_URI";s:21:"/wp-comments-post.php";s:13:"DOCUMENT_ROOT";s:14:"/home/blog";s:15:"SERVER_PROTOCOL";s:8:"HTTP/1.1";s:17:"GATEWAY_INTERFACE";s:7:"CGI/1.1";s:11:"REMOTE_ADDR";s:14:"127.0.0.1";s:11:"REMOTE_PORT";s:5:"52826";s:11:"SERVER_ADDR";s:13:"127.0.0.1";s:11:"SERVER_PORT";s:2:"80";s:11:"SERVER_NAME";s:12:"thisisa.test";s:15:"REDIRECT_STATUS";s:3:"200";s:15:"SCRIPT_FILENAME";s:35:"/home/blog/wp-comments-post.php";s:9:"HTTP_HOST";s:12:"thisisa.test";s:15:"HTTP_USER_AGENT";s:74:"Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:12.0) Gecko/20100101 Firefox/12.0";s:11:"HTTP_ACCEPT";s:63:"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";s:20:"HTTP_ACCEPT_LANGUAGE";s:35:"fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3";s:20:"HTTP_ACCEPT_ENCODING";s:13:"gzip, deflate";s:8:"HTTP_DNT";s:1:"1";s:15:"HTTP_CONNECTION";s:10:"keep-alive";s:11:"HTTP_COOKIE";s:0:"";s:17:"HTTP_CONTENT_TYPE";s:33:"application/x-www-form-urlencoded";s:19:"HTTP_CONTENT_LENGTH";s:3:"277";s:8:"PHP_SELF";s:21:"/wp-comments-post.php";s:12:"REQUEST_TIME";s:10:"1336337164";}]]></wp:meta_value>
+            </wp:commentmeta>
+        </wp:comment>
+        <wp:comment>
+            <wp:comment_id>117</wp:comment_id>
+            <wp:comment_author><![CDATA[Bob]]></wp:comment_author>
+            <wp:comment_author_email>bob@thisisa.test</wp:comment_author_email>
+            <wp:comment_author_url></wp:comment_author_url>
+            <wp:comment_author_IP>127.0.0.1</wp:comment_author_IP>
+            <wp:comment_date>2012-05-06 17:44:06</wp:comment_date>
+            <wp:comment_date_gmt>2012-05-06 22:44:06</wp:comment_date_gmt>
+            <wp:comment_content><![CDATA[Comment content.]]></wp:comment_content>
+            <wp:comment_approved>1</wp:comment_approved>
+            <wp:comment_type></wp:comment_type>
+            <wp:comment_parent>116</wp:comment_parent>
+            <wp:comment_user_id>3</wp:comment_user_id>
+            <wp:commentmeta>
+                <wp:meta_key>akismet_result</wp:meta_key>
+                <wp:meta_value><![CDATA[false]]></wp:meta_value>
+            </wp:commentmeta>
+            <wp:commentmeta>
+                <wp:meta_key>akismet_history</wp:meta_key>
+                <wp:meta_value><![CDATA[a:4:{s:4:"time";s:15:"1336344263.8658";s:7:"message";s:28:"Akismet cleared this comment";s:5:"event";s:9:"check-ham";s:4:"user";s:3:"bob";}]]></wp:meta_value>
+            </wp:commentmeta>
+            <wp:commentmeta>
+                <wp:meta_key>akismet_as_submitted</wp:meta_key>
+                <wp:meta_value><![CDATA[a:74:{s:15:"comment_post_ID";s:3:"422";s:14:"comment_author";s:3:"bob";s:20:"comment_author_email";s:25:"bob@thisisa.test";s:18:"comment_author_url";s:0:"";s:15:"comment_content";s:26:"Merci de l'avoir signalé.";s:14:"comment_parent";s:3:"116";s:7:"user_ID";s:1:"3";s:7:"user_ip";s:14:"127.0.0.1";s:10:"user_agent";s:139:"Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/12.04 Chromium/127.0.0.1 Chrome/127.0.0.1 Safari/535.19";s:8:"referrer";s:29:"http://thisisa.test/wp-admin/";s:4:"blog";s:19:"http://thisisa.test";s:9:"blog_lang";s:5:"en_US";s:12:"blog_charset";s:5:"UTF-8";s:9:"permalink";s:62:"http://thisisa.test/htop-un-visualiseur-interactif-de-process/";s:9:"user_role";s:13:"administrator";s:21:"akismet_comment_nonce";s:6:"passed";s:22:"POST_newcomment_author";s:0:"";s:28:"POST_newcomment_author_email";s:0:"";s:26:"POST_newcomment_author_url";s:0:"";s:12:"POST_user_ID";s:1:"3";s:11:"POST_action";s:15:"replyto-comment";s:15:"POST_comment_ID";s:3:"116";s:20:"POST_comment_post_ID";s:3:"422";s:11:"POST_status";s:0:"";s:13:"POST_position";s:2:"-1";s:13:"POST_checkbox";s:1:"0";s:9:"POST_mode";s:9:"dashboard";s:32:"POST__ajax_nonce-replyto-comment";s:10:"d1ad3bd917";s:32:"POST__wp_unfiltered_html_comment";s:10:"fc11aee860";s:12:"POST_content";s:26:"Merci de l'avoir signalé.";s:7:"POST_id";s:3:"422";s:21:"POST_comments_listing";s:0:"";s:15:"SERVER_SOFTWARE";s:12:"nginx/0.8.55";s:11:"REQUEST_URI";s:24:"/wp-admin/admin-ajax.php";s:4:"TERM";s:5:"xterm";s:17:"PHP_FCGI_CHILDREN";s:1:"4";s:4:"PATH";s:29:"/sbin:/usr/sbin:/bin:/usr/bin";s:1:"_";s:25:"/usr/local/bin/spawn-fcgi";s:3:"PWD";s:1:"/";s:4:"LANG";s:11:"en_US.UTF-8";s:5:"SHLVL";s:1:"2";s:21:"PHP_FCGI_MAX_REQUESTS";s:4:"1000";s:9:"FCGI_ROLE";s:9:"RESPONDER";s:12:"QUERY_STRING";s:0:"";s:14:"REQUEST_METHOD";s:4:"POST";s:12:"CONTENT_TYPE";s:33:"application/x-www-form-urlencoded";s:14:"CONTENT_LENGTH";s:3:"322";s:11:"SCRIPT_NAME";s:24:"/wp-admin/admin-ajax.php";s:12:"DOCUMENT_URI";s:24:"/wp-admin/admin-ajax.php";s:13:"DOCUMENT_ROOT";s:14:"/home/blog";s:15:"SERVER_PROTOCOL";s:8:"HTTP/1.1";s:17:"GATEWAY_INTERFACE";s:7:"CGI/1.1";s:11:"REMOTE_ADDR";s:14:"127.0.0.1";s:11:"REMOTE_PORT";s:5:"10252";s:11:"SERVER_ADDR";s:13:"127.0.0.1";s:11:"SERVER_PORT";s:2:"80";s:11:"SERVER_NAME";s:12:"thisisa.test";s:15:"REDIRECT_STATUS";s:3:"200";s:15:"SCRIPT_FILENAME";s:38:"/home/blog/wp-admin/admin-ajax.php";s:9:"HTTP_HOST";s:12:"thisisa.test";s:15:"HTTP_CONNECTION";s:10:"keep-alive";s:19:"HTTP_CONTENT_LENGTH";s:3:"322";s:11:"HTTP_ORIGIN";s:19:"http://thisisa.test";s:21:"HTTP_X_REQUESTED_WITH";s:14:"XMLHttpRequest";s:15:"HTTP_USER_AGENT";s:139:"Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/12.04 Chromium/127.0.0.1 Chrome/127.0.0.1 Safari/535.19";s:17:"HTTP_CONTENT_TYPE";s:33:"application/x-www-form-urlencoded";s:11:"HTTP_ACCEPT";s:3:"*/*";s:12:"HTTP_REFERER";s:29:"http://thisisa.test/wp-admin/";s:20:"HTTP_ACCEPT_ENCODING";s:17:"gzip,deflate,sdch";s:20:"HTTP_ACCEPT_LANGUAGE";s:35:"fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4";s:19:"HTTP_ACCEPT_CHARSET";s:30:"ISO-8859-1,utf-8;q=0.7,*;q=0.3";s:11:"HTTP_COOKIE";s:0:"";s:8:"PHP_SELF";s:24:"/wp-admin/admin-ajax.php";s:12:"REQUEST_TIME";s:10:"1336344246";}]]></wp:meta_value>
+            </wp:commentmeta>
+        </wp:comment>
+        <wp:comment>
+            <wp:comment_id>156</wp:comment_id>
+            <wp:comment_author><![CDATA[Ping back comment author]]></wp:comment_author>
+            <wp:comment_author_email></wp:comment_author_email>
+            <wp:comment_author_url>http://thisisa.test/to-article-you-ping-back/</wp:comment_author_url>
+            <wp:comment_author_IP>127.0.0.1</wp:comment_author_IP>
+            <wp:comment_date>2012-05-09 19:30:19</wp:comment_date>
+            <wp:comment_date_gmt>2012-05-10 00:30:19</wp:comment_date_gmt>
+            <wp:comment_content><![CDATA[[...]this is a ping pack [...] ]]></wp:comment_content>
+            <wp:comment_approved>trash</wp:comment_approved>
+            <wp:comment_type>pingback</wp:comment_type>
+            <wp:comment_parent>0</wp:comment_parent>
+            <wp:comment_user_id>0</wp:comment_user_id>
+            <wp:commentmeta>
+                <wp:meta_key>akismet_history</wp:meta_key>
+                <wp:meta_value><![CDATA[a:4:{s:4:"time";s:15:"1336610793.7343";s:7:"message";s:39:"bob changed the comment status to trash";s:5:"event";s:12:"status-trash";s:4:"user";s:3:"bob";}]]></wp:meta_value>
+            </wp:commentmeta>
+            <wp:commentmeta>
+                <wp:meta_key>_wp_trash_meta_status</wp:meta_key>
+                <wp:meta_value><![CDATA[0]]></wp:meta_value>
+            </wp:commentmeta>
+            <wp:commentmeta>
+                <wp:meta_key>_wp_trash_meta_time</wp:meta_key>
+                <wp:meta_value><![CDATA[1336610793]]></wp:meta_value>
+            </wp:commentmeta>
+        </wp:comment>
+        <wp:comment>
+            <wp:comment_id>122</wp:comment_id>
+            <wp:comment_author><![CDATA[bob]]></wp:comment_author>
+            <wp:comment_author_email>bob@thisisa.test</wp:comment_author_email>
+            <wp:comment_author_url></wp:comment_author_url>
+            <wp:comment_author_IP>127.0.0.1</wp:comment_author_IP>
+            <wp:comment_date>2012-05-07 14:11:34</wp:comment_date>
+            <wp:comment_date_gmt>2012-05-07 19:11:34</wp:comment_date_gmt>
+            <wp:comment_content><![CDATA[Comment content.]]></wp:comment_content>
+            <wp:comment_approved>1</wp:comment_approved>
+            <wp:comment_type></wp:comment_type>
+            <wp:comment_parent>121</wp:comment_parent>
+            <wp:comment_user_id>3</wp:comment_user_id>
+            <wp:commentmeta>
+                <wp:meta_key>akismet_result</wp:meta_key>
+                <wp:meta_value><![CDATA[false]]></wp:meta_value>
+            </wp:commentmeta>
+            <wp:commentmeta>
+                <wp:meta_key>akismet_history</wp:meta_key>
+                <wp:meta_value><![CDATA[a:4:{s:4:"time";s:15:"1336417895.1821";s:7:"message";s:28:"Akismet cleared this comment";s:5:"event";s:9:"check-ham";s:4:"user";s:3:"bob";}]]></wp:meta_value>
+            </wp:commentmeta>
+            <wp:commentmeta>
+                <wp:meta_key>akismet_as_submitted</wp:meta_key>
+                <wp:meta_value><![CDATA[a:65:{s:15:"comment_post_ID";s:3:"422";s:14:"comment_author";s:3:"bob";s:20:"comment_author_email";s:25:"bob@thisisa.test";s:18:"comment_author_url";s:0:"";s:15:"comment_content";s:832:"Comment content";s:12:"comment_type";s:0:"";s:14:"comment_parent";s:3:"121";s:7:"user_ID";s:1:"3";s:7:"user_ip";s:14:"127.0.0.1";s:10:"user_agent";s:139:"Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/12.04 Chromium/127.0.0.1 Chrome/127.0.0.1 Safari/535.19";s:8:"referrer";s:62:"http://thisisa.test/htop-un-visualiseur-interactif-de-process/";s:4:"blog";s:19:"http://thisisa.test";s:9:"blog_lang";s:5:"en_US";s:12:"blog_charset";s:5:"UTF-8";s:9:"permalink";s:62:"http://thisisa.test/htop-un-visualiseur-interactif-de-process/";s:9:"user_role";s:13:"administrator";s:21:"akismet_comment_nonce";s:6:"passed";s:12:"POST_comment";s:832:"Comment content.";s:11:"POST_submit";s:14:"Submit Comment";s:20:"POST_comment_post_ID";s:3:"422";s:19:"POST_comment_parent";s:3:"121";s:32:"POST__wp_unfiltered_html_comment";s:10:"9dacc22ee8";s:26:"POST_akismet_comment_nonce";s:10:"b9cbdae553";s:15:"SERVER_SOFTWARE";s:12:"nginx/0.8.55";s:11:"REQUEST_URI";s:21:"/wp-comments-post.php";s:4:"TERM";s:5:"xterm";s:17:"PHP_FCGI_CHILDREN";s:1:"4";s:4:"PATH";s:29:"/sbin:/usr/sbin:/bin:/usr/bin";s:1:"_";s:25:"/usr/local/bin/spawn-fcgi";s:3:"PWD";s:1:"/";s:4:"LANG";s:11:"en_US.UTF-8";s:5:"SHLVL";s:1:"2";s:21:"PHP_FCGI_MAX_REQUESTS";s:4:"1000";s:9:"FCGI_ROLE";s:9:"RESPONDER";s:12:"QUERY_STRING";s:0:"";s:14:"REQUEST_METHOD";s:4:"POST";s:12:"CONTENT_TYPE";s:33:"application/x-www-form-urlencoded";s:14:"CONTENT_LENGTH";s:4:"1143";s:11:"SCRIPT_NAME";s:21:"/wp-comments-post.php";s:12:"DOCUMENT_URI";s:21:"/wp-comments-post.php";s:13:"DOCUMENT_ROOT";s:14:"/home/blog";s:15:"SERVER_PROTOCOL";s:8:"HTTP/1.1";s:17:"GATEWAY_INTERFACE";s:7:"CGI/1.1";s:11:"REMOTE_ADDR";s:14:"127.0.0.1";s:11:"REMOTE_PORT";s:5:"10834";s:11:"SERVER_ADDR";s:13:"127.0.0.1";s:11:"SERVER_PORT";s:2:"80";s:11:"SERVER_NAME";s:12:"thisisa.test";s:15:"REDIRECT_STATUS";s:3:"200";s:15:"SCRIPT_FILENAME";s:35:"/home/blog/wp-comments-post.php";s:9:"HTTP_HOST";s:12:"thisisa.test";s:15:"HTTP_CONNECTION";s:10:"keep-alive";s:19:"HTTP_CONTENT_LENGTH";s:4:"1143";s:18:"HTTP_CACHE_CONTROL";s:9:"john-age=0";s:11:"HTTP_ORIGIN";s:19:"http://thisisa.test";s:15:"HTTP_USER_AGENT";s:139:"Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/12.04 Chromium/127.0.0.1 Chrome/127.0.0.1 Safari/535.19";s:17:"HTTP_CONTENT_TYPE";s:33:"application/x-www-form-urlencoded";s:11:"HTTP_ACCEPT";s:63:"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";s:12:"HTTP_REFERER";s:62:"http://thisisa.test/htop-un-visualiseur-interactif-de-process/";s:20:"HTTP_ACCEPT_ENCODING";s:17:"gzip,deflate,sdch";s:20:"HTTP_ACCEPT_LANGUAGE";s:35:"fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4";s:19:"HTTP_ACCEPT_CHARSET";s:30:"ISO-8859-1,utf-8;q=0.7,*;q=0.3";s:11:"HTTP_COOKIE";s:0:"";s:8:"PHP_SELF";s:21:"/wp-comments-post.php";s:12:"REQUEST_TIME";s:10:"1336417893";}]]></wp:meta_value>
+            </wp:commentmeta>
+        </wp:comment>
+    </item>
+    <item>
+        <title>Post with raw data</title>
+        <link>http://thisisa.test/?p=173</link>
+        <pubDate>Thu, 01 Jan 1970 00:00:00 +0000</pubDate>
+        <dc:creator>bob</dc:creator>
+        <guid isPermaLink="false">http://thisisa.test/?p=173</guid>
+        <description></description>
+        <content:encoded><![CDATA[<h1>Pelicans are scary</h1>
+
+Pelicans are supposed to eat fish, damn it!
+
+<iframe width="420" height="315" src="http://www.youtube.com/embed/QNNl_uWmQXE" frameborder="0" allowfullscreen></iframe>
+
+Bottom line: don't mess up with birds]]></content:encoded>
+        <excerpt:encoded><![CDATA[]]></excerpt:encoded>
+        <wp:post_id>173</wp:post_id>
+        <wp:post_date>2012-02-16 15:52:55</wp:post_date>
+        <wp:post_date_gmt>0000-00-00 00:00:00</wp:post_date_gmt>
+        <wp:comment_status>open</wp:comment_status>
+        <wp:ping_status>open</wp:ping_status>
+        <wp:post_name>post-with-raw-data</wp:post_name>
+        <wp:status>publish</wp:status>
+        <wp:post_parent>0</wp:post_parent>
+        <wp:menu_order>0</wp:menu_order>
+        <wp:post_type>post</wp:post_type>
+        <wp:post_password></wp:post_password>
+        <wp:is_sticky>0</wp:is_sticky>
+        <category domain="category" nicename="category-2"><![CDATA[Category 2]]></category>
+        <wp:postmeta>
+            <wp:meta_key>_edit_last</wp:meta_key>
+            <wp:meta_value><![CDATA[3]]></wp:meta_value>
+        </wp:postmeta>
+    </item>
+</channel>
+</rss>

tests/support.py

@@ -6,6 +6,11 @@ __all__ = [
 
 import os
 import subprocess
+import re
+import sys
+import cStringIO
+
+from functools import wraps
 from contextlib import contextmanager
 from tempfile import mkdtemp
 from shutil import rmtree
@@ -28,8 +33,87 @@ def temporary_folder():
             # do whatever you want
     """
     tempdir = mkdtemp()
-    yield tempdir
+    try:
-    rmtree(tempdir)
+        yield tempdir
+    finally:
+        rmtree(tempdir)
+
+
+def isplit(s, sep=None):
+    """
+        Behave like str.split but returns a generator instead of a list.
+
+        >>> list(isplit('\tUse the force\n')) == '\tUse the force\n'.split()
+        True
+        >>> list(isplit('\tUse the force\n')) == ['Use', 'the', 'force']
+        True
+        >>> list(isplit('\tUse the force\n', "e")) == '\tUse the force\n'.split("e")
+        True
+        >>> list(isplit('Use the force', "e")) == 'Use the force'.split("e")
+        True
+        >>> list(isplit('Use the force', "e")) == ['Us', ' th', ' forc', '']
+        True
+
+    """
+    sep, hardsep = r'\s+' if sep is None else re.escape(sep), sep is not None
+    exp, pos, l = re.compile(sep), 0, len(s)
+    while True:
+        m = exp.search(s, pos)
+        if not m:
+            if pos < l or hardsep:
+                #      ^ mimic "split()": ''.split() returns []
+                yield s[pos:]
+            break
+        start = m.start()
+        if pos < start or hardsep:
+            #           ^ mimic "split()": includes trailing empty string
+            yield s[pos:start]
+        pos = m.end()
+
+
+def mute(returns_output=False):
+    """
+        Decorate a function that prints to stdout, intercepting the output.
+        If "returns_output" is True, the function will return a generator
+        yielding the printed lines instead of the return values.
+
+        The decorator litterally hijack sys.stdout during each function
+        execution, so be careful with what you apply it to.
+
+        >>> def numbers():
+            print "42"
+            print "1984"
+        ...
+        >>> numbers()
+        42
+        1984
+        >>> mute()(numbers)()
+        >>> list(mute(True)(numbers)())
+        ['42', '1984']
+
+    """
+
+    def decorator(func):
+
+        @wraps(func)
+        def wrapper(*args, **kwargs):
+
+            saved_stdout = sys.stdout
+            sys.stdout = cStringIO.StringIO()
+
+            try:
+                out = func(*args, **kwargs)
+                if returns_output:
+                    out = isplit(sys.stdout.getvalue().strip())
+            finally:
+                sys.stdout = saved_stdout
+
+            return out
+
+        return wrapper
+
+    return decorator
+
 
 
 def get_article(title, slug, content, lang, extra_metadata=None):

tests/test_importer.py

@@ -0,0 +1,52 @@
+# -*- coding: utf-8 -*-
+
+import os
+
+from pelican.tools.pelican_import import wp2fields, fields2pelican
+from .support import unittest, temporary_folder, mute
+
+CUR_DIR = os.path.dirname(__file__)
+WORDPRESS_XML_SAMPLE = os.path.join(CUR_DIR, 'content', 'wordpressexport.xml')
+
+PANDOC = os.system('pandoc --version') == 0
+try:
+    import BeautifulSoup
+except ImportError:
+    BeautifulSoup = False  # NOQA
+
+
+class TestWordpressXmlImporter(unittest.TestCase):
+
+    def setUp(self):
+        self.posts = wp2fields(WORDPRESS_XML_SAMPLE)
+
+    @unittest.skipUnless(PANDOC and BeautifulSoup,
+                         'Needs Pandoc and BeautifulSoup')
+    def test_ignore_empty_posts(self):
+
+        posts = list(self.posts)
+        self.assertTrue(posts)
+        for title, content, fname, date, author, categ, tags, format in posts:
+            self.assertTrue(title.strip())
+
+    @unittest.skipUnless(PANDOC and BeautifulSoup,
+                         'Needs Pandoc and BeautifulSoup')
+    def test_can_toggle_raw_html_code_parsing(self):
+
+        posts = list(self.posts)
+        r = lambda f: open(f).read()
+        silent_f2p = mute(True)(fields2pelican)
+
+        with temporary_folder() as temp:
+
+            rst_files = (r(f) for f in silent_f2p(posts, 'markdown', temp))
+            self.assertTrue(any('<iframe' in rst for rst in rst_files))
+            rst_files = (r(f) for f in silent_f2p(posts, 'markdown', temp,
+                         strip_raw=True))
+            self.assertFalse(any('<iframe' in rst for rst in rst_files))
+            # no effect in rst
+            rst_files = (r(f) for f in silent_f2p(posts, 'rst', temp))
+            self.assertFalse(any('<iframe' in rst for rst in rst_files))
+            rst_files = (r(f) for f in silent_f2p(posts, 'rst', temp,
+                         strip_raw=True))
+            self.assertFalse(any('<iframe' in rst for rst in rst_files))

tests/test_readers.py

@@ -62,8 +62,10 @@ class RstReaderTest(unittest.TestCase):
         except ImportError:
             return unittest.skip('need the typogrify distribution')
 
+
 class MdReaderTest(unittest.TestCase):
 
+    @unittest.skipUnless(readers.Markdown, "markdown isn't installed")
     def test_article_with_md_extention(self):
         # test to ensure the md extension is being processed by the correct reader
         reader = readers.MarkdownReader({})
@@ -74,6 +76,7 @@ class MdReaderTest(unittest.TestCase):
         
         self.assertEqual(content, expected)
 
+    @unittest.skipUnless(readers.Markdown, "markdown isn't installed")
     def test_article_with_mkd_extension(self):
         # test to ensure the mkd extension is being processed by the correct reader
         reader = readers.MarkdownReader({})

tox.ini

@@ -12,3 +12,4 @@ deps =
     unittest2
     mock
     Markdown
+    BeautifulSoup