Merge branch 'pr/558'

Conflicts:
	docs/settings.rst
	pelican/signals.py
This commit is contained in:
Bruno Binet 2012-11-23 15:32:17 +01:00
commit 801bc755b5
16 changed files with 298 additions and 244 deletions

View file

@ -2,10 +2,15 @@ language: python
python:
- "2.6"
- "2.7"
before_install:
- sudo apt-get update -qq
- sudo apt-get install -qq ruby-sass
install:
- pip install nose unittest2 mock --use-mirrors
- pip install . --use-mirrors
- pip install Markdown
- pip install webassets
- pip install cssmin
script: nosetests -s tests
notifications:
irc:

View file

@ -27,7 +27,7 @@ Pelican currently supports:
* Publication of articles in multiple languages
* Atom/RSS feeds
* Code syntax highlighting
* Compilation of `LESS CSS`_ (optional)
* Asset management with `webassets`_ (optional)
* Import from WordPress, Dotclear, or RSS feeds
* Integration with external tools: Twitter, Google Analytics, etc. (optional)
@ -62,9 +62,9 @@ client handy, use the webchat_ for quick feedback.
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
.. _Markdown: http://daringfireball.net/projects/markdown/
.. _Jinja2: http://jinja.pocoo.org/
.. _`LESS CSS`: http://lesscss.org/
.. _`Pelican documentation`: http://docs.getpelican.com/latest/
.. _`Pelican's internals`: http://docs.getpelican.com/en/latest/internals.html
.. _`#pelican on Freenode`: irc://irc.freenode.net/pelican
.. _webchat: http://webchat.freenode.net/?channels=pelican&uio=d4
.. _contribute: http://docs.getpelican.com/en/latest/contribute.html
.. _webassets: https://github.com/miracle2k/webassets

View file

@ -23,7 +23,7 @@ Pelican currently supports:
* Publication of articles in multiple languages
* Atom/RSS feeds
* Code syntax highlighting
* Compilation of `LESS CSS`_ (optional)
* Asset management with `webassets`_ (optional)
* Import from WordPress, Dotclear, or RSS feeds
* Integration with external tools: Twitter, Google Analytics, etc. (optional)
@ -75,8 +75,8 @@ A French version of the documentation is available at :doc:`fr/index`.
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
.. _Markdown: http://daringfireball.net/projects/markdown/
.. _Jinja2: http://jinja.pocoo.org/
.. _`LESS CSS`: http://lesscss.org/
.. _`Pelican documentation`: http://docs.getpelican.com/latest/
.. _`Pelican's internals`: http://docs.getpelican.com/en/latest/internals.html
.. _`#pelican on Freenode`: irc://irc.freenode.net/pelican
.. _webchat: http://webchat.freenode.net/?channels=pelican&uio=d4
.. _webassets: https://github.com/miracle2k/webassets

View file

@ -62,6 +62,7 @@ finalized pelican object invoked after al
usefull for custom post processing actions, such as:
- minifying js/css assets.
- notify/ping search engines with an updated sitemap.
generator_init generator invoked in the Generator.__init__
article_generate_context article_generator, metadata
article_generate_preread article_generator invoked before a article is read in ArticlesGenerator.generate_context;
use if code needs to do something before every article is parsed
@ -101,6 +102,7 @@ List of plugins
The following plugins are currently included with Pelican under ``pelican.plugins``:
* `Asset management`_
* `GitHub activity`_
* `Global license`_
* `Gravatar`_
@ -116,6 +118,79 @@ Ideas for plugins that haven't been written yet:
Plugin descriptions
===================
Asset management
----------------
This plugin allows you to use the `webassets`_ module to manage assets such as
CSS and JS files. The module must first be installed::
pip install webassets
The `webassets` module allows you to perform a number of useful asset management
functions, including:
* CSS minifier (`cssmin`, `yuicompressor`, ...)
* CSS compiler (`less`, `sass`, ...)
* JS minifier (`uglifyjs`, `yuicompressor`, `closure`, ...)
Others filters include gzip compression, integration of images in CSS via data
URIs, and more. `webassets` can also append a version identifier to your asset
URL to convince browsers to download new versions of your assets when you use
far-future expires headers. Please refer to the `webassets documentation`_ for
more information.
When using with Pelican, `webassets` is configured to process assets in the
``OUTPUT_PATH/theme`` directory. You can use `webassets` in your templates by
including one or more template tags. The jinja variable ``{{ ASSET_URL }}`` to
use in the templates is configured to be relative to the ``theme/`` url.
Hence, it must be used with the ``{{ SITEURL }}`` variable which allows to
have relative urls. For example...
.. code-block:: jinja
{% assets filters="cssmin", output="css/style.min.css", "css/inuit.css", "css/pygment-monokai.css", "css/main.css" %}
<link rel="stylesheet" href="{{ SITEURL }}/{{ ASSET_URL }}">
{% endassets %}
... will produce a minified css file with a version identifier:
.. code-block:: html
<link href="http://{SITEURL}/theme/css/style.min.css?b3a7c807" rel="stylesheet">
These filters can be combined. Here is an example that uses the SASS compiler
and minifies the output:
.. code-block:: jinja
{% assets filters="sass,cssmin", output="css/style.min.css", "css/style.scss" %}
<link rel="stylesheet" href="{{ SITEURL }}/{{ ASSET_URL }}">
{% endassets %}
Another example for Javascript:
.. code-block:: jinja
{% assets filters="uglifyjs,gzip", output="js/packed.js", "js/jquery.js", "js/base.js", "js/widgets.js" %}
<script src="{{ SITEURL }}/{{ ASSET_URL }}"></script>
{% endassets %}
The above will produce a minified and gzipped JS file:
.. code-block:: html
<script src="http://{SITEURL}/theme/js/packed.js?00703b9d"></script>
Pelican's debug mode is propagated to `webassets` to disable asset packaging
and instead work with the uncompressed assets. However, this also means that
the LESS and SASS files are not compiled. This should be fixed in a future
version of `webassets` (cf. the related `bug report
<https://github.com/getpelican/pelican/issues/481>`_).
.. _webassets: https://github.com/miracle2k/webassets
.. _webassets documentation: http://webassets.readthedocs.org/en/latest/builtin_filters.html
GitHub activity
---------------

View file

@ -105,9 +105,6 @@ Setting name (default value) What doe
incorporated into the generated HTML via the `Typogrify
<http://static.mintchaos.com/projects/typogrify/>`_
library, which can be installed via: ``pip install typogrify``
`LESS_GENERATOR` (``FALSE``) Set to True or complete path to `lessc` (if not
found in system PATH) to enable compiling less
css files. Requires installation of `less css`_.
`DIRECT_TEMPLATES` (``('index', 'tags', 'categories', 'archives')``) List of templates that are used directly to render
content. Typically direct templates are used to generate
index pages for collections of content e.g. tags and
@ -128,8 +125,6 @@ Setting name (default value) What doe
.. [#] Default is the system locale.
.. _less css: http://lesscss.org/
URL settings
------------
@ -436,7 +431,6 @@ Setting name (default value) What does it do?
value is `static`, but if your theme has
other static paths, you can put them here.
`CSS_FILE` (``'main.css'``) Specify the CSS file you want to load.
`WEBASSETS` (``False``) Asset management with `webassets` (see below)
================================================ =====================================================
@ -497,75 +491,6 @@ adding the following to your configuration::
CSS_FILE = "wide.css"
Asset management
----------------
The `WEBASSETS` setting allows you to use the `webassets`_ module to manage
assets such as CSS and JS files. The module must first be installed::
pip install webassets
The `webassets` module allows you to perform a number of useful asset management
functions, including:
* CSS minifier (`cssmin`, `yuicompressor`, ...)
* CSS compiler (`less`, `sass`, ...)
* JS minifier (`uglifyjs`, `yuicompressor`, `closure`, ...)
Others filters include gzip compression, integration of images in CSS via data
URIs, and more. `webassets` can also append a version identifier to your asset
URL to convince browsers to download new versions of your assets when you use
far-future expires headers. Please refer to the `webassets documentation`_ for
more information.
When using with Pelican, `webassets` is configured to process assets in the
``OUTPUT_PATH/theme`` directory. You can use `webassets` in your templates by
including one or more template tags. For example...
.. code-block:: jinja
{% assets filters="cssmin", output="css/style.min.css", "css/inuit.css", "css/pygment-monokai.css", "css/main.css" %}
<link rel="stylesheet" href="{{ ASSET_URL }}">
{% endassets %}
... will produce a minified css file with a version identifier:
.. code-block:: html
<link href="http://{SITEURL}/theme/css/style.min.css?b3a7c807" rel="stylesheet">
These filters can be combined. Here is an example that uses the SASS compiler
and minifies the output:
.. code-block:: jinja
{% assets filters="sass,cssmin", output="css/style.min.css", "css/style.scss" %}
<link rel="stylesheet" href="{{ ASSET_URL }}">
{% endassets %}
Another example for Javascript:
.. code-block:: jinja
{% assets filters="uglifyjs,gzip", output="js/packed.js", "js/jquery.js", "js/base.js", "js/widgets.js" %}
<script src="{{ ASSET_URL }}"></script>
{% endassets %}
The above will produce a minified and gzipped JS file:
.. code-block:: html
<script src="http://{SITEURL}/theme/js/packed.js?00703b9d"></script>
Pelican's debug mode is propagated to `webassets` to disable asset packaging
and instead work with the uncompressed assets. However, this also means that
the LESS and SASS files are not compiled. This should be fixed in a future
version of `webassets` (cf. the related `bug report
<https://github.com/getpelican/pelican/issues/481>`_).
.. _webassets: https://github.com/miracle2k/webassets
.. _webassets documentation: http://webassets.readthedocs.org/en/latest/builtin_filters.html
Example settings
================

View file

@ -9,8 +9,7 @@ from pelican import signals
from pelican.generators import (ArticlesGenerator, PagesGenerator,
StaticGenerator, PdfGenerator,
LessCSSGenerator, SourceFileGenerator,
TemplatePagesGenerator)
SourceFileGenerator, TemplatePagesGenerator)
from pelican.log import init
from pelican.settings import read_settings
from pelican.utils import (clean_output_dir, files_changed, file_changed,
@ -159,11 +158,6 @@ class Pelican(object):
writer = self.get_writer()
# pass the assets environment to the generators
if self.settings['WEBASSETS']:
generators[1].env.assets_environment = generators[0].assets_env
generators[2].env.assets_environment = generators[0].assets_env
for p in generators:
if hasattr(p, 'generate_output'):
p.generate_output(writer)
@ -177,8 +171,6 @@ class Pelican(object):
generators.append(TemplatePagesGenerator)
if self.settings['PDF_GENERATOR']:
generators.append(PdfGenerator)
if self.settings['LESS_GENERATOR']: # can be True or PATH to lessc
generators.append(LessCSSGenerator)
if self.settings['OUTPUT_SOURCES']:
generators.append(SourceFileGenerator)

View file

@ -62,6 +62,8 @@ class Generator(object):
custom_filters = self.settings.get('JINJA_FILTERS', {})
self.env.filters.update(custom_filters)
signals.generator_init.send(self)
def get_template(self, name):
"""Return the template by name.
Use self.theme to get the templates to use, and return a list of
@ -477,37 +479,6 @@ class StaticGenerator(Generator):
copy(path, source, os.path.join(output_path, destination),
final_path, overwrite=True)
def generate_context(self):
if self.settings['WEBASSETS']:
from webassets import Environment as AssetsEnvironment
# Define the assets environment that will be passed to the
# generators. The StaticGenerator must then be run first to have
# the assets in the output_path before generating the templates.
# Let ASSET_URL honor Pelican's RELATIVE_URLS setting.
# Hint for templates:
# Current version of webassets seem to remove any relative
# paths at the beginning of the URL. So, if RELATIVE_URLS
# is on, ASSET_URL will start with 'theme/', regardless if we
# set assets_url here to './theme/' or to 'theme/'.
# XXX However, this breaks the ASSET_URL if user navigates to
# a sub-URL, e.g. if he clicks on a category. To workaround this
# issue, I use
# <link rel="stylesheet" href="{{ SITEURL }}/{{ ASSET_URL }}">
# instead of
# <link rel="stylesheet" href="{{ ASSET_URL }}">
if self.settings.get('RELATIVE_URLS'):
assets_url = './theme/'
else:
assets_url = self.settings['SITEURL'] + '/theme/'
assets_src = os.path.join(self.output_path, 'theme')
self.assets_env = AssetsEnvironment(assets_src, assets_url)
if logging.getLevelName(logger.getEffectiveLevel()) == "DEBUG":
self.assets_env.debug = True
def generate_output(self, writer):
self._copy_paths(self.settings['STATIC_PATHS'], self.path,
@ -583,49 +554,3 @@ class SourceFileGenerator(Generator):
logger.info(u' Generating source files...')
for object in chain(self.context['articles'], self.context['pages']):
self._create_source(object, self.output_path)
class LessCSSGenerator(Generator):
"""Compile less css files."""
def _compile(self, less_file, source_dir, dest_dir):
base = os.path.relpath(less_file, source_dir)
target = os.path.splitext(
os.path.join(dest_dir, base))[0] + '.css'
target_dir = os.path.dirname(target)
if not os.path.exists(target_dir):
try:
os.makedirs(target_dir)
except OSError:
logger.error("Couldn't create the less css output folder in " +
target_dir)
subprocess.call([self._lessc, less_file, target])
logger.info(u' [ok] compiled %s' % base)
def generate_output(self, writer=None):
logger.info(u' Compiling less css')
# store out compiler here, so it won't be evaulted on each run of
# _compile
lg = self.settings['LESS_GENERATOR']
self._lessc = lg if isinstance(lg, basestring) else 'lessc'
# walk static paths
for static_path in self.settings['STATIC_PATHS']:
for f in self.get_files(
os.path.join(self.path, static_path),
extensions=['less']):
self._compile(f, self.path, self.output_path)
# walk theme static paths
theme_output_path = os.path.join(self.output_path, 'theme')
for static_path in self.settings['THEME_STATIC_PATHS']:
theme_static_path = os.path.join(self.theme, static_path)
for f in self.get_files(
theme_static_path,
extensions=['less']):
self._compile(f, theme_static_path, theme_output_path)

48
pelican/plugins/assets.py Normal file
View file

@ -0,0 +1,48 @@
# -*- coding: utf-8 -*-
"""
Asset management plugin for Pelican
===================================
This plugin allows you to use the `webassets`_ module to manage assets such as
CSS and JS files.
The ASSET_URL is set to a relative url to honor Pelican's RELATIVE_URLS
setting. This requires the use of SITEURL in the templates::
<link rel="stylesheet" href="{{ SITEURL }}/{{ ASSET_URL }}">
.. _webassets: https://webassets.readthedocs.org/
"""
import os
import logging
from pelican import signals
from webassets import Environment
from webassets.ext.jinja2 import AssetsExtension
def add_jinja2_ext(pelican):
"""Add Webassets to Jinja2 extensions in Pelican settings."""
pelican.settings['JINJA_EXTENSIONS'].append(AssetsExtension)
def create_assets_env(generator):
"""Define the assets environment and pass it to the generator."""
assets_url = 'theme/'
assets_src = os.path.join(generator.output_path, 'theme')
generator.env.assets_environment = Environment(assets_src, assets_url)
logger = logging.getLogger(__name__)
if logging.getLevelName(logger.getEffectiveLevel()) == "DEBUG":
generator.env.assets_environment.debug = True
def register():
"""Plugin registration."""
signals.initialized.connect(add_jinja2_ext)
signals.generator_init.connect(create_assets_env)

View file

@ -75,9 +75,7 @@ _DEFAULT_CONFIG = {'PATH': '.',
'DEFAULT_STATUS': 'published',
'ARTICLE_PERMALINK_STRUCTURE': '',
'TYPOGRIFY': False,
'LESS_GENERATOR': False,
'SUMMARY_MAX_LENGTH': 50,
'WEBASSETS': False,
'PLUGINS': [],
'MARKDOWN_EXTENSIONS': ['toc', ],
'TEMPLATE_PAGES': {}
@ -133,7 +131,7 @@ def configure_settings(settings):
"""
if not 'PATH' in settings or not os.path.isdir(settings['PATH']):
raise Exception('You need to specify a path containing the content'
' (see pelican --help for more information)')
' (see pelican --help for more information)')
# find the theme in pelican.theme if the given one does not exists
if not os.path.isdir(settings['THEME']):
@ -143,7 +141,7 @@ def configure_settings(settings):
settings['THEME'] = theme_path
else:
raise Exception("Impossible to find the theme %s"
% settings['THEME'])
% settings['THEME'])
# if locales is not a list, make it one
locales = settings['LOCALE']
@ -197,13 +195,9 @@ def configure_settings(settings):
"http://docs.notmyidea.org/alexis/pelican/settings.html#timezone "
"for more information")
if 'WEBASSETS' in settings and settings['WEBASSETS'] is not False:
try:
from webassets.ext.jinja2 import AssetsExtension
settings['JINJA_EXTENSIONS'].append(AssetsExtension)
except ImportError:
logger.warn("You must install the webassets module to use WEBASSETS.")
settings['WEBASSETS'] = False
if 'LESS_GENERATOR' in settings:
logger.warn("The LESS_GENERATOR setting has been removed in favor "
"of the Webassets plugin")
if 'OUTPUT_SOURCES_EXTENSION' in settings:
if not isinstance(settings['OUTPUT_SOURCES_EXTENSION'], str):

View file

@ -3,6 +3,7 @@ from blinker import signal
initialized = signal('pelican_initialized')
finalized = signal('pelican_finalized')
article_generate_preread = signal('article_generate_preread')
generator_init = signal('generator_init')
article_generate_context = signal('article_generate_context')
article_generator_init = signal('article_generator_init')
article_generator_finalized = signal('article_generate_finalized')

View file

@ -3,11 +3,12 @@ __all__ = [
'unittest',
]
import os
import subprocess
import re
import sys
import cStringIO
import importlib
import os
import re
import subprocess
import sys
from functools import wraps
from contextlib import contextmanager
@ -138,3 +139,14 @@ def skipIfNoExecutable(executable):
return unittest.skip('{0} executable not found'.format(executable))
return lambda func: func
def module_exists(module_name):
"""Test if a module is importable."""
try:
importlib.import_module(module_name)
except ImportError:
return False
else:
return True

View file

@ -2,15 +2,16 @@
from mock import MagicMock
import os
import re
from codecs import open
from tempfile import mkdtemp
from shutil import rmtree
from pelican.generators import ArticlesGenerator, LessCSSGenerator, \
PagesGenerator, TemplatePagesGenerator
from pelican.generators import ArticlesGenerator, PagesGenerator, \
TemplatePagesGenerator
from pelican.writers import Writer
from pelican.settings import _DEFAULT_CONFIG
from .support import unittest, skipIfNoExecutable
from .support import unittest
CUR_DIR = os.path.dirname(__file__)
@ -238,55 +239,3 @@ class TestTemplatePagesGenerator(unittest.TestCase):
# output content is correct
with open(output_filename, 'r') as output_file:
self.assertEquals(output_file.read(), 'foo: bar')
class TestLessCSSGenerator(unittest.TestCase):
LESS_CONTENT = """
@color: #4D926F;
#header {
color: @color;
}
h2 {
color: @color;
}
"""
def setUp(self):
self.temp_content = mkdtemp()
self.temp_output = mkdtemp()
def tearDown(self):
rmtree(self.temp_content)
rmtree(self.temp_output)
@skipIfNoExecutable('lessc')
def test_less_compiler(self):
settings = _DEFAULT_CONFIG.copy()
settings['STATIC_PATHS'] = ['static']
settings['LESS_GENERATOR'] = True
generator = LessCSSGenerator(None, settings, self.temp_content,
_DEFAULT_CONFIG['THEME'], self.temp_output, None)
# create a dummy less file
less_dir = os.path.join(self.temp_content, 'static', 'css')
less_filename = os.path.join(less_dir, 'test.less')
less_output = os.path.join(self.temp_output, 'static', 'css',
'test.css')
os.makedirs(less_dir)
with open(less_filename, 'w') as less_file:
less_file.write(self.LESS_CONTENT)
generator.generate_output()
# we have the file ?
self.assertTrue(os.path.exists(less_output))
# was it compiled ?
self.assertIsNotNone(re.search(r'^\s+color:\s*#4D926F;$',
open(less_output).read(), re.MULTILINE | re.IGNORECASE))

101
tests/test_webassets.py Normal file
View file

@ -0,0 +1,101 @@
# -*- coding: utf-8 -*-
import hashlib
import os
from codecs import open
from tempfile import mkdtemp
from shutil import rmtree
from pelican import Pelican
from pelican.settings import read_settings
from .support import unittest, skipIfNoExecutable, module_exists
CUR_DIR = os.path.dirname(__file__)
THEME_DIR = os.path.join(CUR_DIR, 'themes', 'assets')
CSS_REF = open(os.path.join(THEME_DIR, 'static', 'css',
'style.min.css')).read()
CSS_HASH = hashlib.md5(CSS_REF).hexdigest()[0:8]
@unittest.skipUnless(module_exists('webassets'), "webassets isn't installed")
@skipIfNoExecutable(['scss', '-v'])
@skipIfNoExecutable(['cssmin', '--version'])
class TestWebAssets(unittest.TestCase):
"""Base class for testing webassets."""
def setUp(self, override=None):
self.temp_path = mkdtemp()
settings = {
'PATH': os.path.join(CUR_DIR, 'content', 'TestCategory'),
'OUTPUT_PATH': self.temp_path,
'PLUGINS': ['pelican.plugins.assets', ],
'THEME': THEME_DIR,
}
if override:
settings.update(override)
self.settings = read_settings(override=settings)
pelican = Pelican(settings=self.settings)
pelican.run()
def tearDown(self):
rmtree(self.temp_path)
def check_link_tag(self, css_file, html_file):
"""Check the presence of `css_file` in `html_file`."""
link_tag = '<link rel="stylesheet" href="{css_file}">'.\
format(css_file=css_file)
html = open(html_file).read()
self.assertRegexpMatches(html, link_tag)
class TestWebAssetsRelativeURLS(TestWebAssets):
"""Test pelican with relative urls."""
def test_jinja2_ext(self):
"""Test that the Jinja2 extension was correctly added."""
from webassets.ext.jinja2 import AssetsExtension
self.assertIn(AssetsExtension, self.settings['JINJA_EXTENSIONS'])
def test_compilation(self):
"""Compare the compiled css with the reference."""
gen_file = os.path.join(self.temp_path, 'theme', 'gen',
'style.{0}.min.css'.format(CSS_HASH))
self.assertTrue(os.path.isfile(gen_file))
css_new = open(gen_file).read()
self.assertEqual(css_new, CSS_REF)
def test_template(self):
"""Look in the output files for the link tag."""
css_file = './theme/gen/style.{0}.min.css'.format(CSS_HASH)
html_files = ['index.html', 'archives.html',
'this-is-an-article-with-category.html']
for f in html_files:
self.check_link_tag(css_file, os.path.join(self.temp_path, f))
self.check_link_tag(
'.././theme/gen/style.{0}.min.css'.format(CSS_HASH),
os.path.join(self.temp_path, 'category/misc.html'))
class TestWebAssetsAbsoluteURLS(TestWebAssets):
"""Test pelican with absolute urls."""
def setUp(self):
TestWebAssets.setUp(self, override={'RELATIVE_URLS': False,
'SITEURL': 'http://localhost'})
def test_absolute_url(self):
"""Look in the output files for the link tag with absolute url."""
css_file = 'http://localhost/theme/gen/style.{0}.min.css'.\
format(CSS_HASH)
html_files = ['index.html', 'archives.html',
'this-is-an-article-with-category.html']
for f in html_files:
self.check_link_tag(css_file, os.path.join(self.temp_path, f))

View file

@ -0,0 +1 @@
body{font:14px/1.5 "Droid Sans",sans-serif;background-color:#e4e4e4;color:#242424}a{color:red}a:hover{color:orange}

View file

@ -0,0 +1,19 @@
/* -*- scss-compile-at-save: nil -*- */
$baseFontFamily : "Droid Sans", sans-serif;
$textColor : #242424;
$bodyBackground : #e4e4e4;
body {
font: 14px/1.5 $baseFontFamily;
background-color: $bodyBackground;
color: $textColor;
}
a {
color: red;
&:hover {
color: orange;
}
}

View file

@ -0,0 +1,7 @@
{% extends "!simple/base.html" %}
{% block head %}
{% assets filters="scss,cssmin", output="gen/style.%(version)s.min.css", "css/style.scss" %}
<link rel="stylesheet" href="{{ SITEURL }}/{{ ASSET_URL }}">
{% endassets %}
{% endblock %}