github/pelican
1
0
Fork 1
mirror of https://github.com/getpelican/pelican.git synced 2025-10-15 20:28:56 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Make StaticGenerator skip content sources. Refs #1019.

Browse source 
This change partially addresses issue #1019, by teaching Pelican to distinguish
between static files and content source files. A user can now safely add the
same directory to both STATIC_PATHS and PAGE_PATHS (or ARTICLE_PATHS). Pelican
will then process the content source files in that directory normally, and
treat the remaining files as static, without copying the raw content source
files to the output directory. (The OUTPUT_SOURCES setting still works.)

In other words, images and markdown/reST files can now safely live together.

To keep those files together in the generated site, STATIC_SAVE_AS and
PAGE_SAVE_AS (or ARTICLE_SAVE_AS) should point to the same output directory.

There are two new configuration settings:

STATIC_EXCLUDES=[]  # This works just like PAGE_EXCLUDES and ARTICLE_EXCLUDES.
STATIC_EXCLUDE_SOURCES=True  # Set this to False to get the old behavior.

Two small but noteworthy internal changes:

StaticGenerator now runs after all the other generators. This allows it to see
which files are meant to be processed by other generators, and avoid them.

Generators now include files that they fail to process (e.g. those with missing
mandatory metadata) along with all the other paths in context['filenames'].
This allows such files to be excluded from StaticGenerator's file list, so they
won't end up accidentally published. Since these files have no Content object,
their value in context['filenames'] is None. The code that uses that dict has
been updated accordingly.
This commit is contained in:
Forest 2014-10-18 13:11:59 -07:00
commit 48f4f0850d
10 changed files with 131 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

29
pelican/generators.py
View file

@ -141,9 +141,27 @@ class Generator(object):
return files
def add_source_path(self, content):
"""Record a source file path that a Generator found and processed.
Store a reference to its Content object, for url lookups later.
"""
location = content.get_relative_source_path()
self.context['filenames'][location] = content
def _add_failed_source_path(self, path):
"""Record a source file path that a Generator failed to process.
(For example, one that was missing mandatory metadata.)
The path argument is expected to be relative to self.path.
"""
self.context['filenames'][os.path.normpath(path)] = None
def _is_potential_source_path(self, path):
"""Return True if path was supposed to be used as a source file.
(This includes all source files that have been found by generators
before this method is called, even if they failed to process.)
The path argument is expected to be relative to self.path.
"""
return os.path.normpath(path) in self.context['filenames']
def _update_context(self, items):
"""Update the context with the given items from the currrent
processor.
@ -477,9 +495,11 @@ class ArticlesGenerator(CachingGenerator):
except Exception as e:
logger.error('Could not process %s\n%s', f, e,
exc_info=self.settings.get('DEBUG', False))
self._add_failed_source_path(f)
continue
if not is_valid_content(article, f):
self._add_failed_source_path(f)
continue
self.cache_data(f, article)
@ -602,9 +622,11 @@ class PagesGenerator(CachingGenerator):
except Exception as e:
logger.error('Could not process %s\n%s', f, e,
exc_info=self.settings.get('DEBUG', False))
self._add_failed_source_path(f)
continue
if not is_valid_content(page, f):
self._add_failed_source_path(f)
continue
self.cache_data(f, page)
@ -663,7 +685,14 @@ class StaticGenerator(Generator):
def generate_context(self):
self.staticfiles = []
for f in self.get_files(self.settings['STATIC_PATHS'],
exclude=self.settings['STATIC_EXCLUDES'],
extensions=False):
# skip content source files unless the user explicitly wants them
if self.settings['STATIC_EXCLUDE_SOURCES']:
if self._is_potential_source_path(f):
continue
static = self.readers.read_file(
base_path=self.path, path=f, content_class=Static,
fmt='static', context=self.context,