1
0
Fork 0
forked from github/pelican
pelican-theme/docs/contribute.rst
Deniz Turgut ed3209888a Refactor logging handling
Old system was using manual string formatting for log messages.
This caused issues with common operations like exception logging
because often they need to be handled differently for Py2/Py3
compatibility. In order to unify the effort:

 - All logging is changed to `logging.level(msg, arg1, arg2)` style.
 - A `SafeLogger` is implemented to auto-decode exceptions properly
in the args (ref #1403).
 - Custom formatters were overriding useful logging functionality
like traceback outputing (ref #1402). They are refactored to be
more transparent. Traceback information is provided in `--debug`
mode for `read_file` errors in generators.
 - Formatters will now auto-format multiline log messages in order
to make them look related. Similarly, traceback will be formatted in
the same fashion.
 - `pelican.log.LimitFilter` was (ab)using logging message which
would result in awkward syntax for argumented logging style. This
functionality is moved to `extra` keyword argument.
 - Levels for errors that would result skipping a file (`read_file`)
changed from `warning` to `error` in order to make them stand out
among other logs.
 - Small consistency changes to log messages (i.e. changing all
to start with an uppercase letter) and quality-of-life improvements
(some log messages were dumping raw object information).
2014-07-22 12:39:39 -04:00

206 lines
7.5 KiB
ReStructuredText

Contributing and feedback guidelines
####################################
There are many ways to contribute to Pelican. You can improve the
documentation, add missing features, and fix bugs (or just report them). You
can also help out by reviewing and commenting on
`existing issues <https://github.com/getpelican/pelican/issues>`_.
Don't hesitate to fork Pelican and submit an issue or pull request on GitHub.
When doing so, please adhere to the following guidelines.
.. include:: ../CONTRIBUTING.rst
Setting up the development environment
======================================
While there are many ways to set up one's development environment, following
is a method that uses `virtualenv <http://www.virtualenv.org/>`_. If you don't
have ``virtualenv`` installed, you can install it via::
$ pip install virtualenv
Virtual environments allow you to work on Python projects which are isolated
from one another so you can use different packages (and package versions) with
different projects.
To create and activate a virtual environment, use the following syntax::
$ virtualenv ~/virtualenvs/pelican
$ cd ~/virtualenvs/pelican
$ . bin/activate
To clone the Pelican source::
$ git clone https://github.com/getpelican/pelican.git src/pelican
To install the development dependencies::
$ cd src/pelican
$ pip install -r dev_requirements.txt
To install Pelican and its dependencies::
$ python setup.py develop
Or using ``pip``::
$ pip install -e .
Building the docs
=================
If you make changes to the documentation, you should preview your changes
before committing them::
$ pip install sphinx
$ cd src/pelican/docs
$ make html
Open ``_build/html/index.html`` in your browser to preview the documentation.
Running the test suite
======================
Each time you add a feature, there are two things to do regarding tests:
check that the existing tests pass, and add tests for the new feature
or bugfix.
The tests live in ``pelican/tests`` and you can run them using the
"discover" feature of ``unittest``::
$ python -m unittest discover
After making your changes and running the tests, you may see a test failure
mentioning that "some generated files differ from the expected functional tests
output." If you have made changes that affect the HTML output generated by
Pelican, and the changes to that output are expected and deemed correct given
the nature of your changes, then you should update the output used by the
functional tests. To do so, you can use the following two commands::
$ LC_ALL=en_US.utf8 pelican -o pelican/tests/output/custom/ \
-s samples/pelican.conf.py samples/content/
$ LC_ALL=fr_FR.utf8 pelican -o pelican/tests/output/custom_locale/ \
-s samples/pelican.conf_FR.py samples/content/
$ LC_ALL=en_US.utf8 pelican -o pelican/tests/output/basic/ \
samples/content/
Testing on Python 2 and 3
-------------------------
Testing on Python 3 currently requires some extra steps: installing
Python 3-compatible versions of dependent packages and plugins.
Tox_ is a useful tool to run tests on both versions. It will install the
Python 3-compatible version of dependent packages.
.. _Tox: http://testrun.org/tox/latest/
Python 3 development tips
=========================
Here are some tips that may be useful when doing some code for both Python 2.7
and Python 3 at the same time:
- Assume every string and literal is unicode (import unicode_literals):
- Do not use prefix ``u'``.
- Do not encode/decode strings in the middle of sth. Follow the code to the
source (or target) of a string and encode/decode at the first/last possible
point.
- In other words, write your functions to expect and to return unicode.
- Encode/decode strings if e.g. the source is a Python function that is known
to handle this badly, e.g. strftime() in Python 2.
- Use new syntax: print function, "except ... *as* e" (not comma) etc.
- Refactor method calls like ``dict.iteritems()``, ``xrange()`` etc. in a way
that runs without code change in both Python versions.
- Do not use magic method ``__unicode()__`` in new classes. Use only ``__str()__``
and decorate the class with ``@python_2_unicode_compatible``.
- Do not start int literals with a zero. This is a syntax error in Py3k.
- Unfortunately I did not find an octal notation that is valid in both
Pythons. Use decimal instead.
- use six, e.g.:
- ``isinstance(.., basestring) -> isinstance(.., six.string_types)``
- ``isinstance(.., unicode) -> isinstance(.., six.text_type)``
- ``setlocale()`` in Python 2 bails when we give the locale name as unicode,
and since we are using ``from __future__ import unicode_literals``, we do
that everywhere! As a workaround, I enclosed the localename with ``str()``;
in Python 2 this casts the name to a byte string, in Python 3 this should do
nothing, because the locale name already had been unicode.
- Kept range() almost everywhere as-is (2to3 suggests list(range())), just
changed it where I felt necessary.
- Changed xrange() back to range(), so it is valid in both Python versions.
Logging tips
============
Try to use logging with appropriate levels.
For logging messages that are not repeated, use the usual Python way::
# at top of file
import logging
logger = logging.getLogger(__name__)
# when needed
logger.warning("A warning with %s formatting", arg_to_be_formatted)
Do not format log messages yourself. Use ``%s`` formatting in messages and pass
arguments to logger. This is important, because Pelican logger will preprocess
some arguments (like Exceptions) for Py2/Py3 compatibility.
Limiting extraneous log messages
--------------------------------
If the log message can occur several times, you may want to limit the log to
prevent flooding. In order to do that, use the ``extra`` keyword argument for
the logging message in the following format::
logger.warning("A warning with %s formatting", arg_to_be_formatted,
extra={'limit_msg': 'A generic message for too many warnings'})
Optionally, you can also set ``'limit_args'`` as a tuple of arguments in
``extra`` dict if your generic message needs formatting.
Limit is set to ``5``, i.e, first four logs with the same ``'limit_msg'`` are
outputted normally but the fifth one will be logged using
``'limit_msg'`` (and ``'limit_args'`` if present). After the fifth,
corresponding log messages will be ignored.
For example, if you want to log missing resources, use the following code::
for resource in resources:
if resource.is_missing:
logger.warning(
'The resource %s is missing', resource.name,
extra={'limit_msg': 'Other resources were missing'})
The log messages will be displayed as follows::
WARNING: The resource prettiest_cat.jpg is missing
WARNING: The resource best_cat_ever.jpg is missing
WARNING: The resource cutest_cat.jpg is missing
WARNING: The resource lolcat.jpg is missing
WARNING: Other resources were missing
Outputting traceback in the logs
--------------------------------
If you're logging inside an ``except`` block, you may want to provide the
traceback information as well. You can do that by setting ``exc_info`` keyword
argument to ``True`` during logging. However, doing so by default can be
undesired because tracebacks are long and can be confusing to regular users.
Try to limit them to ``--debug`` mode like the following::
try:
some_action()
except Exception as e:
logger.error('Exception occured: %s', e,
exc_info=settings.get('DEBUG', False))