Also introduced a mechanism whereby table counts are calculated against a time limit
but immutable databases have their table counts calculated on server startup.
Thanks @russss!
* Add register_output_renderer hook
This changeset refactors out the JSON renderer and then adds a hook and
dispatcher system to allow custom output renderers to be registered.
The CSV output renderer is untouched because supporting streaming
renderers through this system would be significantly more complex, and
probably not worthwhile.
We can't simply allow hooks to be called at request time because we need
a list of supported file extensions when the request is being routed in
order to resolve ambiguous database/table names. So, renderers need to
be registered at startup.
I've tried to make this API independent of Sanic's request/response
objects so that this can remain stable during the switch to ASGI. I'm
using dictionaries to keep it simple and to make adding additional
options in the future easy.
Fixes#440
Prior to this commit Datasette would calculate the content hash of every
database and redirect to a URL containing that hash, like so:
https://v0-27.datasette.io/fixtures => https://v0-27.datasette.io/fixtures-dd88475
This assumed that all databases were opened in immutable mode and were not
expected to change.
This will be changing as a result of #419 - so this commit takes the first step
in implementing that change by changing this default behaviour. Datasette will
now only redirect hash-free URLs under two circumstances:
* The new `hash_urls` config option is set to true (it defaults to false).
* The user passes `?_hash=1` in the URL
If you start Datasette with no files, it will connect to :memory: instead.
When starting it with files you can add --memory to also get a :memory: database.
Means `datasette publish heroku` can work under Travis, unlike this failure:
https://travis-ci.org/simonw/fivethirtyeight-datasette/builds/488047550
```
2.25s$ datasette publish heroku fivethirtyeight.db -m metadata.json -n fivethirtyeight-datasette
tar: unrecognized option '--exclude-vcs-ignores'
Try 'tar --help' or 'tar --usage' for more information.
▸ Command failed: tar cz -C /tmp/tmpuaxm7i8f --exclude-vcs-ignores --exclude
▸ .git --exclude .gitmodules . >
▸ /tmp/f49440e0-1bf3-4d3f-9eb0-fbc2967d1fd4.tar.gz
▸ tar: unrecognized option '--exclude-vcs-ignores'
▸ Try 'tar --help' or 'tar --usage' for more information.
▸
The command "datasette publish heroku fivethirtyeight.db -m metadata.json -n fivethirtyeight-datasette" exited with 0.
```
The fix for that issue is to call the heroku command like this:
heroku builds:create -a app_name --include-vcs-ignore
The extra_css_urls and extra_js_urls hooks now take additional optional
parameters.
Also refactored them out of the Datasette class and into RenderMixin.
Plus improved plugin documentation to explicitly list parameters.
Also removed xfail from test_view_classes_are_documented, so any future *View
classes that are added without documentation will cause the tests to fail.
More documentation unit tests. These ones check that every single **View class
imported into the datasette/app.py module are covered by our documentation.
Just one problem: they aren't documented yet. So I'm using the xfail pytest
decorator to mark these tests as allowed-to-fail. When you run the test suite
you now get a report of how many views still need to be documented, but it
doesn't fail the tests.
The output looks something like this:
$ pytest tests/test_docs.py
collected 31 items
tests/test_docs.py ..........................XXXxx. [100%]
============ 26 passed, 2 xfailed, 3 xpassed in 1.06 seconds ============
Once I have documented all the views I will remove the xfail so any future
views that are added without documentation will cause a test failure.
We can detect that a view is documented by looking for ReST label in the docs,
for example:
.. _IndexView:
Some view classes can be used to power multiple URLs - the JsonDataView class
for example is used to power /-/metadata and /-/config and /-/plugins
In this case, the second part of the label can indicate the variety of page, e.g:
.. _JsonDataView_metadata:
The test will pass as long as there is at least one label that starts with
_JsonDataView.
This change introduces a new plugin hook, publish_subcommand, which can be
used to implement new subcommands for the "datasette publish" command family.
I've used this new hook to refactor out the "publish now" and "publish heroku"
implementations into separate modules. I've also added unit tests for these
two publishers, mocking the subprocess.call and subprocess.check_output
functions.
As part of this, I introduced a mechanism for loading default plugins. These
are defined in the new "default_plugins" list inside datasette/app.py
Closes#217 (Plugin support for datasette publish)
Closes#348 (Unit tests for "datasette publish")
Refs #14, #59, #102, #103, #146, #236, #347
Unit tests now check that docs/*.txt help examples are all up-to-date.
I ran into a problem here in that the terminal_width needed to be more
accurately defined - so I replaced update-docs-help.sh with update-docs-
help.py which hard-codes the terminal width.
