New 'Testing plugins' page, closes #687

docs/config.rst

@@ -52,7 +52,7 @@ The files that can be included in this directory are as follows. All are optiona
 * ``inspect-data.json`` - the result of running ``datasette inspect`` - any database files listed here will be treated as immutable, so they should not be changed while Datasette is running
 * ``config.json`` - settings that would normally be passed using ``--config`` - here they should be stored as a JSON object of key/value pairs
 * ``templates/`` - a directory containing :ref:`customization_custom_templates`
-* ``plugins/`` - a directory containing plugins, see :ref:`plugins_writing_one_off`
+* ``plugins/`` - a directory containing plugins, see :ref:`writing_plugins_one_off`
 * ``static/`` - a directory containing static files - these will be served from ``/static/filename.txt``, see :ref:`customization_static_files`
 
 Configuration options

docs/contributing.rst

@@ -43,7 +43,7 @@ The next step is to create a virtual environment for your project and use it to
 
 That last line does most of the work: ``pip install -e`` means "install this package in a way that allows me to edit the source code in place". The ``.[test]`` option means "use the setup.py in this directory and install the optional testing dependencies as well".
 
-Once you have done this, you can run the Datasette unit tests from inside your ``datasette/`` directory using `pytest <https://docs.pytest.org/en/latest/>`__ like so::
+Once you have done this, you can run the Datasette unit tests from inside your ``datasette/`` directory using `pytest <https://docs.pytest.org/>`__ like so::
 
     pytest
 

docs/index.rst

@@ -53,6 +53,7 @@ Contents
    plugins
    writing_plugins
    plugin_hooks
+   testing_plugins
    internals
    contributing
    changelog

docs/internals.rst

@@ -182,7 +182,7 @@ This object is an instance of the ``Datasette`` class, passed to many plugin hoo
 ``table`` - None or string
     The table the user is interacting with.
 
-This method lets you read plugin configuration values that were set in ``metadata.json``. See :ref:`plugins_plugin_config` for full details of how this method should be used.
+This method lets you read plugin configuration values that were set in ``metadata.json``. See :ref:`writing_plugins_configuration` for full details of how this method should be used.
 
 .. _datasette_render_template:
 

docs/testing_plugins.rst

@@ -0,0 +1,123 @@
+.. _testing_plugins:
+
+Testing plugins
+===============
+
+We recommend using `pytest <https://docs.pytest.org/>`__ to write automated tests for your plugins.
+
+If you use the template described in :ref:`writing_plugins_cookiecutter` your plugin will start with a single test in your ``tests/`` directory that looks like this:
+
+.. code-block:: python
+
+    from datasette.app import Datasette
+    import pytest
+    import httpx
+
+    @pytest.mark.asyncio
+    async def test_plugin_is_installed():
+        app = Datasette([], memory=True).app()
+        async with httpx.AsyncClient(app=app) as client:
+            response = await client.get("http://localhost/-/plugins.json")
+            assert 200 == response.status_code
+            installed_plugins = {p["name"] for p in response.json()}
+            assert "datasette-plugin-template-demo" in installed_plugins
+
+This test uses the `HTTPX <https://www.python-httpx.org/>`__ Python library to run mock HTTP requests through a fresh instance of Datasette. This is the recommended way to write tests against a Datasette instance.
+
+It also uses the `pytest-asyncio <https://pypi.org/project/pytest-asyncio/>`__ package to add support for ``async def`` test functions running under pytest.
+
+You can install these packages like so::
+
+    pip install pytest pytest-asyncio httpx
+
+If you are building an installable package you can add them as test dependencies to your ``setup.py`` module like this:
+
+.. code-block:: python
+
+    setup(
+        name="datasette-my-plugin",
+        # ...
+        extras_require={
+            "test": ["pytest", "pytest-asyncio", "httpx"]
+        },
+        tests_require=["datasette-my-plugin[test]"],
+    )
+
+You can then install the test dependencies like so::
+
+    pip install -e '.[test]'
+
+Then run the tests using pytest like so::
+
+    pytest
+
+.. _testing_plugins_fixtures:
+
+Using pytest fixtures
+---------------------
+
+`Pytest fixtures <https://docs.pytest.org/en/stable/fixture.html>`__ can be used to create initial testable objects which can then be used by multiple tests.
+
+A common pattern for Datasette plugins is to create a fixture which sets up a temporary test database and wraps it in a Datasette instance.
+
+Here's an example that uses the `sqlite-utils library <https://sqlite-utils.readthedocs.io/en/stable/python-api.html>`__ to populate a temporary test database. It also sets the title of that table using a simulated ``metadata.json`` congiguration:
+
+.. code-block:: python
+
+    from datasette.app import Datasette
+    import httpx
+    import pytest
+    import sqlite_utils
+
+    @pytest.fixture(scope="session")
+    def ds(tmp_path_factory):
+        db_directory = tmp_path_factory.mktemp("dbs")
+        db_path = db_directory / "test.db"
+        db = sqlite_utils.Database(db_path)
+        db["dogs"].insert_all([
+            {"id": 1, "name": "Cleo", "age": 5},
+            {"id": 2, "name": "Pancakes", "age": 4}
+        ], pk="id")
+        ds = Datasette(
+            [db_path],
+            metadata={
+                "databases": {
+                    "test": {
+                        "tables": {
+                            "dogs": {
+                                "title": "Some dogs"
+                            }
+                        }
+                    }
+                }
+            }
+        )
+        return ds
+
+    @pytest.mark.asyncio
+    async def test_example_table_json(ds):
+        async with httpx.AsyncClient(app=ds.app()) as client:
+            response = await client.get("http://localhost/test/dogs.json?_shape=array")
+            assert 200 == response.status_code
+            assert [
+                {"id": 1, "name": "Cleo", "age": 5},
+                {"id": 2, "name": "Pancakes", "age": 4},
+            ] == response.json()
+
+    @pytest.mark.asyncio
+        async def test_example_table_html(ds):
+            async with httpx.AsyncClient(app=ds.app()) as client:
+                response = await client.get("http://localhost/test/dogs")
+                assert ">Some dogs</h1>" in response.text
+
+Here the ``ds()`` function defines the fixture, which is than automatically passed to the two test functions based on pytest automatically matching their ``ds`` function parameters.
+
+The ``@pytest.fixture(scope="session")`` line here ensures the fixture is reused for the full ``pytest`` execution session. This means that the temporary database file will be created once and reused for each test.
+
+If you want to create that test database repeatedly for every individual test function, write the fixture function like this instead. You may want to do this if your plugin modifies the database contents in some way:
+
+.. code-block:: python
+
+    @pytest.fixture
+    def ds(tmp_path_factory):
+        # ...

docs/writing_plugins.rst

@@ -5,7 +5,7 @@ Writing plugins
 
 You can write one-off plugins that apply to just one Datasette instance, or you can write plugins which can be installed using ``pip`` and can be shipped to the Python Package Index (`PyPI <https://pypi.org/>`__) for other people to install.
 
-.. _plugins_writing_one_off:
+.. _writing_plugins_one_off:
 
 Writing one-off plugins
 -----------------------
@@ -30,8 +30,10 @@ Now you can navigate to http://localhost:8001/mydb and run this SQL::
 
 To see the output of your plugin.
 
-Writing an installable plugin
+.. _writing_plugins_cookiecutter:
------------------------------
+
+Starting an installable plugin using cookiecutter
+-------------------------------------------------
 
 Plugins that can be installed should be written as Python packages using a ``setup.py`` file.
 
@@ -43,6 +45,8 @@ The easiest way to start writing one an installable plugin is to use the `datase
 
 Read `a cookiecutter template for writing Datasette plugins <https://simonwillison.net/2020/Jun/20/cookiecutter-plugins/>`__ for more information about this template.
 
+.. _writing_plugins_packaging:
+
 Packaging a plugin
 ------------------
 
@@ -102,6 +106,8 @@ You can then install your new plugin into a Datasette virtual environment or Doc
 
 To learn how to upload your plugin to `PyPI <https://pypi.org/>`_ for use by other people, read the PyPA guide to `Packaging and distributing projects <https://packaging.python.org/tutorials/distributing-packages/>`_.
 
+.. _writing_plugins_static_assets:
+
 Static assets
 -------------
 
@@ -111,6 +117,8 @@ If your plugin has a ``static/`` directory, Datasette will automatically configu
 
 See `the datasette-plugin-demos repository <https://github.com/simonw/datasette-plugin-demos/tree/0ccf9e6189e923046047acd7878d1d19a2cccbb1>`_ for an example of how to create a package that includes a static folder.
 
+.. _writing_plugins_custom_templates:
+
 Custom templates
 ----------------
 
@@ -124,7 +132,7 @@ The priority order for template loading is:
 
 See :ref:`customization` for more details on how to write custom templates, including which filenames to use to customize which parts of the Datasette UI.
 
-.. _plugins_plugin_config:
+.. _writing_plugins_configuration:
 
 Writing plugins that accept configuration
 -----------------------------------------