From 6341f8cbc7833022012804dea120b838ec1f6558 Mon Sep 17 00:00:00 2001 From: Simon Willison Date: Mon, 24 Jun 2019 09:28:42 -0700 Subject: [PATCH] Better coverage of sqlite-utils in FTS docs, closes #525 --- docs/full_text_search.rst | 32 ++++++++++++++++++++++---------- setup.py | 4 ++++ 2 files changed, 26 insertions(+), 10 deletions(-) diff --git a/docs/full_text_search.rst b/docs/full_text_search.rst index 97656bb8..138b8e0b 100644 --- a/docs/full_text_search.rst +++ b/docs/full_text_search.rst @@ -28,7 +28,28 @@ To set up full-text search for a table, you need to do two things: * Create a new FTS virtual table associated with your table * Populate that FTS table with the data that you would like to be able to run searches against -To enable full-text search for a table called ``items`` that works against the ``name`` and ``description`` columns, you would run the following SQL to create a new ``items_fts`` FTS virtual table: +Configuring FTS using sqlite-utils +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +`sqlite-utils `__ is a CLI utility and Python library for manipulating SQLite databases. You can use `it from Python code `__ to configure FTS search, or you can achieve the same goal `using the accompanying command-line tool `__. + +Here's how to use ``sqlite-utils`` to enable full-text search for an ``items`` table across the ``name`` and ``description`` columns:: + + $ sqlite-utils enable-fts mydatabase.db items name description + +Configuring FTS using csvs-to-sqlite +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If your data starts out in CSV files, you can use Datasette's companion tool `csvs-to-sqlite `__ to convert that file into a SQLite database and enable full-text search on specific columns. For a file called ``items.csv`` where you want full-text search to operate against the ``name`` and ``description`` columns you would run the following:: + + $ csvs-to-sqlite items.csv items.db -f name -f description + +Configuring FTS by hand +~~~~~~~~~~~~~~~~~~~~~~~ + +We recommend using `sqlite-utils `__, but if you want to hand-roll a SQLite full-text search table you can do so using the following SQL. + +To enable full-text search for a table called ``items`` that works against the ``name`` and ``description`` columns, you would run this SQL to create a new ``items_fts`` FTS virtual table: .. code-block:: sql @@ -71,8 +92,6 @@ And then populate it like this: You can use this technique to populate the full-text search index from any combination of tables and joins that makes sense for your project. -The `sqlite-utils tool `__ provides a command-line mechanism that can be used to implement the above steps. - .. _full_text_search_table_or_view: Configuring full-text search for a table or view @@ -103,13 +122,6 @@ Here is an example which enables full-text search for a ``display_ads`` view whi } } -Setting up full-text search using csvs-to-sqlite ------------------------------------------------- - -If your data starts out in CSV files, you can use Datasette's companion tool `csvs-to-sqlite `_ to convert that file into a SQLite database and enable full-text search on specific columns. For a file called ``items.csv`` where you want full-text search to operate against the ``name`` and ``description`` columns you would run the following:: - - csvs-to-sqlite items.csv items.db -f name -f description - The table view API ------------------ diff --git a/setup.py b/setup.py index f66d03da..edb8d51e 100644 --- a/setup.py +++ b/setup.py @@ -56,6 +56,10 @@ setup( """, setup_requires=["pytest-runner"], extras_require={ + "docs": [ + "sphinx_rtd_theme", + "sphinx-autobuild", + ], "test": [ "pytest==4.6.1", "pytest-asyncio==0.10.0",