remote/datasette
1
0
Fork 0
mirror of https://github.com/simonw/datasette.git synced 2026-09-29 05:14:21 +02:00
Code Issues Projects Releases Packages Wiki Activity

Remove datasette publish and datasette package commands entirely

Browse source 
These features are being extracted into a plugin. This removes:

- The `datasette publish` command group (cloudrun and heroku subcommands)
- The `datasette package` command
- The `publish_subcommand` plugin hook from hookspecs
- The `datasette.publish` module (cloudrun, heroku, common)
- Docker-related utilities: `make_dockerfile`, `temporary_docker_directory`,
  `link_or_copy`, `link_or_copy_directory`, `APT_GET_DOCKERFILE_EXTRAS`
- All publish/package tests
- All publish/package documentation

https://claude.ai/code/session_01Ty3D3TwBAaic9vVXAr3VU5
This commit is contained in:
Claude 2026-02-11 18:35:51 +00:00
commit 109cba4a30
No known key found for this signature in database
22 changed files with 3 additions and 2139 deletions
Download patch file Download diff file Expand all files Collapse all files

198
docs/cli-reference.rst
View file

@ -51,9 +51,7 @@ Running ``datasette --help`` shows a list of all of the available commands.
create-token Create a signed API token for the specified actor ID
inspect Generate JSON summary of provided database files
install Install plugins and packages from PyPI into the same...
package Package SQLite files into a Datasette Docker container
plugins List currently installed plugins
publish Publish specified SQLite database files to the internet...
uninstall Uninstall plugins and Python packages from the Datasette...
@ -409,200 +407,6 @@ Uninstall one or more plugins.
.. [[[end]]]
.. _cli_help_publish___help:
datasette publish
=================
Shows a list of available deployment targets for :ref:`publishing data <publishing>` with Datasette.
Additional deployment targets can be added by plugins that use the :ref:`plugin_hook_publish_subcommand` hook.
.. [[[cog
help(["publish", "--help"])
.. ]]]
::
Usage: datasette publish [OPTIONS] COMMAND [ARGS]...
Publish specified SQLite database files to the internet along with a
Datasette-powered interface and API
Options:
--help Show this message and exit.
Commands:
cloudrun Publish databases to Datasette running on Cloud Run
heroku Publish databases to Datasette running on Heroku
.. [[[end]]]
.. _cli_help_publish_cloudrun___help:
datasette publish cloudrun
==========================
See :ref:`publish_cloud_run`.
.. [[[cog
help(["publish", "cloudrun", "--help"])
.. ]]]
::
Usage: datasette publish cloudrun [OPTIONS] [FILES]...
Publish databases to Datasette running on Cloud Run
Options:
-m, --metadata FILENAME Path to JSON/YAML file containing metadata to
publish
--extra-options TEXT Extra options to pass to datasette serve
--branch TEXT Install datasette from a GitHub branch e.g.
main
--template-dir DIRECTORY Path to directory containing custom templates
--plugins-dir DIRECTORY Path to directory containing custom plugins
--static MOUNT:DIRECTORY Serve static files from this directory at
/MOUNT/...
--install TEXT Additional packages (e.g. plugins) to install
--plugin-secret <TEXT TEXT TEXT>...
Secrets to pass to plugins, e.g. --plugin-
secret datasette-auth-github client_id xxx
--version-note TEXT Additional note to show on /-/versions
--secret TEXT Secret used for signing secure values, such as
signed cookies
--title TEXT Title for metadata
--license TEXT License label for metadata
--license_url TEXT License URL for metadata
--source TEXT Source label for metadata
--source_url TEXT Source URL for metadata
--about TEXT About label for metadata
--about_url TEXT About URL for metadata
-n, --name TEXT Application name to use when building
--service TEXT Cloud Run service to deploy (or over-write)
--spatialite Enable SpatialLite extension
--show-files Output the generated Dockerfile and
metadata.json
--memory TEXT Memory to allocate in Cloud Run, e.g. 1Gi
--cpu [1|2|4] Number of vCPUs to allocate in Cloud Run
--timeout INTEGER Build timeout in seconds
--apt-get-install TEXT Additional packages to apt-get install
--max-instances INTEGER Maximum Cloud Run instances (use 0 to remove
the limit) [default: 1]
--min-instances INTEGER Minimum Cloud Run instances
--artifact-repository TEXT Artifact Registry repository to store the
image [default: datasette]
--artifact-region TEXT Artifact Registry location (region or multi-
region) [default: us]
--artifact-project TEXT Project ID for Artifact Registry (defaults to
the active project)
--help Show this message and exit.
.. [[[end]]]
.. _cli_help_publish_heroku___help:
datasette publish heroku
========================
See :ref:`publish_heroku`.
.. [[[cog
help(["publish", "heroku", "--help"])
.. ]]]
::
Usage: datasette publish heroku [OPTIONS] [FILES]...
Publish databases to Datasette running on Heroku
Options:
-m, --metadata FILENAME Path to JSON/YAML file containing metadata to
publish
--extra-options TEXT Extra options to pass to datasette serve
--branch TEXT Install datasette from a GitHub branch e.g.
main
--template-dir DIRECTORY Path to directory containing custom templates
--plugins-dir DIRECTORY Path to directory containing custom plugins
--static MOUNT:DIRECTORY Serve static files from this directory at
/MOUNT/...
--install TEXT Additional packages (e.g. plugins) to install
--plugin-secret <TEXT TEXT TEXT>...
Secrets to pass to plugins, e.g. --plugin-
secret datasette-auth-github client_id xxx
--version-note TEXT Additional note to show on /-/versions
--secret TEXT Secret used for signing secure values, such as
signed cookies
--title TEXT Title for metadata
--license TEXT License label for metadata
--license_url TEXT License URL for metadata
--source TEXT Source label for metadata
--source_url TEXT Source URL for metadata
--about TEXT About label for metadata
--about_url TEXT About URL for metadata
-n, --name TEXT Application name to use when deploying
--tar TEXT --tar option to pass to Heroku, e.g.
--tar=/usr/local/bin/gtar
--generate-dir DIRECTORY Output generated application files and stop
without deploying
--help Show this message and exit.
.. [[[end]]]
.. _cli_help_package___help:
datasette package
=================
Package SQLite files into a Datasette Docker container, see :ref:`cli_package`.
.. [[[cog
help(["package", "--help"])
.. ]]]
::
Usage: datasette package [OPTIONS] FILES...
Package SQLite files into a Datasette Docker container
Options:
-t, --tag TEXT Name for the resulting Docker container, can
optionally use name:tag format
-m, --metadata FILENAME Path to JSON/YAML file containing metadata to
publish
--extra-options TEXT Extra options to pass to datasette serve
--branch TEXT Install datasette from a GitHub branch e.g. main
--template-dir DIRECTORY Path to directory containing custom templates
--plugins-dir DIRECTORY Path to directory containing custom plugins
--static MOUNT:DIRECTORY Serve static files from this directory at /MOUNT/...
--install TEXT Additional packages (e.g. plugins) to install
--spatialite Enable SpatialLite extension
--version-note TEXT Additional note to show on /-/versions
--secret TEXT Secret used for signing secure values, such as
signed cookies
-p, --port INTEGER RANGE Port to run the server on, defaults to 8001
[1<=x<=65535]
--title TEXT Title for metadata
--license TEXT License label for metadata
--license_url TEXT License URL for metadata
--source TEXT Source label for metadata
--source_url TEXT Source URL for metadata
--about TEXT About label for metadata
--about_url TEXT About URL for metadata
--help Show this message and exit.
.. [[[end]]]
.. _cli_help_inspect___help:
datasette inspect
@ -615,7 +419,7 @@ If you are opening an immutable database, you can pass this file to the ``--insp
datasette inspect mydatabase.db > inspect-data.json
datasette serve -i mydatabase.db --inspect-file inspect-data.json
This performance optimization is used automatically by some of the ``datasette publish`` commands. You are unlikely to need to apply this optimization manually.
You are unlikely to need to apply this optimization manually.
.. [[[cog
help(["inspect", "--help"])

11
docs/custom_templates.rst
View file

@ -151,17 +151,6 @@ You can reference those files from ``datasette.yaml`` like this, see :ref:`custo
}
.. [[[end]]]
Publishing static assets
~~~~~~~~~~~~~~~~~~~~~~~~
The :ref:`cli_publish` command can be used to publish your static assets,
using the same syntax as above::
datasette publish cloudrun mydb.db --static assets:static-files/
This will upload the contents of the ``static-files/`` directory as part of the
deployment, and configure Datasette to correctly serve the assets from ``/assets/``.
.. _customization_custom_templates:
Custom templates

4
docs/deploying.rst
View file

@ -4,9 +4,7 @@
Deploying Datasette
=====================
The quickest way to deploy a Datasette instance on the internet is to use the ``datasette publish`` command, described in :ref:`publishing`. This can be used to quickly deploy Datasette to a number of hosting providers including Heroku, Google Cloud Run and Vercel.
You can deploy Datasette to other hosting providers using the instructions on this page.
You can deploy Datasette to various hosting providers using the instructions on this page.
.. _deploying_fundamentals:

1
docs/index.rst
View file

@ -43,7 +43,6 @@ Contents
ecosystem
cli-reference
pages
publish
deploying
json_api
sql_queries

2
docs/performance.rst
View file

@ -43,7 +43,7 @@ Then later you can start Datasette against the ``counts.json`` file and use it t
You need to use the ``-i`` immutable mode against the database file here or the counts from the JSON file will be ignored.
You will rarely need to use this optimization in every-day use, but several of the ``datasette publish`` commands described in :ref:`publishing` use this optimization for better performance when deploying a database file to a hosting provider.
You will rarely need to use this optimization in every-day use.
HTTP caching
------------

58
docs/plugin_hooks.rst
View file

@ -414,64 +414,6 @@ This will add the following to the end of your page:
Example: `datasette-cluster-map <https://datasette.io/plugins/datasette-cluster-map>`_
.. _plugin_hook_publish_subcommand:
publish_subcommand(publish)
---------------------------
``publish`` - Click publish command group
The Click command group for the ``datasette publish`` subcommand
This hook allows you to create new providers for the ``datasette publish``
command. Datasette uses this hook internally to implement the default ``cloudrun``
and ``heroku`` subcommands, so you can read
`their source <https://github.com/simonw/datasette/tree/main/datasette/publish>`_
to see examples of this hook in action.
Let's say you want to build a plugin that adds a ``datasette publish my_hosting_provider --api_key=xxx mydatabase.db`` publish command. Your implementation would start like this:
.. code-block:: python
from datasette import hookimpl
from datasette.publish.common import (
add_common_publish_arguments_and_options,
)
import click
@hookimpl
def publish_subcommand(publish):
@publish.command()
@add_common_publish_arguments_and_options
@click.option(
"-k",
"--api_key",
help="API key for talking to my hosting provider",
)
def my_hosting_provider(
files,
metadata,
extra_options,
branch,
template_dir,
plugins_dir,
static,
install,
plugin_secret,
version_note,
secret,
title,
license,
license_url,
source,
source_url,
about,
about_url,
api_key,
): ...
Examples: `datasette-publish-fly <https://datasette.io/plugins/datasette-publish-fly>`_, `datasette-publish-vercel <https://datasette.io/plugins/datasette-publish-vercel>`_
.. _plugin_hook_render_cell:
render_cell(row, value, column, table, database, datasette, request)

21
docs/plugins.rst
View file

@ -69,19 +69,6 @@ You can also define one-off per-project plugins by saving them as ``plugin_name.
datasette mydb.db --plugins-dir=plugins/
Deploying plugins using datasette publish
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The ``datasette publish`` and ``datasette package`` commands both take an optional ``--install`` argument. You can use this one or more times to tell Datasette to ``pip install`` specific plugins as part of the process::
datasette publish cloudrun mydb.db --install=datasette-vega
You can use the name of a package on PyPI or any of the other valid arguments to ``pip install`` such as a URL to a ``.zip`` file::
datasette publish cloudrun mydb.db \
--install=https://url-to-my-package.zip
.. _plugins_datasette_load_plugins:
Controlling which plugins are loaded
@ -484,14 +471,6 @@ Some plugins may need configuration that should stay secret - API keys for examp
}
.. [[[end]]]
If you are publishing your data using the :ref:`datasette publish <cli_publish>` family of commands, you can use the ``--plugin-secret`` option to set these secrets at publish time. For example, using Heroku you might run the following command::
datasette publish heroku my_database.db \
--name my-heroku-app-demo \
--install=datasette-auth-github \
--plugin-secret datasette-auth-github client_id your_client_id \
--plugin-secret datasette-auth-github client_secret your_client_secret
This will set the necessary environment variables and add the following to the deployed ``metadata.yaml``:
.. [[[cog

187
docs/publish.rst
View file

@ -1,187 +0,0 @@
.. _publishing:
=================
Publishing data
=================
Datasette includes tools for publishing and deploying your data to the internet. The ``datasette publish`` command will deploy a new Datasette instance containing your databases directly to a Heroku or Google Cloud hosting account. You can also use ``datasette package`` to create a Docker image that bundles your databases together with the datasette application that is used to serve them.
.. _cli_publish:
datasette publish
=================
Once you have created a SQLite database (e.g. using `csvs-to-sqlite <https://github.com/simonw/csvs-to-sqlite/>`_) you can deploy it to a hosting account using a single command.
You will need a hosting account with `Heroku <https://www.heroku.com/>`__ or `Google Cloud <https://cloud.google.com/>`__. Once you have created your account you will need to install and configure the ``heroku`` or ``gcloud`` command-line tools.
.. _publish_cloud_run:
Publishing to Google Cloud Run
------------------------------
`Google Cloud Run <https://cloud.google.com/run/>`__ allows you to publish data in a scale-to-zero environment, so your application will start running when the first request is received and will shut down again when traffic ceases. This means you only pay for time spent serving traffic.
.. warning::
Cloud Run is a great option for inexpensively hosting small, low traffic projects - but costs can add up for projects that serve a lot of requests.
Be particularly careful if your project has tables with large numbers of rows. Search engine crawlers that index a page for every row could result in a high bill.
The `datasette-block-robots <https://datasette.io/plugins/datasette-block-robots>`__ plugin can be used to request search engine crawlers omit crawling your site, which can help avoid this issue.
You will first need to install and configure the Google Cloud CLI tools by following `these instructions <https://cloud.google.com/sdk/>`__.
You can then publish one or more SQLite database files to Google Cloud Run using the following command::
datasette publish cloudrun mydatabase.db --service=my-database
A Cloud Run **service** is a single hosted application. The service name you specify will be used as part of the Cloud Run URL. If you deploy to a service name that you have used in the past your new deployment will replace the previous one.
If you omit the ``--service`` option you will be asked to pick a service name interactively during the deploy.
You may need to interact with prompts from the tool. Many of the prompts ask for values that can be `set as properties for the Google Cloud SDK <https://cloud.google.com/sdk/docs/properties>`_ if you want to avoid the prompts.
For example, the default region for the deployed instance can be set using the command::
gcloud config set run/region us-central1
You should replace ``us-central1`` with your desired `region <https://cloud.google.com/about/locations>`_. Alternately, you can specify the region by setting the ``CLOUDSDK_RUN_REGION`` environment variable.
Once it has finished it will output a URL like this one::
Service [my-service] revision [my-service-00001] has been deployed
and is serving traffic at https://my-service-j7hipcg4aq-uc.a.run.app
Cloud Run provides a URL on the ``.run.app`` domain, but you can also point your own domain or subdomain at your Cloud Run service - see `mapping custom domains <https://cloud.google.com/run/docs/mapping-custom-domains>`__ in the Cloud Run documentation for details.
See :ref:`cli_help_publish_cloudrun___help` for the full list of options for this command.
.. _publish_heroku:
Publishing to Heroku
--------------------
To publish your data using `Heroku <https://www.heroku.com/>`__, first create an account there and install and configure the `Heroku CLI tool <https://devcenter.heroku.com/articles/heroku-cli>`_.
You can publish one or more databases to Heroku using the following command::
datasette publish heroku mydatabase.db
This will output some details about the new deployment, including a URL like this one::
https://limitless-reef-88278.herokuapp.com/ deployed to Heroku
You can specify a custom app name by passing ``-n my-app-name`` to the publish command. This will also allow you to overwrite an existing app.
Rather than deploying directly you can use the ``--generate-dir`` option to output the files that would be deployed to a directory::
datasette publish heroku mydatabase.db --generate-dir=/tmp/deploy-this-to-heroku
See :ref:`cli_help_publish_heroku___help` for the full list of options for this command.
.. _publish_vercel:
Publishing to Vercel
--------------------
`Vercel <https://vercel.com/>`__ - previously known as Zeit Now - provides a layer over AWS Lambda to allow for quick, scale-to-zero deployment. You can deploy Datasette instances to Vercel using the `datasette-publish-vercel <https://github.com/simonw/datasette-publish-vercel>`__ plugin.
::
pip install datasette-publish-vercel
datasette publish vercel mydatabase.db --project my-database-project
Not every feature is supported: consult the `datasette-publish-vercel README <https://github.com/simonw/datasette-publish-vercel/blob/main/README.md>`__ for more details.
.. _publish_fly:
Publishing to Fly
-----------------
`Fly <https://fly.io/>`__ is a `competitively priced <https://fly.io/docs/pricing/>`__ Docker-compatible hosting platform that supports running applications in globally distributed data centers close to your end users. You can deploy Datasette instances to Fly using the `datasette-publish-fly <https://github.com/simonw/datasette-publish-fly>`__ plugin.
::
pip install datasette-publish-fly
datasette publish fly mydatabase.db --app="my-app"
Consult the `datasette-publish-fly README <https://github.com/simonw/datasette-publish-fly/blob/main/README.md>`__ for more details.
.. _publish_custom_metadata_and_plugins:
Custom metadata and plugins
---------------------------
``datasette publish`` accepts a number of additional options which can be used to further customize your Datasette instance.
You can define your own :ref:`metadata` and deploy that with your instance like so::
datasette publish cloudrun --service=my-service mydatabase.db -m metadata.json
If you just want to set the title, license or source information you can do that directly using extra options to ``datasette publish``::
datasette publish cloudrun mydatabase.db --service=my-service \
--title="Title of my database" \
--source="Where the data originated" \
--source_url="http://www.example.com/"
You can also specify plugins you would like to install. For example, if you want to include the `datasette-vega <https://github.com/simonw/datasette-vega>`_ visualization plugin you can use the following::
datasette publish cloudrun mydatabase.db --service=my-service --install=datasette-vega
If a plugin has any :ref:`plugins_configuration_secret` you can use the ``--plugin-secret`` option to set those secrets at publish time. For example, using Heroku with `datasette-auth-github <https://github.com/simonw/datasette-auth-github>`__ you might run the following command::
datasette publish heroku my_database.db \
--name my-heroku-app-demo \
--install=datasette-auth-github \
--plugin-secret datasette-auth-github client_id your_client_id \
--plugin-secret datasette-auth-github client_secret your_client_secret
.. _cli_package:
datasette package
=================
If you have docker installed (e.g. using `Docker for Mac <https://www.docker.com/docker-mac>`_) you can use the ``datasette package`` command to create a new Docker image in your local repository containing the datasette app bundled together with one or more SQLite databases::
datasette package mydatabase.db
Here's example output for the package command::
datasette package parlgov.db --extra-options="--setting sql_time_limit_ms 2500"
Sending build context to Docker daemon 4.459MB
Step 1/7 : FROM python:3.11.0-slim-bullseye
---> 79e1dc9af1c1
Step 2/7 : COPY . /app
---> Using cache
---> cd4ec67de656
Step 3/7 : WORKDIR /app
---> Using cache
---> 139699e91621
Step 4/7 : RUN pip install datasette
---> Using cache
---> 340efa82bfd7
Step 5/7 : RUN datasette inspect parlgov.db --inspect-file inspect-data.json
---> Using cache
---> 5fddbe990314
Step 6/7 : EXPOSE 8001
---> Using cache
---> 8e83844b0fed
Step 7/7 : CMD datasette serve parlgov.db --port 8001 --inspect-file inspect-data.json --setting sql_time_limit_ms 2500
---> Using cache
---> 1bd380ea8af3
Successfully built 1bd380ea8af3
You can now run the resulting container like so::
docker run -p 8081:8001 1bd380ea8af3
This exposes port 8001 inside the container as port 8081 on your host machine, so you can access the application at ``http://localhost:8081/``
You can customize the port that is exposed by the container using the ``--port`` option::
datasette package mydatabase.db --port 8080
A full list of options can be seen by running ``datasette package --help``:
See :ref:`cli_help_package___help` for the full list of options for this command.

12
docs/settings.rst
View file

@ -376,15 +376,3 @@ One way to generate a secure random secret is to use Python like this::
Plugin authors can make use of this signing mechanism in their plugins using the :ref:`datasette.sign() <datasette_sign>` and :ref:`datasette.unsign() <datasette_unsign>` methods.
.. _setting_publish_secrets:
Using secrets with datasette publish
------------------------------------
The :ref:`cli_publish` and :ref:`cli_package` commands both generate a secret for you automatically when Datasette is deployed.
This means that every time you deploy a new version of a Datasette project, a new secret will be generated. This will cause signed cookies to become invalid on every fresh deploy.
You can fix this by creating a secret that will be used for multiple deploys and passing it using the ``--secret`` option::
datasette publish cloudrun mydb.db --service=my-service --secret=cdb19e94283a20f9d42cca5