From 4705b0cd2171098d4946d30ca5cb72923a976b82 Mon Sep 17 00:00:00 2001 From: Simon Willison Date: Tue, 18 Jul 2023 19:33:35 -0700 Subject: [PATCH] Documentation for the dclient insert feature, refs #8 --- README.md | 7 +++++ docs/authentication.md | 2 ++ docs/index.md | 1 + docs/inserting.md | 64 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 74 insertions(+) create mode 100644 docs/inserting.md diff --git a/README.md b/README.md index 77996fb..511095c 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,13 @@ A client CLI utility for [Datasette](https://datasette.io/) instances +## Things you can do with dclient + +- Run SQL queries against Datasette and returning the results as JSON +- Run queries against authenticated Datasette instances +- Create aliases and store authentication tokens for convenient access to Datasette +- Insert data into Datasette using the [insert API](https://docs.datasette.io/en/latest/json_api.html#the-json-write-api) (Datasette 1.0 alpha or higher) + ## Installation Install this tool using `pip`: diff --git a/docs/authentication.md b/docs/authentication.md index 5f0bd18..40e2dd7 100644 --- a/docs/authentication.md +++ b/docs/authentication.md @@ -1,3 +1,5 @@ +(authentication)= + # Authentication `dclient` can handle API tokens for Datasette instances that require authentication. diff --git a/docs/index.md b/docs/index.md index 90f9eba..b330d2e 100644 --- a/docs/index.md +++ b/docs/index.md @@ -41,4 +41,5 @@ maxdepth: 3 queries aliases authentication +inserting ``` diff --git a/docs/inserting.md b/docs/inserting.md new file mode 100644 index 0000000..1e50489 --- /dev/null +++ b/docs/inserting.md @@ -0,0 +1,64 @@ +# Inserting data + +The `dclient insert` command can be used to insert data from a local file directly into a Datasette instance, via the [Write API](https://docs.datasette.io/en/latest/json_api.html#the-json-write-api) introduced in the Datasette 1.0 alphas. + +First you'll need to {ref}`authenticate ` with the instance. + +To insert data from a `data.csv` file into a table called `my_table`, creating that table if it does not exist: + +```bash +dclient insert https://my-private-space.datasette.cloud/data my_table data.csv --create +``` + +## Supported formats + +Data can be inserted from CSV, TSV, JSON or newline-delimited JSON files. + +The format of the file will be automatically detected. You can override this by using one of the following options: + +- `--csv` +- `--tsv` +- `--json` +- `--nl` for newline-delimited JSON + +Use `--encoding ` to specify the encoding of the file. The default is `utf-8`. + +### JSON + +JSON files should be formatted like this: +```json +[ + { + "id": 1 + "column1": "value1", + "column2": "value2" + }, + { + "id": 2 + "column1": "value1", + "column2": "value2" + } +] +``` +Newline-delimited files like this: +``` +{"id": 1, "column1": "value1", "column2": "value2"} +{"id": 2, "column1": "value1", "column2": "value2"} +``` + +### CSV and TSV + +CSV and TSV files should have a header row containing the names of the columns. + +By default, `dclient` will attempt to detect the types of the different columns in the CSV and TSV files - so if a column only ever contains numeric integers it will be stored as integers in the SQLite database. + +You can disable this and have every value treated as a string using `--no-detect-types`. + +### Other options + +- `--create` - create the table if it doesn't already exist +- `--replace` - replace any rows with a matching primary key +- `--ignore` - ignore any rows with a matching existing primary key +- `--pk id` - set a primary key (for if the table is being created) + +If you use `--create` a table will be created with rows to match the columns in your uploaded data - using the correctly detected types, unless you use `--no-detect-types` in which case every column will be of type `text`.