Inserting data

The dclient insert command can be used to insert data from a local file directly into a Datasette instance, via the Write API introduced in the Datasette 1.0 alphas.

First you'll need to {ref}authenticate <authentication> with the instance.

To insert data from a data.csv file into a table called my_table, creating that table if it does not exist:

dclient insert https://my-private-space.datasette.cloud/data my_table data.csv --create

You can also pipe data into standard input:

curl -s 'https://api.github.com/repos/simonw/dclient/issues' | \
  dclient insert \
    https://my-private-space.datasette.cloud/data \
    issues - --create

Supported formats

Data can be inserted from CSV, TSV, JSON or newline-delimited JSON files.

The format of the file will be automatically detected. You can override this by using one of the following options:

--csv
--tsv
--json
--nl for newline-delimited JSON

Use --encoding <encoding> to specify the encoding of the file. The default is utf-8.

JSON

JSON files should be formatted like this:

[
    {
        "id": 1
        "column1": "value1",
        "column2": "value2"
    },
    {
        "id": 2
        "column1": "value1",
        "column2": "value2"
    }
]

Newline-delimited files like this:

{"id": 1, "column1": "value1", "column2": "value2"}
{"id": 2, "column1": "value1", "column2": "value2"}

CSV and TSV

CSV and TSV files should have a header row containing the names of the columns.

By default, dclient will attempt to detect the types of the different columns in the CSV and TSV files - so if a column only ever contains numeric integers it will be stored as integers in the SQLite database.

You can disable this and have every value treated as a string using --no-detect-types.

Other options

--create - create the table if it doesn't already exist
--replace - replace any rows with a matching primary key
--ignore - ignore any rows with a matching existing primary key
--pk id - set a primary key (for if the table is being created)

If you use --create a table will be created with rows to match the columns in your uploaded data - using the correctly detected types, unless you use --no-detect-types in which case every column will be of type text.

dclient insert --help

Usage: dclient insert [OPTIONS] URL_OR_ALIAS TABLE FILEPATH

  Insert data into a remote Datasette instance

  Example usage:

      dclient insert \
        https://private.datasette.cloud/data \
        mytable data.csv --pk id --create

Options:
  --csv                 Input is CSV
  --tsv                 Input is TSV
  --json                Input is JSON
  --nl                  Input is newline-delimited JSON
  --encoding TEXT       Character encoding for CSV/TSV
  --no-detect-types     Don't detect column types for CSV/TSV
  --replace             Replace rows with a matching primary key
  --ignore              Ignore rows with a matching primary key
  --create              Create table if it does not exist
  --pk TEXT             Columns to use as the primary key when creating the
                        table
  --batch-size INTEGER  Send rows in batches of this size
  -t, --token TEXT      API token
  --silent              Don't output progress
  --help                Show this message and exit.

3.7 KiB Raw Blame History