Commit graph

62 commits

Author SHA1 Message Date
Simon Willison
ebc802f7ff sqlite-utils insert-files command, closes #122 2020-07-27 00:08:57 -07:00
Simon Willison
1a61a6d3d6 Ability to insert base64 binary data as JSON, closes #126 2020-07-26 20:59:34 -07:00
Simon Willison
814d4a7f90 -p for passing named params to query, closes #124 2020-07-26 20:53:51 -07:00
Simon Willison
20e543e9a4 Output binary columns as base64 in JSON, closes #125 2020-07-26 17:48:36 -07:00
Simon Willison
bc8409941f --raw option, refs #123 2020-07-26 09:43:45 -07:00
Thomas Sibley
ae4593316c Add insert --truncate option
Deletes all rows in the table (if it exists) before inserting new rows.
SQLite doesn't implement a TRUNCATE TABLE statement but does optimize an
unqualified DELETE FROM.

This can be handy if you want to refresh the entire contents of a table
but a) don't have a PK (so can't use --replace), b) don't want the table
to disappear (even briefly) for other connections, and c) have to handle
records that used to exist being deleted.

Ideally the replacement of rows would appear instantaneous to other
connections by putting the DELETE + INSERT in a transaction, but this is
very difficult without breaking other code as the current transaction
handling is inconsistent and non-systematic.  There exists the
possibility for the DELETE to succeed but the INSERT to fail, leaving an
empty table.  This is not much worse, however, than the current
possibility of one chunked INSERT succeeding and being committed while
the next chunked INSERT fails, leaving a partially complete operation.
2020-07-08 10:26:20 -07:00
Thomas Sibley
f8277d0fb9
sqlite-utils query can now run DML (#120)
* Failing test showing that DML in `sqlite-utils query` doesn't work
* Run `sqlite-utils query` in a transaction so that DML is committed

Thanks, @tsibley!
2020-07-07 22:14:04 -07:00
Simon Willison
03ee97d225 CLI now supports upsert/insert - closes #115 2020-06-12 10:40:53 -07:00
Simon Willison
98019e92d0 Ran black, refs #111 2020-05-10 18:26:16 -07:00
Simon Willison
4e9cb739c7 drop-table and drop-view commands, closes #111 2020-05-10 17:44:21 -07:00
Simon Willison
d16097231c Added sqlite-utils create-view command, closes #107 2020-05-03 08:36:29 -07:00
Simon Willison
9f6085b4e4 create-table --ignore and --replace, refs #27 2020-05-03 08:24:39 -07:00
Simon Willison
4d5916075d create-table --not-null, --default, --fk, refs #27 2020-05-03 08:09:00 -07:00
Simon Willison
36d256b047 Initial implementation of create-table command, refs #27 2020-05-02 20:55:40 -07:00
Simon Willison
344e9573ca Added sqlite-utils views command, closes #105 2020-05-01 13:38:28 -07:00
Simon Willison
147b52622d sqlite-utils tables ... --schema option, closes #104 2020-05-01 10:12:16 -07:00
Simon Willison
f9473ace14 disable-fts and .disable_fts(), closes #88 2020-02-26 20:40:35 -08:00
Simon Willison
a0f0175d64 Updated help for upsert, refs #66 2019-12-29 22:05:31 -08:00
Simon Willison
84bcabd093 New upsert implementation, refs #66 2019-12-30 05:29:23 +00:00
Simon Willison
cfbc09967e Ran black, plus added comments for next step
Refs #66
2019-12-30 05:29:23 +00:00
Simon Willison
607a2a9ff6 insert --replace and insert(..., replace=True)
Refs #66
2019-12-30 05:29:23 +00:00
Amjith Ramanujam
405e092d59 Option to add triggers when enabling FTS (#57)
--create-triggers CLI option and create_triggers=True in the Python library

* Add an option to create triggers for fts table.
* Add cli option for the create-update-trigger.
* Add tests for the create-update-trigger option.
* Change FTS table escaping to square brackets.
2019-09-02 16:42:28 -07:00
Simon Willison
152eb2afaf Use pysqlite3 if available 2019-07-22 15:39:35 -07:00
Simon Willison
09316835e6 Implemented --tsv option, closes #41 2019-07-18 21:50:38 -07:00
Simon Willison
c65b67ca46
.get() method plus support for compound primary keys (#40)
* create_table now handles compound primary keys
* CLI now accepts multiple --pk for compound primary keys
* Docs for compound primary keys with CLI and Python library
* New .get() method plus documentation

Closes #36, closes #39
2019-07-14 21:28:51 -07:00
Simon Willison
e8f887ef4a
sqlite-utils index-foreign-keys / db.index_foreign_keys()
Closes #33
2019-06-30 16:50:54 -07:00
Simon Willison
208f56fbaf --not-null and --default options to insert/update, closes #24 2019-06-12 23:30:16 -07:00
Simon Willison
d645032cfa add_foreign_key can now detect table and pk, refs #25 2019-06-12 21:51:09 -07:00
Simon Willison
2fed87da6e --not-null-default and not_null_default=, refs #24 2019-06-12 18:35:02 -07:00
Simon Willison
50e2f94b58 Ability to add a column that is a foreign key reference
Python API:

    db["dogs"].add_column("species_id", fk="species")
    # or
    db["dogs"].add_column("species_id", fk="species", fk_col="ref")

CLI:

    $ sqlite-utils add-column mydb.db dogs species_id --fk species
    # or
    $ sqlite-utils add-column mydb.db dogs species_id --fk species --fk-col ref

Closes #16
2019-05-28 21:54:43 -07:00
Simon Willison
00c5a49a87 Option to ignore inserts if primary key exists already
Support for SQLite's INSERT OR IGNORE

In the API layer it looks like this:

    fresh_db["test"].insert({"id": 1, "bar": 3}, ignore=True)

For the CLI layer it looks like this:

    $ sqlite-utils insert data.db dogs dogs.json --ignore

Closes #21
2019-05-28 21:15:57 -07:00
Simon Willison
cec8c3c482 sqlite-utils rows now obeys --json-cols 2019-05-27 17:47:59 -07:00
Simon Willison
251e473041 --json-cols option for outputting nested JSON, closes #20 2019-05-24 17:56:44 -07:00
Simon Willison
747ea5d6bc Fixed help text for sqlite-utils enable-fts
Closes #19
2019-05-24 17:43:46 -07:00
Simon Willison
eff52023c6 alter=True/--alter option to automatically add missing columns
Closes #18
2019-05-24 17:41:04 -07:00
Simon Willison
3cab079d3e add-column col_type now optional, defaults to str 2019-02-24 14:24:00 -08:00
Simon Willison
f8d3b7cfe5 sqlite-utils add-foreign-key books.db books author_id authors id
Command for adding foreign keys to existing tables. Closes #2
2019-02-24 13:33:45 -08:00
Simon Willison
0bc49e938e 'sqlite-utils add-column name type' command, closes #15 2019-02-24 12:04:33 -08:00
Simon Willison
8a5d0d80c3 sqlite-utils create-index command, closes #14 2019-02-24 11:11:21 -08:00
Simon Willison
6c3f5e6479 Added --table and --fmt options for table output using tabulate 2019-02-23 22:45:17 -08:00
Simon Willison
e3c0ed1b1a sqlite-utils tables now takes --counts, --columns, --csv, --nl 2019-02-22 18:12:53 -08:00
Simon Willison
c7dbb03a10 Added 'sqlite-utils rows db.db tablename' command 2019-02-22 17:52:17 -08:00
Simon Willison
f2ca48c0da Default command now executes queries, --csv or --json
I replaced the following commands:

    sqlite-utils json db.db "select * from table"
    sqlite-utils csv db.db "select * from table"

With a unified 'query' command, which is now set as the default:

    sqlite-utils db.db "select * from table"
    sqlite-utils db.db "select * from table" --csv
2019-02-22 17:40:21 -08:00
Simon Willison
fd1b6eb82d Help text for enable-fts and populate-fts 2019-02-07 21:25:26 -08:00
Simon Willison
3f8ef35b2a enable-fts and populate-fts CLI commands 2019-02-07 21:18:24 -08:00
Simon Willison
ac9311aaf5 Refactored insert and upsert to share the same options 2019-02-06 21:50:25 -08:00
Simon Willison
441c131db5 Better --help text for insert command 2019-01-29 07:40:34 -08:00
Simon Willison
1843d55615 @cli.command(name="csv") instead of import csv as csv_std 2019-01-29 07:37:12 -08:00
Simon Willison
82f0f6076a sqlite-utils insert db.db foo - --csv
New option for efficiently inserting rows from a CSV.

Uses a generator so this will happily consume enormous CSV files without
needing to slurp the whole thing into memory first.
2019-01-29 07:28:12 -08:00
Simon Willison
50589f8523 CLI insert tool now uses generators, closes #7
Also cleaned up the logic so we commit rows in batches too.
2019-01-27 22:26:45 -08:00