remote/sqlite-utils
1
0
Fork 0
mirror of https://github.com/simonw/sqlite-utils.git synced 2026-07-23 17:34:32 +02:00
Code Issues Projects Releases Packages Wiki Activity
398 commits 70 branches 137 tags 7.3 MiB
Commit graph

140 commits

Author SHA1 Message Date
Simon Willison
7c0ef116ed pk=['id'] now equivalent to pk='id', closes #181 2020-10-14 14:59:38 -07:00
Simon Willison
5a63b9e88c Simplify drop-foreign-key, and drop_foreign_keys, closes #177 2020-09-24 09:19:07 -07:00
Simon Willison
022cdd97a9
 		Much, much faster extract() implementation 
Takes my test down from ten minutes to four seconds!

* Removed unnecessary update() optimization
* Added column_order= to .transform() and .transform_sql()
* Tests for reusing lookup table in extract()

Closes #172
 2020-09-24 08:43:55 -07:00
Simon Willison
5eb14d1c1f Added several missing 'return self' to support chaining 2020-09-24 07:51:36 -07:00
Simon Willison
66d506587e Some optimizations for extract() 
Refs #172 - seems to give me about 20% speedup.
 2020-09-23 13:12:19 -07:00
Simon Willison
5534c320e4 Applied Black 2020-09-22 17:32:40 -07:00
Simon Willison
b8e0048485 Fixed PRAGMA foreign_keys handling for .transform, closes #167 2020-09-22 17:12:56 -07:00
Simon Willison
5c4d58d152 Progress bar for "sqlite-utils extract", closes #169 2020-09-22 17:02:29 -07:00
Simon Willison
2db6c5b2d5 table.extract() now works with rowid tables, refs #42 2020-09-22 16:11:28 -07:00
Simon Willison
71782311ce New .rows_where(select=) argument 2020-09-22 16:10:14 -07:00
Simon Willison
c3210f2ffb Added table.extract(rename=) option, refs #42 2020-09-22 15:57:02 -07:00
Simon Willison
f8553799d3 table.extract() method, refs #42 2020-09-22 15:20:18 -07:00
Simon Willison
f29f6821f2 Applied Black 2020-09-22 00:49:27 -07:00
Simon Willison
752d261229 Implemented sqlite-utils transform command, closes #164 2020-09-22 00:47:58 -07:00
Simon Willison
f8e10df00e
 		Keyword only arguments for transform() 
Also renamed columns= to types=

Closes #165
 2020-09-21 23:39:10 -07:00
Simon Willison
987dd123f2
 		table.transform() method - closes #114 2020-09-21 21:20:01 -07:00
Simon Willison
482477585a @db.register_function decorator, closes #162 2020-09-21 17:31:43 -07:00
Simon Willison
e23eedb4ce ignore=True argument for add_foreign_key, closes #112 
Also --ignore for add-foreign-key command

Plus table.add_foreign_key(...) now returns self, allowing more chaining
 2020-09-20 15:17:25 -07:00
Simon Willison
ecb50c8f76 .enable_fts(..., replace=True) argument, closes #160 2020-09-20 15:05:46 -07:00
Simon Wiles
e6d202b742
 		Handle case where subsequent records (after first batch) include extra columns 
Refs #145.

* Extract build_insert_queries_and_params
* Extract insert_chunk so it can be called recursively

Thanks, @simonwiles
 2020-09-08 16:20:36 -07:00
Simon Willison
176f4e0ef4 sqlite-utils rebuild-fts command, closes #155 2020-09-08 16:16:03 -07:00
Simon Willison
9680a0291c 'Soundness check' is better 2020-09-08 15:24:39 -07:00
Simon Willison
64799df78b .optimize() no longer cleans up _docsize 
This isn't necessary any more since the new .rebuild_fts()
method can achieve the same thing.

Refs #155, #153
 2020-09-08 15:18:12 -07:00
Simon Willison
4c0f79398f table.rebuild_fts() method, refs #155 2020-09-08 15:09:25 -07:00
Simon Willison
cf2cb244fa
 		Tracer mechanism for showing underlying SQL queries 
* Pass a tracer= function to Database constructor
* New db.tracer() contextmanager
* Neater SQL indentation, because tracer means it could be visible now
* New db.execute() and db.executescript() methods

Closes #150
 2020-09-07 14:56:59 -07:00
Simon Willison
3e87500e15 table.optimize() deletes junk docsize rows 
Closes #153. Closes #149.
 2020-09-07 14:16:13 -07:00
Simon Willison
de10590344 recursive_triggers=on by default, closes #152 
Refs #149
 2020-09-07 13:45:06 -07:00
Simon Willison
e878f2a8fe Applied latest black 2020-09-07 12:45:54 -07:00
Simon Willison
59e3d4d171 Neater indentation for SQL used in schemas, closes #148 2020-09-07 11:12:45 -07:00
Simon Wiles
947bb7626f
 		insert_all(..., alter=True) works for columns introduced after first 100 records 
* Insert all columns for every chunk
* Update unit test to reflect new behaviour
* Test that exception is raised
* Update documentation

Closes #139. Thanks, Simon Wiles!
 2020-08-28 15:30:13 -07:00
Simon Willison
2d2d724e32 Tools for enabling and disabling WAL, closes #132 2020-08-10 11:59:21 -07:00
Simon Willison
57e4eb8e55 sqlite-utils populate-fts --tokenize= option, closes #130 2020-08-01 13:51:05 -07:00
Simon Willison
617e6f070c enable_fts(..., tokenize=X) parameter, refs #130 2020-08-01 13:40:36 -07:00
Simon Willison
f804690274 Support inserting UUID and memoryview, closes #128 2020-07-29 18:10:33 -07:00
Thomas Sibley
ae4593316c Add insert --truncate option 
Deletes all rows in the table (if it exists) before inserting new rows.
SQLite doesn't implement a TRUNCATE TABLE statement but does optimize an
unqualified DELETE FROM.

This can be handy if you want to refresh the entire contents of a table
but a) don't have a PK (so can't use --replace), b) don't want the table
to disappear (even briefly) for other connections, and c) have to handle
records that used to exist being deleted.

Ideally the replacement of rows would appear instantaneous to other
connections by putting the DELETE + INSERT in a transaction, but this is
very difficult without breaking other code as the current transaction
handling is inconsistent and non-systematic.  There exists the
possibility for the DELETE to succeed but the INSERT to fail, leaving an
empty table.  This is not much worse, however, than the current
possibility of one chunked INSERT succeeding and being committed while
the next chunked INSERT fails, leaving a partially complete operation.
 2020-07-08 10:26:20 -07:00
Simon Willison
af3f81b540 Store decimal.Decimal in DB as FLOAT, closes #110 2020-05-10 18:50:03 -07:00
Simon Willison
5c1df4e306 replace=True and ignore=True parameters for create_view(), closes #106 2020-05-02 09:02:04 -07:00
Simon Willison
d56029549a Serialize JSON with non-JSON values, closes #102 2020-05-01 13:45:39 -07:00
Simon Willison
2ab62bcd54 New columns= parameter for over-riding column types, closes #100 2020-04-17 16:53:25 -07:00
Simon Willison
125c625fbc .rows_where(..., order_by=) argument, closes #76 2020-04-15 20:12:55 -07:00
Simon Willison
7e4b9997c2 Database(..., recreate=True) option, refs #97 2020-04-12 20:46:51 -07:00
Simon Willison
635c91475a Only set last_pk on singular .insert()/.update(), refs #98 2020-04-12 20:22:49 -07:00
bob
8ea626e5fc
 		Add type conversion for Panda's Timestamp (#96) 
Thanks, @b0b5h4rp13!
 2020-03-30 21:40:48 -07:00
Simon Willison
1125460497 Improved column type introspection, closes #92 2020-03-14 13:04:06 -07:00
Simon Willison
0c36feb6ca .enable_fts() now works with columns with spaces in them, closes #90 2020-03-01 22:10:43 -08:00
Simon Willison
04ec53c039 Validate column names in more places, refs #86 2020-02-26 20:55:17 -08:00
Simon Willison
f9473ace14 disable-fts and .disable_fts(), closes #88 2020-02-26 20:40:35 -08:00
Simon Willison
b0ca657f49 Disallow square braces in column names, closes #86 2020-02-26 20:16:02 -08:00
Simon Willison
685e6a1bb3 Detect subclasses of dict/tuple/list, fixes #87 2020-02-15 18:20:39 -08:00
Simon Willison
6f3cb2c106 create_index now works with columns with spaces, closes #85 2020-02-10 21:13:15 -08:00