Commit graph

156 commits

Author SHA1 Message Date
Simon Willison
f722b0a730
allow_sql config option to disable custom SQL, closes #284 2018-05-24 22:50:50 -07:00
Simon Willison
50920cfe3d
allow_facet, allow_download, suggest_facets boolean --config
Refs #284
2018-05-24 18:12:27 -07:00
Simon Willison
81df47e8d9
Moved .execute() method from BaseView to Datasette class
Also introduced new Results() class with results.truncated, results.description, results.rows
2018-05-24 17:15:53 -07:00
Ravi Kotecha
bd30c696e1 Build Dockerfile with recent Sqlite + Spatialite (#280)
Closes #278

```bash
  $ docker run --rm -it datasette spatialite
  SpatiaLite version ..: 4.4.0-RC0	Supported Extensions:
    - 'VirtualShape'	[direct Shapefile access]
    - 'VirtualDbf'		[direct DBF access]
    - 'VirtualXL'		[direct XLS access]
    - 'VirtualText'		[direct CSV/TXT access]
    - 'VirtualNetwork'	[Dijkstra shortest path]
    - 'RTree'		[Spatial Index - R*Tree]
    - 'MbrCache'		[Spatial Index - MBR cache]
    - 'VirtualSpatialIndex'	[R*Tree metahandler]
    - 'VirtualElementary'	[ElemGeoms metahandler]
    - 'VirtualKNN'	[K-Nearest Neighbors metahandler]
    - 'VirtualXPath'	[XML Path Language - XPath]
    - 'VirtualFDO'		[FDO-OGR interoperability]
    - 'VirtualGPKG'	[OGC GeoPackage interoperability]
    - 'VirtualBBox'		[BoundingBox tables]
    - 'SpatiaLite'		[Spatial SQL - OGC]
  PROJ.4 version ......: Rel. 4.9.3, 15 August 2016
  GEOS version ........: 3.5.1-CAPI-1.9.1 r4246
  TARGET CPU ..........: x86_64-linux-gnu
  the SPATIAL_REF_SYS table already contains some row(s)
  SQLite version ......: 3.23.1
  Enter ".help" for instructions
  SQLite version 3.23.1 2018-04-10 17:39:29
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  spatialite>
```

```bash
$ docker run --rm -it datasette python -c "import sqlite3; print(sqlite3.sqlite_version)"
3.23.1
```

Also updates the query used to check for FTS5 as the old version wasn't
detecting FTS5 for some reason.
2018-05-23 10:43:34 -07:00
Russ Garrett
58b5a37dbb Refactor inspect logic 2018-05-22 07:03:06 -07:00
Simon Willison
f6183ff5fa
Renamed --limit to --config, added --help-config, closes #274
Removed the --page_size= argument to datasette serve in favour of:

    datasette serve --config default_page_size:50 mydb.db

Added new help section:

    $ datasette --help-config
    Config options:
      default_page_size            Default page size for the table view
                                   (default=100)
      max_returned_rows            Maximum rows that can be returned from a table
                                   or custom query (default=1000)
      sql_time_limit_ms            Time limit for a SQL query in milliseconds
                                   (default=1000)
      default_facet_size           Number of values to return for requested facets
                                   (default=30)
      facet_time_limit_ms          Time limit for calculating a requested facet
                                   (default=200)
      facet_suggest_time_limit_ms  Time limit for calculating a suggested facet
                                   (default=50)
2018-05-20 10:01:49 -07:00
Simon Willison
39426ff0e4
Added /-/limits and /-/limits.json, closes #270 2018-05-17 23:16:28 -07:00
Simon Willison
cef9a9a870
--limit= mechanism plus new limits for facets
Replaced the --max_returned_rows and --sql_time_limit_ms options to
"datasette serve" with a new --limit option, which supports a larger
list of limits.

Example usage:

	datasette serve --limit max_returned_rows:1000 \
		--limit sql_time_limit_ms:2500 \
		--limit default_facet_size:50 \
		--limit facet_time_limit_ms:1000 \
		--limit facet_suggest_time_limit_ms:500

New docs: https://datasette.readthedocs.io/en/latest/limits.html

Closes #270
Closes #264
2018-05-17 22:08:26 -07:00
Simon Willison
eaf715a60a Extract /-/plugins view into a method 2018-05-14 00:04:23 -03:00
Simon Willison
cf1fe693e5 Used isort to re-order my imports 2018-05-14 00:04:23 -03:00
Simon Willison
3686385551 Ran black source formatting tool against new views/ and app.py 2018-05-14 00:04:23 -03:00
Simon Willison
1f69269fe9 Refactored views into new views/ modules, refs #256 2018-05-14 00:04:23 -03:00
Simon Willison
eaaa3ea149
Initial implementation of facets, plus tests and docs
Refs #255
2018-05-12 19:29:41 -03:00
Simon Willison
04d21ccd08
/-/versions now includes SQLite fts_versions, closes #252 2018-05-11 10:19:25 -03:00
Simon Willison
e10f8e1274
Slight simplification of /-/inspect 2018-05-06 10:05:38 -03:00
Simon Willison
1259b8ac0b
Support _search_COLUMN=text searches, closes #237 2018-05-05 19:33:08 -03:00
Simon Willison
ca290719ed
Show version on /-/plugins page, closes #248 2018-05-04 15:04:33 -03:00
Simon Willison
d4da4c92c8
?_size=max option, closes #249 2018-05-04 15:03:40 -03:00
Simon Willison
bb87cf8730
Added /-/versions and /-/versions.json, closes #244
Sample output:

    {
      "python": {
        "version": "3.6.3",
        "full": "3.6.3 (default, Oct  4 2017, 06:09:38) \n[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.37)]"
      },
      "datasette": {
        "version": "0.20"
      },
      "sqlite": {
        "version": "3.23.1",
        "extensions": {
          "json1": null,
          "spatialite": "4.3.0a"
        }
      }
    }
2018-05-03 11:09:27 -03:00
Simon Willison
349e262bb1 Renamed ?_sql_time_limit_ms= to ?_timelimit, closes #242 2018-05-01 17:20:39 -07:00
Simon Willison
a4cc5dc813 New ?_shape=array option + tweaks to _shape, closes #245
* Default is now ?_shape=arrays (renamed from lists)
* New ?_shape=array returns an array of objects as the root object
* Changed ?_shape=object to return the object as the root
* Updated docs
2018-05-01 17:20:39 -07:00
Simon Willison
2565d623af ?_shape=array experimental feature 2018-05-01 17:20:39 -07:00
Simon Willison
aa954382c3
FTS tables now detected by inspect(), closes #240 2018-04-28 17:04:32 -07:00
Simon Willison
f188ceaa2a
New ?_size=XXX querystring parameter for table view, closes #229
Also added documentation for all of the _special arguments.

Plus deleted some duplicate logic implementing _group_count.
2018-04-25 21:47:12 -07:00
Simon Willison
4504d5160b
If max_returned_rows==page_size, increment max_returned_rows
Fixes #230, where if the two were equal pagination didn't work correctly.
2018-04-25 21:04:12 -07:00
Simon Willison
02ee31c8b4
New hidden: True option for table metadat, closes #239 2018-04-25 20:42:57 -07:00
Simon Willison
d3a0069c54
Hide idx_* tables if spatialite detected, closes #228 2018-04-25 20:25:21 -07:00
Simon Willison
f3f4295712
label_column now defined on the table-being-linked-to, fixes #234 2018-04-22 13:46:18 -07:00
Simon Willison
f27cabbaf3
label_column option in metadata.json - closes #234 2018-04-22 10:53:16 -07:00
Simon Willison
e2750c7cc0
Add col-X classes to HTML table on custom query page 2018-04-19 08:36:14 -07:00
Simon Willison
b52171db1e
Plugins can now bundle custom templates, closes #224
Refs #14
2018-04-18 22:50:27 -07:00
Simon Willison
b55809a1e2
Added /-/metadata /-/plugins /-/inspect, closes #225 2018-04-18 22:25:22 -07:00
Simon Willison
4be6deb947
Fix for plugins in Python 3.5 (#222)
ModuleNotFoundError is not a thing in Python 3.5, so catch KeyError/ImportError instead.
2018-04-17 20:24:20 -07:00
Simon Willison
1c36d07dd4
New plugin hooks: extra_css_urls() and extra_js_urls()
Closes #214
2018-04-17 20:12:21 -07:00
Simon Willison
0bb483ca5b
/-/static-plugins/PLUGIN_NAME/ now serves static/ from plugins
Refs #214
2018-04-17 19:32:48 -07:00
Simon Willison
a5792a8c61
<th> now gets class="col-X" - plus added col-X documentation
Refs #209
2018-04-17 19:11:54 -07:00
Russ Garrett
4586aa506a Don't duplicate simple primary keys in the link column
When there's a simple (single-column) primary key, it looks weird to
duplicate it in the link column.

This change removes the second PK column and treats the link column as
if it were the PK column from a header/sorting perspective.
2018-04-17 18:13:02 -07:00
Russ Garrett
b231d4243d Correct escaping for HTML display of row links 2018-04-17 18:13:02 -07:00
Simon Willison
5ebc7137d7
Use application/octet-stream for downloadable databses
I'd also like to send the Content-Length here but that's not currently
possible in Sanic - see bug report here:

https://github.com/channelcat/sanic/issues/1194
2018-04-17 17:31:54 -07:00
Simon Willison
e7c769ef30
Working implementation of #216 which passes the tests
Reverted commit 5364fa7f33 (where I removed the
code that didn't work).

Added primary keys to order-by clause for sorting to get tests to pass
2018-04-16 18:41:55 -07:00
Simon Willison
5364fa7f33
Revert #216 until I can get tests to pass in Travis
Revert "Fix for _sort_desc=sortable_with_nulls test, refs #216"

This reverts commit 07fc2d113e.

Revert "Fixed #216 - paginate correctly when sorting by nullable column"

This reverts commit 2abe539a0f.
2018-04-16 17:53:37 -07:00
Simon Willison
2abe539a0f
Fixed #216 - paginate correctly when sorting by nullable column 2018-04-16 16:52:50 -07:00
Simon Willison
10a34f995c
Removed rogue print() call 2018-04-16 08:38:09 -07:00
Simon Willison
b2955d9065
New --plugins-dir=plugins/ option (#212)
* New --plugins-dir=plugins/ option

New option causing Datasette to load and evaluate all of the Python files in
the specified directory and register any plugins that are defined in those
files.

This new option is available for the following commands:

    datasette serve mydb.db --plugins-dir=plugins/
    datasette publish now/heroku mydb.db --plugins-dir=plugins/
    datasette package mydb.db --plugins-dir=plugins/

* Unit tests for --plugins-dir=plugins/

Closes #211
2018-04-15 22:22:01 -07:00
Simon Willison
33c7c53ff8
Start of the plugin system, based on pluggy (#210)
Uses https://pluggy.readthedocs.io/ originally created for the py.test project

We're starting with two plugin hooks:

prepare_connection(conn)

This is called when a new SQLite connection is created. It can be used to register custom SQL functions.

prepare_jinja2_environment(env)

This is called with the Jinja2 environment. It can be used to register custom template tags and filters.

An example plugin which uses these two hooks can be found at https://github.com/simonw/datasette-plugin-demos or installed using `pip install datasette-plugin-demos`

Refs #14
2018-04-15 17:56:15 -07:00
Russ Garrett
efbb4e8337 Return HTTP 405 on InvalidUsage rather than 500
This also stops it filling up the logs. This happens for HEAD requests
at the moment - which perhaps should be handled better, but that's a
different issue.
2018-04-14 11:00:38 -07:00
Simon Willison
dd4491dd81
Update number of expected tables 2018-04-14 08:16:54 -07:00
Russ Garrett
f2b940d602 Link foreign keys which don't have labels
This renders unlabeled FKs as simple links. I can't see why this would
cause any major problems.

Also includes bonus fixes for two minor issues:

* In foreign key link hrefs the primary key was escaped using HTML
  escaping rather than URL escaping. This broke some non-integer PKs.
* Print tracebacks to console when handling 500 errors.
2018-04-14 07:59:59 -07:00
Russ Garrett
1cc5161089 Fix sqlite error when loading rows with no incoming FKs
This fixes `ERROR: conn=<sqlite3.Connection object at 0x10bbb9f10>, sql
= 'select ', params = {'id': '1'}` caused by an invalid query when
loading incoming FKs.

The error was ignored due to async but it still got printed to the
console.
2018-04-14 07:24:24 -07:00
Russ Garrett
3c985ec271
Allow custom units to be registered with Pint 2018-04-14 12:27:06 +01:00