github/datasette
1
0
Fork 0
mirror of https://github.com/simonw/datasette.git synced 2025-12-10 16:51:24 +01:00
Code Issues Projects Releases Packages Wiki Activity
516 commits 143 branches 163 tags 44 MiB
Commit graph

319 commits

Author SHA1 Message Date
Simon Willison
4e82d0e64d
 		Removed unused imports 2018-07-23 08:39:00 -07:00
Simon Willison
6e37f091ed
 		Support title/description for canned queries, closes #342 
Demo here: https://latest.datasette.io/fixtures/neighborhood_search
 2018-07-15 19:33:30 -07:00
Russ Garrett
58fec99ab0 Allow app names for datasette publish heroku 
Lets you supply the `-n` parameter for Heroku deploys, which also lets
you update existing Heroku deployments.
 2018-07-14 07:04:42 -07:00
Simon Willison
fc9b1cc136
 		New truncate_cells_html config for truncating cells, closes #330 2018-07-10 09:20:41 -07:00
Simon Willison
2db2ae4f21
 		Show custom error message if SpatiaLite needed, closes #331 2018-07-10 08:13:27 -07:00
Simon Willison
17863d108b
 		datasette publish heroku now supports --extra-options, closes #334 2018-07-09 18:45:56 -07:00
Simon Willison
6541ce633e
 		Fix for row pages for tables with / in, closes #325 2018-07-07 22:21:51 -07:00
Simon Willison
8ac71a6127
 		Removed rogue print statement left over from #309 2018-07-07 20:09:45 -07:00
Simon Willison
71b46fd9f5
 		Cleaned up view constructors to accept just a datasette instance 2018-07-07 19:58:11 -07:00
Simon Willison
d08faa8987
 		Fix for weird nested exception in RequestTimeout 
I saw this error:

    sanic.exceptions.RequestTimeout: Request Timeout

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/Users/simonw/Dropbox/Development/datasette/venv/lib/python3.6/site-packages/sanic/handlers.py", line 82, in response
        response = handler(request=request, exception=exception)
      File "/Users/simonw/Dropbox/Development/datasette/datasette/app.py", line 512, in on_exception
        if request.path.split("?")[0].endswith(".json"):
    AttributeError: 'NoneType' object has no attribute 'path'

Strangely "if request and request.path..." did not work here, because the
Sanic Request class extends builtins.dict and hence evaluates to False if it
has no headers.
 2018-07-07 19:58:11 -07:00
Simon Willison
64c2fea8df
 		CSV export now respects --cors, fixes #326 2018-06-23 17:59:37 -07:00
Simon Willison
e843ce26e5
 		Correctly display empty strings in HTML table, closes #314 2018-06-21 08:26:18 -07:00
Simon Willison
120f20cd97
 		Allow "." in database filenames, closes #302 2018-06-21 08:21:09 -07:00
Simon Willison
97ae66ccab
 		404s ending in slash redirect to remove that slash, closes #309 2018-06-21 08:13:07 -07:00
Simon Willison
3b53eea382
 		Fixed incorrect display of compound primary keys with foreign key references 
Closes #319
 2018-06-21 07:56:28 -07:00
Simon Willison
aeeb50f61b
 		Correctly display facets with value of 0 - fixes #318 
Also added comprehensive unit test for facet display HTML.
 2018-06-20 21:30:13 -07:00
Simon Willison
fdfbbbb9ee
 		Default 'expand labels' to checked in CSV advanced export 2018-06-20 09:15:33 -07:00
Simon Willison
54f805dca3
 		Advanced export box now obeys allow_csv_stream config - refs #266 2018-06-18 08:11:11 -07:00
Simon Willison
04c896160a
 		Renamed 'stream all records' to 'stream all rows', refs #266 2018-06-18 07:25:44 -07:00
Simon Willison
398d29c287
 		Don't link to #export on custom query results 
The advanced CSV export options don't work for custom SQL queries.

Refs #266
 2018-06-17 23:14:48 -07:00
Simon Willison
83f4ef7ec7
 		Improved UI for CSV/JSON export, closes #266 2018-06-17 23:05:18 -07:00
Simon Willison
fc3660cfad
 		Streaming mode for downloading all rows as a CSV (#315) 
* table.csv?_stream=1 to download all rows - refs #266

This option causes Datasette to serve ALL rows in the table, by internally
following the _next= pagination links and serving everything out as a stream.

Also added new config option, allow_csv_stream, which can be used to disable
this feature.

* New config option max_csv_mb limiting size of CSV export
 2018-06-17 20:21:02 -07:00
Simon Willison
0d7ba1ba67
 		Default to _labels=on on JSON/CSV links with foreign keys, refs #266 2018-06-17 15:56:55 -07:00
Simon Willison
db1e6bc182
 		--version-note for datasette, datasette publish and datasette package 
This is a relatively obscure new command-line argument that helps solve the
problem of showing accurate version information in deployed instances of
Datasette even if they were deployed directly from source code.

You can pass --version-note to datasette publish and package and it will then
in turn be passed to datasette when it starts:

    datasette --version-note=hello fixtures.db

Now if you visit /-/versions.json you will see this:

    {
        "datasette": {
            "note": "hello",
            "version": "0+unknown"
        },
        "python": {
            "full": "3.6.5 (default, Jun  6 2018, 19:19:24) \n[GCC 6.3.0 20170516]",
            "version": "3.6.5"
        },
        ...
    }

I plan to use this in some Travis CI configuration, refs #313
 2018-06-17 14:19:39 -07:00
Simon Willison
538565de3a
 		datasette publish now --token=X argument 
Lets you specify the auth token to use when deploying to Now.

Tokens can be created at https://zeit.co/account/tokens
 2018-06-17 14:16:08 -07:00
Simon Willison
ed631e690b
 		?_labels= and ?_label=COL to expand foreign keys in JSON/CSV 
These new querystring arguments can be used to request expanded foreign keys
in both JSON and CSV formats.

?_labels=on turns on expansions for ALL foreign key columns

?_label=COLUMN1&_label=COLUMN2 can be used to pick specific columns to expand

e.g. `Street_Tree_List.json?_label=qSpecies&_label=qLegalStatus`

    {
        "rowid": 233,
        "TreeID": 121240,
        "qLegalStatus": {
            "value" 2,
            "label": "Private"
        }
        "qSpecies": {
            "value": 16,
            "label": "Sycamore"
        }
        "qAddress": "91 Commonwealth Ave",
        ...
    }

The labels option also works for the HTML and CSV views.

HTML defaults to `?_labels=on`, so if you pass `?_labels=off` you can disable
foreign key expansion entirely - or you can use `?_label=COLUMN` to request
just specific columns.

If you expand labels on CSV you get additional columns in the output:

`/Street_Tree_List.csv?_label=qLegalStatus`

    rowid,TreeID,qLegalStatus,qLegalStatus_label...
    1,141565,1,Permitted Site...
    2,232565,2,Undocumented...

I also refactored the existing foreign key expansion code.

Closes #233. Refs #266.
 2018-06-16 15:18:57 -07:00
Simon Willison
0c22fa8f09
 		Cleaned up view_definition/table_definition code in table view 
Also moved those out of standard JSON into just the HTML template context
 2018-06-16 10:33:17 -07:00
Simon Willison
7e0caa1e62
 		Extract string-to-bool logic into utils.py 2018-06-16 09:44:31 -07:00
Simon Willison
d0a578c0fc
 		Switch back from python:3.6-slim-stretch to python:3.6 
Turns out slim-stretch doesn't include gcc which means it can't build various
Sanic dependencies. So `datasette publish now ...` was broken.

Fixes #310
 2018-06-16 09:28:55 -07:00
Simon Willison
3a79ad98ea
 		Basic CSV export, refs #266 
Tables and custom SQL query results can now be exported as CSV.

The easiest way to do this is to use the .csv extension, e.g.

	/test_tables/facet_cities.csv

By default this is served as Content-Type: text/plain so you can see it in
your browser. If you want to download the file (using text/csv and with an
appropriate Content-Disposition: attachment header) you can do so like this:

	/test_tables/facet_cities.csv?_dl=1

We link to the CSV and downloadable CSV URLs from the table and query pages.

The links use ?_size=max and so by default will return 1,000 rows.

Also fixes #303 - table names ending in .json or .csv are now detected and
URLs are generated that look like this instead:

	/test_tables/table%2Fwith%2Fslashes.csv?_format=csv

The ?_format= option is available for everything else too, but we link to the
.csv / .json versions in most cases because they are aesthetically pleasing.
 2018-06-14 23:51:23 -07:00
Simon Willison
a246f476b4
 		Moved JsonDataView into views/special,py 2018-06-07 08:22:29 -07:00
Simon Willison
29edbe90ff
 		New cache_size_kb config for SQLite, closes #304 2018-06-04 09:02:07 -07:00
Simon Willison
7381af85e6
 		Hide sort select on larger screens, closes #300 2018-05-31 07:43:06 -07:00
Simon Willison
a80e1d50b9
 		Show error on 'datasette publish heroku --spatialite', refs #301 2018-05-31 07:21:27 -07:00
Simon Willison
b18e451585
 		datasette publish/package --spatialite, closes #243 
New command-line argument which causes SpatiaLite to be installed and
configured for the published Datasette.

	datasette publish now --spatialite mydb.db
 2018-05-31 07:16:50 -07:00
Simon Willison
b0a95da963
 		Show more useful error message for SQL interrupted, closes #142 2018-05-28 14:24:19 -07:00
Simon Willison
76d11eb768
 		New ?_json=colname argument for returning unescaped JSON 
Also extracted docs for special JSON arguments into a new section.

Closes #31
 2018-05-28 11:08:39 -07:00
Simon Willison
02870e5731
 		Filter out duplicate JS/CSS URLs, refs #291 (testme) 2018-05-27 01:45:03 -07:00
Simon Willison
7944a8b0de
 		Added num_sql_threads config option, closes #285 2018-05-26 17:43:22 -07:00
Simon Willison
276913b748
 		?_shape=arrayfirst, closes #287 2018-05-26 17:32:15 -07:00
Simon Willison
b463f60158
 		?_ttl= parameter and default_cache_ttl config 
Refs #285, Closes #289
 2018-05-26 15:17:33 -07:00
Simon Willison
f98e62fe5a
 		Fix for 500 error on /db?sql=x 2018-05-25 15:08:57 -07:00
Simon Willison
b9b9358346
 		boolean --config are now case insensitive, refs #284 2018-05-24 22:53:58 -07:00
Simon Willison
f722b0a730
 		allow_sql config option to disable custom SQL, closes #284 2018-05-24 22:50:50 -07:00
Simon Willison
50920cfe3d
 		allow_facet, allow_download, suggest_facets boolean --config 
Refs #284
 2018-05-24 18:12:27 -07:00
Simon Willison
81df47e8d9
 		Moved .execute() method from BaseView to Datasette class 
Also introduced new Results() class with results.truncated, results.description, results.rows
 2018-05-24 17:15:53 -07:00
Ravi Kotecha
bd30c696e1 Build Dockerfile with recent Sqlite + Spatialite (#280) 
Closes #278

```bash
  $ docker run --rm -it datasette spatialite
  SpatiaLite version ..: 4.4.0-RC0	Supported Extensions:
    - 'VirtualShape'	[direct Shapefile access]
    - 'VirtualDbf'		[direct DBF access]
    - 'VirtualXL'		[direct XLS access]
    - 'VirtualText'		[direct CSV/TXT access]
    - 'VirtualNetwork'	[Dijkstra shortest path]
    - 'RTree'		[Spatial Index - R*Tree]
    - 'MbrCache'		[Spatial Index - MBR cache]
    - 'VirtualSpatialIndex'	[R*Tree metahandler]
    - 'VirtualElementary'	[ElemGeoms metahandler]
    - 'VirtualKNN'	[K-Nearest Neighbors metahandler]
    - 'VirtualXPath'	[XML Path Language - XPath]
    - 'VirtualFDO'		[FDO-OGR interoperability]
    - 'VirtualGPKG'	[OGC GeoPackage interoperability]
    - 'VirtualBBox'		[BoundingBox tables]
    - 'SpatiaLite'		[Spatial SQL - OGC]
  PROJ.4 version ......: Rel. 4.9.3, 15 August 2016
  GEOS version ........: 3.5.1-CAPI-1.9.1 r4246
  TARGET CPU ..........: x86_64-linux-gnu
  the SPATIAL_REF_SYS table already contains some row(s)
  SQLite version ......: 3.23.1
  Enter ".help" for instructions
  SQLite version 3.23.1 2018-04-10 17:39:29
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  spatialite>
```

```bash
$ docker run --rm -it datasette python -c "import sqlite3; print(sqlite3.sqlite_version)"
3.23.1
```

Also updates the query used to check for FTS5 as the old version wasn't
detecting FTS5 for some reason.
 2018-05-23 10:43:34 -07:00
Simon Willison
2bfd111d65
 		Faceting no longer breaks pagination, fixes #282 2018-05-23 06:41:14 -07:00
Robert Gieseke
49f317752c Move version info back to separate module 2018-05-22 12:35:21 -07:00
Robert Gieseke
fc47dacbbd Add __version_info__ derived from __version__ 
This might be tuple of more than two values (major and minor
version) if commits have been made after a release.
 2018-05-22 12:35:21 -07:00