github/datasette
1
0
Fork 0
mirror of https://github.com/simonw/datasette.git synced 2025-12-10 16:51:24 +01:00
Code Issues Projects Releases Packages Wiki Activity
522 commits 143 branches 163 tags 44 MiB
Commit graph

48 commits

Author SHA1 Message Date
Simon Willison
581b4c97ee
 		URLify URLs in custom SQL queries, closes #298 2018-07-23 20:56:32 -07:00
Simon Willison
700d83d8ad
 		?_json_infinity=1 for handling Infinity/-Infinity - fixes #332 2018-07-23 20:07:57 -07:00
Simon Willison
f24b49a1a8
 		New force_https_urls option, refs #333 2018-07-23 08:58:29 -07:00
Simon Willison
6e37f091ed
 		Support title/description for canned queries, closes #342 
Demo here: https://latest.datasette.io/fixtures/neighborhood_search
 2018-07-15 19:33:30 -07:00
Simon Willison
fc9b1cc136
 		New truncate_cells_html config for truncating cells, closes #330 2018-07-10 09:20:41 -07:00
Simon Willison
6541ce633e
 		Fix for row pages for tables with / in, closes #325 2018-07-07 22:21:51 -07:00
Simon Willison
71b46fd9f5
 		Cleaned up view constructors to accept just a datasette instance 2018-07-07 19:58:11 -07:00
Simon Willison
64c2fea8df
 		CSV export now respects --cors, fixes #326 2018-06-23 17:59:37 -07:00
Simon Willison
e843ce26e5
 		Correctly display empty strings in HTML table, closes #314 2018-06-21 08:26:18 -07:00
Simon Willison
54f805dca3
 		Advanced export box now obeys allow_csv_stream config - refs #266 2018-06-18 08:11:11 -07:00
Simon Willison
83f4ef7ec7
 		Improved UI for CSV/JSON export, closes #266 2018-06-17 23:05:18 -07:00
Simon Willison
fc3660cfad
 		Streaming mode for downloading all rows as a CSV (#315) 
* table.csv?_stream=1 to download all rows - refs #266

This option causes Datasette to serve ALL rows in the table, by internally
following the _next= pagination links and serving everything out as a stream.

Also added new config option, allow_csv_stream, which can be used to disable
this feature.

* New config option max_csv_mb limiting size of CSV export
 2018-06-17 20:21:02 -07:00
Simon Willison
0d7ba1ba67
 		Default to _labels=on on JSON/CSV links with foreign keys, refs #266 2018-06-17 15:56:55 -07:00
Simon Willison
ed631e690b
 		?_labels= and ?_label=COL to expand foreign keys in JSON/CSV 
These new querystring arguments can be used to request expanded foreign keys
in both JSON and CSV formats.

?_labels=on turns on expansions for ALL foreign key columns

?_label=COLUMN1&_label=COLUMN2 can be used to pick specific columns to expand

e.g. `Street_Tree_List.json?_label=qSpecies&_label=qLegalStatus`

    {
        "rowid": 233,
        "TreeID": 121240,
        "qLegalStatus": {
            "value" 2,
            "label": "Private"
        }
        "qSpecies": {
            "value": 16,
            "label": "Sycamore"
        }
        "qAddress": "91 Commonwealth Ave",
        ...
    }

The labels option also works for the HTML and CSV views.

HTML defaults to `?_labels=on`, so if you pass `?_labels=off` you can disable
foreign key expansion entirely - or you can use `?_label=COLUMN` to request
just specific columns.

If you expand labels on CSV you get additional columns in the output:

`/Street_Tree_List.csv?_label=qLegalStatus`

    rowid,TreeID,qLegalStatus,qLegalStatus_label...
    1,141565,1,Permitted Site...
    2,232565,2,Undocumented...

I also refactored the existing foreign key expansion code.

Closes #233. Refs #266.
 2018-06-16 15:18:57 -07:00
Simon Willison
0c22fa8f09
 		Cleaned up view_definition/table_definition code in table view 
Also moved those out of standard JSON into just the HTML template context
 2018-06-16 10:33:17 -07:00
Simon Willison
3a79ad98ea
 		Basic CSV export, refs #266 
Tables and custom SQL query results can now be exported as CSV.

The easiest way to do this is to use the .csv extension, e.g.

	/test_tables/facet_cities.csv

By default this is served as Content-Type: text/plain so you can see it in
your browser. If you want to download the file (using text/csv and with an
appropriate Content-Disposition: attachment header) you can do so like this:

	/test_tables/facet_cities.csv?_dl=1

We link to the CSV and downloadable CSV URLs from the table and query pages.

The links use ?_size=max and so by default will return 1,000 rows.

Also fixes #303 - table names ending in .json or .csv are now detected and
URLs are generated that look like this instead:

	/test_tables/table%2Fwith%2Fslashes.csv?_format=csv

The ?_format= option is available for everything else too, but we link to the
.csv / .json versions in most cases because they are aesthetically pleasing.
 2018-06-14 23:51:23 -07:00
Simon Willison
a246f476b4
 		Moved JsonDataView into views/special,py 2018-06-07 08:22:29 -07:00
Simon Willison
b0a95da963
 		Show more useful error message for SQL interrupted, closes #142 2018-05-28 14:24:19 -07:00
Simon Willison
76d11eb768
 		New ?_json=colname argument for returning unescaped JSON 
Also extracted docs for special JSON arguments into a new section.

Closes #31
 2018-05-28 11:08:39 -07:00
Simon Willison
276913b748
 		?_shape=arrayfirst, closes #287 2018-05-26 17:32:15 -07:00
Simon Willison
b463f60158
 		?_ttl= parameter and default_cache_ttl config 
Refs #285, Closes #289
 2018-05-26 15:17:33 -07:00
Simon Willison
f98e62fe5a
 		Fix for 500 error on /db?sql=x 2018-05-25 15:08:57 -07:00
Simon Willison
f722b0a730
 		allow_sql config option to disable custom SQL, closes #284 2018-05-24 22:50:50 -07:00
Simon Willison
50920cfe3d
 		allow_facet, allow_download, suggest_facets boolean --config 
Refs #284
 2018-05-24 18:12:27 -07:00
Simon Willison
81df47e8d9
 		Moved .execute() method from BaseView to Datasette class 
Also introduced new Results() class with results.truncated, results.description, results.rows
 2018-05-24 17:15:53 -07:00
Simon Willison
2bfd111d65
 		Faceting no longer breaks pagination, fixes #282 2018-05-23 06:41:14 -07:00
Russ Garrett
58b5a37dbb Refactor inspect logic 2018-05-22 07:03:06 -07:00
Simon Willison
f6183ff5fa
 		Renamed --limit to --config, added --help-config, closes #274 
Removed the --page_size= argument to datasette serve in favour of:

    datasette serve --config default_page_size:50 mydb.db

Added new help section:

    $ datasette --help-config
    Config options:
      default_page_size            Default page size for the table view
                                   (default=100)
      max_returned_rows            Maximum rows that can be returned from a table
                                   or custom query (default=1000)
      sql_time_limit_ms            Time limit for a SQL query in milliseconds
                                   (default=1000)
      default_facet_size           Number of values to return for requested facets
                                   (default=30)
      facet_time_limit_ms          Time limit for calculating a requested facet
                                   (default=200)
      facet_suggest_time_limit_ms  Time limit for calculating a suggested facet
                                   (default=50)
 2018-05-20 10:01:49 -07:00
Simon Willison
08f4b7658f
 		Show facets that timed out using new InterruptedError 
If the user requests some _facet= options that do not successfully execute in
the configured facet_time_limit_ms, we now show a warning message like this:

    These facets timed out: rowid, Title

To build this I had to clean up our SQLite interrupted logic. We now raise a
custom InterruptedError exception when SQLite terminates due to exceeding a
time limit.

In implementing this I found and fixed a logic error where invalid SQL was
being generated in some cases for our faceting calculations but the resulting
sqlite3.OperationalError had been incorrectly captured and treated as a
timeout.

Refs #255
Closes #269
 2018-05-17 23:11:23 -07:00
Simon Willison
cef9a9a870
 		--limit= mechanism plus new limits for facets 
Replaced the --max_returned_rows and --sql_time_limit_ms options to
"datasette serve" with a new --limit option, which supports a larger
list of limits.

Example usage:

	datasette serve --limit max_returned_rows:1000 \
		--limit sql_time_limit_ms:2500 \
		--limit default_facet_size:50 \
		--limit facet_time_limit_ms:1000 \
		--limit facet_suggest_time_limit_ms:500

New docs: https://datasette.readthedocs.io/en/latest/limits.html

Closes #270
Closes #264
 2018-05-17 22:08:26 -07:00
Simon Willison
566f2d31d6 Clarified relationship between metadata and _facet= facets, updated docs - refs @255 2018-05-16 08:27:24 -07:00
Simon Willison
b263da78e3 Reliable sort order for facets in Python 3.5, fixing test - refs #255 2018-05-16 08:27:24 -07:00
Simon Willison
91bf5f56bb Show enabled facets in flexbox columns, refs #255 2018-05-16 08:27:24 -07:00
Simon Willison
6d12580ed7 Foreign key facets are now expanded to labels, refs #255 2018-05-16 08:27:24 -07:00
Simon Willison
a892f9a0bd Use escape_sqlite() more consistently 2018-05-16 08:27:24 -07:00
Simon Willison
8a0bd68c39 Undid some slightly weird code formatting by 'black' 2018-05-16 08:27:24 -07:00
Simon Willison
514873c629 Fix bug with toggle_url on integer facets 2018-05-16 08:27:24 -07:00
Simon Willison
1dc94f6eaa Facets can now be toggled off again, refs #255 2018-05-16 08:27:24 -07:00
Simon Willison
142a550a99 Facet results now have "truncated" field 
To indicate if there was more than 20 distinct values. Refs #255
 2018-05-16 08:27:24 -07:00
Simon Willison
a82175276c _sort/_next links now use new path_with_replaced_args method 2018-05-16 08:27:24 -07:00
Simon Willison
2f81552cd2 Never suggest a facet if it only results in on option 2018-05-16 08:27:24 -07:00
Simon Willison
b0d2bb3e43 Facets no longer consider null values 2018-05-16 08:27:24 -07:00
Simon Willison
ddef229850 Initial implementation of suggested facets 
Causes tests to break at the moment
 2018-05-16 08:27:24 -07:00
Simon Willison
f36c9d4b4c Facet toggling now works for integer columns, refs #255 2018-05-16 08:27:24 -07:00
Simon Willison
de05cf21aa Facet "selected" key and toggle_url now toggles, refs #255 2018-05-16 08:27:24 -07:00
Simon Willison
cf1fe693e5 Used isort to re-order my imports 2018-05-14 00:04:23 -03:00
Simon Willison
3686385551 Ran black source formatting tool against new views/ and app.py 2018-05-14 00:04:23 -03:00
Simon Willison
1f69269fe9 Refactored views into new views/ modules, refs #256 2018-05-14 00:04:23 -03:00