github/datasette
1
0
Fork 0
mirror of https://github.com/simonw/datasette.git synced 2025-12-10 16:51:24 +01:00
Code Issues Projects Releases Packages Wiki Activity
501 commits 143 branches 163 tags 44 MiB
Commit graph

501 commits

This branch
This branch
All branches
Author SHA1 Message Date
Simon Willison
be018ce47f
 		Trying to get pytest-dist working on multiple cores 2018-06-23 17:55:00 -07:00
Simon Willison
7d7f5f61fd
 		Use extras_require so pip can install test dependencies 
https://github.com/pypa/pip/issues/1197#issuecomment-228939212
 2018-06-21 20:03:54 -07:00
Simon Willison
69b3a6c7dd
 		Run pytest manually 
python setup.py test appeared to still download a bunch of stuff
https://travis-ci.org/simonw/datasette/jobs/395306188
 2018-06-21 19:59:26 -07:00
Simon Willison
22831a1ac8
 		Don't use --download-cache argument 2018-06-21 19:53:22 -07:00
Simon Willison
b9046332c4
 		Tell pip to use explicit download cache 2018-06-21 19:51:10 -07:00
Simon Willison
749b6f0d74
 		Cache pip wheels between runs in Travis, refs #323 2018-06-21 16:57:49 -07:00
Simon Willison
e7566cc59d
 		Link to 0.23.1 release notes from news 2018-06-21 09:01:38 -07:00
Simon Willison
dea86b9fba
 		Changelog for 0.23.1 0.23.1 2018-06-21 08:37:13 -07:00
Simon Willison
0104dccca9
 		Updated Travis password, refs #317 2018-06-21 08:33:14 -07:00
Simon Willison
e843ce26e5
 		Correctly display empty strings in HTML table, closes #314 2018-06-21 08:26:18 -07:00
Simon Willison
120f20cd97
 		Allow "." in database filenames, closes #302 2018-06-21 08:21:09 -07:00
Simon Willison
97ae66ccab
 		404s ending in slash redirect to remove that slash, closes #309 2018-06-21 08:13:07 -07:00
Simon Willison
3b53eea382
 		Fixed incorrect display of compound primary keys with foreign key references 
Closes #319
 2018-06-21 07:56:28 -07:00
Simon Willison
3683a6b626
 		Docs + example of canned SQL query using || concatenation 
Closes #321
 2018-06-20 21:54:36 -07:00
Simon Willison
aeeb50f61b
 		Correctly display facets with value of 0 - fixes #318 
Also added comprehensive unit test for facet display HTML.
 2018-06-20 21:30:13 -07:00
Simon Willison
fdfbbbb9ee
 		Default 'expand labels' to checked in CSV advanced export 2018-06-20 09:15:33 -07:00
Simon Willison
e04f5b0d34
 		Release notes for 0.23 0.23 2018-06-18 08:11:57 -07:00
Simon Willison
54f805dca3
 		Advanced export box now obeys allow_csv_stream config - refs #266 2018-06-18 08:11:11 -07:00
Simon Willison
04c896160a
 		Renamed 'stream all records' to 'stream all rows', refs #266 2018-06-18 07:25:44 -07:00
Simon Willison
bb4a9fbf36
 		Docs for CSV export, refs #266 2018-06-18 07:12:21 -07:00
Simon Willison
398d29c287
 		Don't link to #export on custom query results 
The advanced CSV export options don't work for custom SQL queries.

Refs #266
 2018-06-17 23:14:48 -07:00
Simon Willison
83f4ef7ec7
 		Improved UI for CSV/JSON export, closes #266 2018-06-17 23:05:18 -07:00
Simon Willison
fc3660cfad
 		Streaming mode for downloading all rows as a CSV (#315) 
* table.csv?_stream=1 to download all rows - refs #266

This option causes Datasette to serve ALL rows in the table, by internally
following the _next= pagination links and serving everything out as a stream.

Also added new config option, allow_csv_stream, which can be used to disable
this feature.

* New config option max_csv_mb limiting size of CSV export
 2018-06-17 20:21:02 -07:00
Simon Willison
0d7ba1ba67
 		Default to _labels=on on JSON/CSV links with foreign keys, refs #266 2018-06-17 15:56:55 -07:00
Simon Willison
6a32684ebb
 		Release tagged versions to PyPI and now alias 
When a new tagged version is pushed, this should:

* bdist_wheel it and release to https://pypi.python.org/pypi/datasette
* Set an alias of v0-22-1.datasette.io for the deployed demo
 2018-06-17 15:24:47 -07:00
Simon Willison
c25c3e51c9
 		Link to latest.datasette.io from README 2018-06-17 15:02:20 -07:00
Simon Willison
958b75c698
 		Set Now deployment name with datasette publish, refs #313 2018-06-17 14:46:30 -07:00
Simon Willison
438244c285
 		Try using a different name for each Now deploy 
Refs #313
 2018-06-17 14:39:45 -07:00
Simon Willison
f0c17229b7
 		Deploy latest.datasette.io on commit to master - #313 
If the tests pass in Travis CI, deploy an instance containing Datasette's test
fixtures to https://latest.datasette.io/

Also set up an alias of truncated-commit-hash.datasette.io
 2018-06-17 14:24:39 -07:00
Simon Willison
db1e6bc182
 		--version-note for datasette, datasette publish and datasette package 
This is a relatively obscure new command-line argument that helps solve the
problem of showing accurate version information in deployed instances of
Datasette even if they were deployed directly from source code.

You can pass --version-note to datasette publish and package and it will then
in turn be passed to datasette when it starts:

    datasette --version-note=hello fixtures.db

Now if you visit /-/versions.json you will see this:

    {
        "datasette": {
            "note": "hello",
            "version": "0+unknown"
        },
        "python": {
            "full": "3.6.5 (default, Jun  6 2018, 19:19:24) \n[GCC 6.3.0 20170516]",
            "version": "3.6.5"
        },
        ...
    }

I plan to use this in some Travis CI configuration, refs #313
 2018-06-17 14:19:39 -07:00
Simon Willison
538565de3a
 		datasette publish now --token=X argument 
Lets you specify the auth token to use when deploying to Now.

Tokens can be created at https://zeit.co/account/tokens
 2018-06-17 14:16:08 -07:00
Simon Willison
6a32df334d
 		Improved fixtures to support publication 
The fixtures database created by our unit tests makes for a good "live" demo
of Datasette in action.

I've improved the metadata it ships with to better support this use-case.

I've also improved the mechanism for writing out fixtures: you can do this:

	python tests/fixtures.py fixtures.db

To get just the fixtures database written out... or you can do this:

	python tests/fixtures.py fixtures.db fixtures.json

To get metadata which you can then serve like so:

	datasette fixtures.db -m fixtures.json

Refs #313
 2018-06-17 12:16:04 -07:00
Simon Willison
0357774c94
 		Renamed test_tables.db to fixtures.db in unit tests 2018-06-17 11:34:16 -07:00
Simon Willison
ed631e690b
 		?_labels= and ?_label=COL to expand foreign keys in JSON/CSV 
These new querystring arguments can be used to request expanded foreign keys
in both JSON and CSV formats.

?_labels=on turns on expansions for ALL foreign key columns

?_label=COLUMN1&_label=COLUMN2 can be used to pick specific columns to expand

e.g. `Street_Tree_List.json?_label=qSpecies&_label=qLegalStatus`

    {
        "rowid": 233,
        "TreeID": 121240,
        "qLegalStatus": {
            "value" 2,
            "label": "Private"
        }
        "qSpecies": {
            "value": 16,
            "label": "Sycamore"
        }
        "qAddress": "91 Commonwealth Ave",
        ...
    }

The labels option also works for the HTML and CSV views.

HTML defaults to `?_labels=on`, so if you pass `?_labels=off` you can disable
foreign key expansion entirely - or you can use `?_label=COLUMN` to request
just specific columns.

If you expand labels on CSV you get additional columns in the output:

`/Street_Tree_List.csv?_label=qLegalStatus`

    rowid,TreeID,qLegalStatus,qLegalStatus_label...
    1,141565,1,Permitted Site...
    2,232565,2,Undocumented...

I also refactored the existing foreign key expansion code.

Closes #233. Refs #266.
 2018-06-16 15:18:57 -07:00
Simon Willison
0c22fa8f09
 		Cleaned up view_definition/table_definition code in table view 
Also moved those out of standard JSON into just the HTML template context
 2018-06-16 10:33:17 -07:00
Simon Willison
7e0caa1e62
 		Extract string-to-bool logic into utils.py 2018-06-16 09:44:31 -07:00
Simon Willison
d0a578c0fc
 		Switch back from python:3.6-slim-stretch to python:3.6 
Turns out slim-stretch doesn't include gcc which means it can't build various
Sanic dependencies. So `datasette publish now ...` was broken.

Fixes #310
 2018-06-16 09:28:55 -07:00
Simon Willison
5bda4a477c
 		Fixed CSV tests - Python 3.6.5 and 3.6.3 apparently differ 
The test used to expect CSV to come back like this:

	hello
	world
	""

With the final blank value encoded in quotes.

Judging by Travis failures, this behaviour changed between Python 3.6.3 and
3.6.5:

	https://travis-ci.org/simonw/datasette/jobs/392586661
 2018-06-15 00:01:48 -07:00
Simon Willison
3a79ad98ea
 		Basic CSV export, refs #266 
Tables and custom SQL query results can now be exported as CSV.

The easiest way to do this is to use the .csv extension, e.g.

	/test_tables/facet_cities.csv

By default this is served as Content-Type: text/plain so you can see it in
your browser. If you want to download the file (using text/csv and with an
appropriate Content-Disposition: attachment header) you can do so like this:

	/test_tables/facet_cities.csv?_dl=1

We link to the CSV and downloadable CSV URLs from the table and query pages.

The links use ?_size=max and so by default will return 1,000 rows.

Also fixes #303 - table names ending in .json or .csv are now detected and
URLs are generated that look like this instead:

	/test_tables/table%2Fwith%2Fslashes.csv?_format=csv

The ?_format= option is available for everything else too, but we link to the
.csv / .json versions in most cases because they are aesthetically pleasing.
 2018-06-14 23:51:23 -07:00
Simon Willison
a246f476b4
 		Moved JsonDataView into views/special,py 2018-06-07 08:22:29 -07:00
Simon Willison
864328e074
 		Test client wrapper removing need for gather_request - refs #272 
As part of decoupling from Sanic, this will make it easier to run tests
against ASGI instead.
 2018-06-04 20:53:12 -07:00
Simon Willison
b252bdb740
 		Documented how to set multiple --config at once 2018-06-04 18:45:47 -07:00
Simon Willison
29edbe90ff
 		New cache_size_kb config for SQLite, closes #304 2018-06-04 09:02:07 -07:00
Simon Willison
9277c6236a
 		Fixed tests I broke in b18e45158 2018-05-31 07:47:22 -07:00
Simon Willison
7381af85e6
 		Hide sort select on larger screens, closes #300 2018-05-31 07:43:06 -07:00
Simon Willison
a80e1d50b9
 		Show error on 'datasette publish heroku --spatialite', refs #301 2018-05-31 07:21:27 -07:00
Simon Willison
b18e451585
 		datasette publish/package --spatialite, closes #243 
New command-line argument which causes SpatiaLite to be installed and
configured for the published Datasette.

	datasette publish now --spatialite mydb.db
 2018-05-31 07:16:50 -07:00
Simon Willison
969771770f
 		Upgrade pytest to 3.6.0 
https://github.com/pytest-dev/pytest/issues/1875 made it impossible to declare
a function as a fixture multiple times, which we were doing across different
modules. The fix was to move our @pytest.fixture calls into decorators in the
tests/fixtures.py module.
 2018-05-31 06:42:37 -07:00
Simon Willison
16398641d9
 		SQL syntax highlighting in docs 2018-05-31 06:17:53 -07:00
Simon Willison
d69ebce533
 		Fixed broken :ref: 2018-05-29 20:11:24 -07:00