Commit graph

1,093 commits

Author SHA1 Message Date
Simon Willison
7a7e4b2ed8
Started unit tests for row/table HTML pages
Refs #167

Thanks to the new tests, spotted and fixed a bug where pages that were
supposed to have 100 things on them were actually displaying 101.
2017-12-09 15:32:54 -08:00
Simon Willison
b01304d707
Updated tests I broke in c195ee4 2017-12-09 10:39:19 -08:00
Simon Willison
cbfd6b745e
Publish should now work if /tmp is on different device
Fixes #141
2017-12-08 08:06:24 -08:00
Simon Willison
709f4f2798
Fixed bug with detecting FTS tables
Closes #135
2017-12-06 20:54:37 -08:00
Simon Willison
0cfd7ce59d
Allow WITH query (previously we required SELECT at start)
Fixes #161
2017-12-03 20:51:31 -08:00
Simon Willison
8ab3a169d4
CSS styling hooks as classes on the body
Refs #153

Every template now gets CSS classes in the body designed to support custom
styling.

The index template (the top level page at /) gets this:

    <body class="index">

The database template (/dbname/) gets this:

    <body class="db db-dbname">

The table template (/dbname/tablename) gets:

    <body class="table db-dbname table-tablename">

The row template (/dbname/tablename/rowid) gets:

    <body class="row db-dbname table-tablename">

The db-x and table-x classes use the database or table names themselves IF
they are valid CSS identifiers. If they aren't, we strip any invalid
characters out and append a 6 character md5 digest of the original name, in
order to ensure that multiple tables which resolve to the same stripped
character version still have different CSS classes.

Some examples (extracted from the unit tests):

    "simple" => "simple"
    "MixedCase" => "MixedCase"
    "-no-leading-hyphens" => "no-leading-hyphens-65bea6"
    "_no-leading-underscores" => "no-leading-underscores-b921bc"
    "no spaces" => "no-spaces-7088d7"
    "-" => "336d5e"
    "no $ characters" => "no--characters-59e024"
2017-11-29 23:09:54 -08:00
Simon Willison
b67890d15d
Auto-link column values that look like URLs
Refs #153
2017-11-29 09:05:24 -08:00
Simon Willison
c5c923d93c
Don't incorrectly detect VIEWs as supporting FTS 2017-11-24 14:51:00 -08:00
Simon Willison
a802cbee74
Search now applies to current filters
Combined search into the same form as filters.

Closes #133
2017-11-24 14:29:54 -08:00
Simon Willison
b450778391
Added column__not=blah filter
Closes #148
2017-11-23 14:09:50 -08:00
Simon Willison
a30c5b220c
Fixed bug on rows page, added unit test
Rows page for rows that linked to the same table in more
than one columns were display incorrectly. Fixed that and added a test.

Also introduced /db/table/row-pk.json?_extras=foreign_key_tables

This is used by the new unit test, but is the first example of a new
?_extras=comma-separated-list pattern I am introducing.
2017-11-23 13:09:45 -08:00
Simon Willison
ef3eacf622
Select option for removing filters 2017-11-23 12:32:54 -08:00
Simon Willison
0071b5d6f5
Added UI for editing table filters
Refs #86
2017-11-22 20:33:55 -08:00
Simon Willison
53534b6e9d
Even more complicated redirect scheme
This:

    ?_filter_column_1=name&_filter_op_1=contains&_filter_value_1=hello
    &_filter_column_2=age&_filter_op_2=gte&_filter_value_2=12

Now redirects to this:

    ?name__contains=hello&age__gte=12

This is needed for the filter editing interface, refs #86
2017-11-22 20:03:46 -08:00
Simon Willison
c8e7c85fd3
Updated test_database_page test 2017-11-22 12:25:51 -08:00
Simon Willison
a9b9d42791
Mark FTS-related tables as 'hidden' in inspect()
Refs #129
2017-11-21 12:55:59 -08:00
Simon Willison
b4e6211a97
Refactored filter logic and added human descriptions - refs #86 2017-11-19 22:03:24 -08:00
Simon Willison
a5881e105a
?_filter_column=col&_filter_op=isnull__1 redirect
if filter_op contains a __ the value is set to the right hand side.

e.g.

    ?_filter_column=col&_filter_op=isnull__1&_filter_value=x

Redirects to:

    ?col__isnull=1

Refs #86
2017-11-19 12:33:30 -08:00
Simon Willison
386fb11d42
?_filter_column=col&_filter_op=op&_filter_value=value redirect
Part of implementing the filters UI (refs #86) - the following:

    /trees/Trees?_filter_column=SiteOrder&_filter_op=gt&_filter_value=2

Now redirects to this;

    /trees/Trees?SiteOrder__gt=2
2017-11-19 12:25:29 -08:00
Simon Willison
eed6a0fe36
Implemented ?_search=XXX + UI if a FTS table is detected
Closes #131
2017-11-19 08:59:26 -08:00
Simon Willison
45e502aace Added unit tests for inspect() foreign key detection
Used them to fix a bug with it.

Refs #85
2017-11-17 08:08:11 -08:00
Simon Willison
084350b0f1 Switched to gather_request=False for Sanic tests
Gets rid of those ugly _, response = lines.
2017-11-17 06:53:37 -08:00
Ray N
ed2b3f25be add support for ?field__isnull=1 (#107)
* add support for ?field__isnull=1

* Add unit test and conditional formatting for ?field__isnull
2017-11-17 05:29:22 -08:00
Simon Willison
9cb69cbd45 New ?_sql_time_limit_ms=10 argument to database and table page
Allows callers to opt for a lower time limit.

Closes #95
2017-11-14 18:55:10 -08:00
Simon Willison
0b8c1b0a6d Test for sql_time_limit_ms + sqlite_functions mechanism
Added a unit test for the sql_time_limit_ms option.

To test this, I needed to add a custom SQLite sleep() function. I've added a
simple mechanism to the Datasette class for registering custom functions.

I also had to modify the sqlite_timelimit() function. It makes use of a magic
value, N, which is the number of SQLite virtual machine instructions that
should execute in between calls to my termination decision function.

The value of N was not finely grained enough for my test to work - so I've
added logic that says that if the time limit is less than 50ms, N is set to 1.
This got the tests working.

Refs #95
2017-11-14 18:43:34 -08:00
Simon Willison
cbb59e3801 Handle tables with names that start with digits 2017-11-14 18:03:21 -08:00
Simon Willison
50e817801f Fixed #83
Turns out we had a redirect bug as well.
2017-11-13 16:44:43 -08:00
Simon Willison
44a199a062 Stop using sqlite WITH RECURSIVE in our tests
The version of Python 3 running in Travis CI doesn't support this.
2017-11-13 14:15:21 -08:00
Simon Willison
847f3e0c92 Implemented offset/limit pagination for views
Closes #70
2017-11-13 13:10:55 -08:00
Simon Willison
7dac1c05cd Improved pagination
Closes #78
2017-11-13 12:34:56 -08:00
Simon Willison
8252e71da4 Limit on max rows returned, controlled by --max_returned_rows option
If someone executes 'select * from table' against a table with a million rows
in it, we could run into problems: just serializing that much data as JSON is
likely to lock up the server.

Solution: we now have a hard limit on the maximum number of rows that can be
returned by a query. If that limit is exceeded, the server will return a
`"truncated": true` field in the JSON.

This limit can be optionally controlled by the new `--max_returned_rows`
option. Setting that option to 0 disables the limit entirely.

Closes #69
2017-11-13 11:33:01 -08:00
Simon Willison
26370b14d8 Handle table names with slashes in them
e.g. https://datasette-wdlexdiaoz.now.sh/fivethirtyeight-75d605c/bob-ross%2Felements-by-episode.csv?CABIN=1&BUSHES=1&CLOUDS=1
2017-11-12 15:17:00 -08:00
Simon Willison
666aa03253 Improved error handling
Invalid SQL now shows a special error.html template, and is covered by tests.
2017-11-12 13:16:15 -08:00
Simon Willison
ff2ab9dc7d Views now show their SQL, are handled a bit better
Refs #66
2017-11-12 12:32:25 -08:00
Simon Willison
8acdc2fd14 Test for table with space in name
Tests code in b51836f846
2017-11-12 12:08:32 -08:00
Simon Willison
59580d02da Implemented custom SQL via textarea
Closes #65
2017-11-11 18:35:35 -08:00
Simon Willison
fa42a56c6a Bulked out table test a bit
I'm closing #50 - more tests will be added in the future, but the framework
is neatly in place for them now.
2017-11-11 14:22:47 -08:00
Simon Willison
e9e1def4c0 Revised JSON design a bit
Closes #63
2017-11-11 14:20:00 -08:00
Simon Willison
407795b612 Initial unit tests against our Sanic app
Refs #50

I had to disable the build metadata function to get these tests to work
sensibly. I need to completely rethink how that mechanism works.
2017-11-11 09:47:59 -08:00
Simon Willison
3280972c89 Fixed tests I broke earlier
Broke these tests in 21c9c04310
2017-11-11 08:52:17 -08:00
Simon Willison
21c9c04310 Implemented cursor-based pagination for table view
Closes #5
2017-11-10 12:41:14 -08:00
Simon Willison
a8a293cd71 Refactored util functions into new utils module 2017-11-10 11:25:54 -08:00
Simon Willison
40d3b3eae6 Refactored tests into new tests/ folder
Guided by https://docs.pytest.org/en/latest/goodpractices.html
2017-11-10 10:48:16 -08:00