Preserve duplicate column names in query results

Queries returning duplicate column names - e.g. joins between tables
sharing column names - silently lost values because rows were built
with dict(zip(keys, row)), where the last duplicate wins.

Later occurrences are now renamed with a numeric suffix: id, id
becomes id, id_2 - skipping any suffix that would collide with a
real column in the same query.

The new utils.dedupe_keys() helper transforms the key list once per
query, so the per-row dict construction is unchanged and there is no
measurable performance impact.

Applied in Database.query() (including the PRAGMA and RETURNING
paths), Table.rows_where(), Table.search() and the CLI's JSON output.
CSV, TSV and table output keep the original duplicate headers.

Closes #624
This commit is contained in:
Simon Willison 2026-07-05 21:20:39 -07:00 committed by GitHub
commit 07b603e562
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 128 additions and 4 deletions

View file

@ -104,3 +104,10 @@ def test_pks_and_rows_where_compound_pk(fresh_db):
(("number", 1), {"type": "number", "number": 1, "plusone": 2}),
(("number", 2), {"type": "number", "number": 2, "plusone": 3}),
]
def test_rows_where_duplicate_select_columns_are_deduped(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/624
fresh_db["t"].insert({"id": 1, "name": "Cleo"})
rows = list(fresh_db["t"].rows_where(select="id, id, name"))
assert rows == [{"id": 1, "id_2": 1, "name": "Cleo"}]