Preserve duplicate column names in query results

Queries returning duplicate column names - e.g. joins between tables
sharing column names - silently lost values because rows were built
with dict(zip(keys, row)), where the last duplicate wins.

Later occurrences are now renamed with a numeric suffix: id, id
becomes id, id_2 - skipping any suffix that would collide with a
real column in the same query.

The new utils.dedupe_keys() helper transforms the key list once per
query, so the per-row dict construction is unchanged and there is no
measurable performance impact.

Applied in Database.query() (including the PRAGMA and RETURNING
paths), Table.rows_where(), Table.search() and the CLI's JSON output.
CSV, TSV and table output keep the original duplicate headers.

Closes #624
This commit is contained in:
Simon Willison 2026-07-05 21:20:39 -07:00 committed by GitHub
commit 07b603e562
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 128 additions and 4 deletions

View file

@ -83,3 +83,20 @@ def test_maximize_csv_field_size_limit():
)
def test_flatten(input, expected):
assert utils.flatten(input) == expected
@pytest.mark.parametrize(
"input,expected",
(
([], []),
(["id", "name"], ["id", "name"]),
(["id", "id"], ["id", "id_2"]),
(["id", "id", "id"], ["id", "id_2", "id_3"]),
# A renamed duplicate must not clobber a real column called id_2
(["id", "id", "id_2"], ["id", "id_3", "id_2"]),
(["id_2", "id", "id"], ["id_2", "id", "id_3"]),
(["id", "id", "id_2", "id_2"], ["id", "id_3", "id_2", "id_2_2"]),
),
)
def test_dedupe_keys(input, expected):
assert utils.dedupe_keys(input) == expected