remote/sqlite-utils
1
0
Fork 0
mirror of https://github.com/simonw/sqlite-utils.git synced 2026-11-02 16:37:43 +01:00
Code Issues Projects Releases Packages Wiki Activity

Preserve carriage returns in quoted CSV/TSV fields (#895)

Browse source 
When reading CSV or TSV files through rows_from_file() or the CLI insert command, io.TextIOWrapper defaulted to universal-newline mode (newline=None), causing carriage returns (\r\n or lone \r) embedded inside quoted fields to be silently translated to \n before csv.reader could process them.

Setting newline='' in TextIOWrapper preserves carriage returns in quoted fields according to the Python standard library csv module specification.

Fixes #895
This commit is contained in:
agustin18 2026-10-03 23:08:20 +00:00
commit c27092220d
4 changed files with 49 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

2
sqlite_utils/utils.py
View file

@ -346,7 +346,7 @@ def rows_from_file(
return (json.loads(line) for line in fp if line.strip()), Format.NL
elif format == Format.CSV:
use_encoding: str = encoding or "utf-8-sig"
decoded_fp = io.TextIOWrapper(fp, encoding=use_encoding)
decoded_fp = io.TextIOWrapper(fp, encoding=use_encoding, newline="")
if dialect is not None:
reader = csv.DictReader(decoded_fp, dialect=dialect)
else: