Use REAL not FLOAT as SQLite column type (#680)

* Use REAL not FLOAT as SQLite column type, refs #645
* Fix for REAL columns by CSV --detect-types

Refs https://github.com/simonw/sqlite-utils/issues/645#issuecomment-3568947189

* Removed note about strict and REAL
This commit is contained in:
Simon Willison 2025-11-23 21:37:59 -08:00 committed by GitHub
commit c872d27bb3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 61 additions and 37 deletions

View file

@ -10,6 +10,7 @@ Unreleased
----------
- The ``table.insert_all()`` and ``table.upsert_all()`` methods can now accept an iterator of lists or tuples as an alternative to dictionaries. The first item should be a list/tuple of column names. See :ref:`python_api_insert_lists` for details. (:issue:`672`)
- **Breaking change:** The default floating point column type has been changed from ``FLOAT`` to ``REAL``, which is the correct SQLite type for floating point values. This affects auto-detected columns when inserting data. (:issue:`645`)
.. _v4_0a0:

View file

@ -457,8 +457,8 @@ See :ref:`cli_transform_table`.
--rename column2 column_renamed
Options:
--type <TEXT CHOICE>... Change column type to INTEGER, TEXT, FLOAT or
BLOB
--type <TEXT CHOICE>... Change column type to INTEGER, TEXT, FLOAT,
REAL or BLOB
--drop TEXT Drop this column
--rename <TEXT TEXT>... Rename this column to X
-o, --column-order TEXT Reorder columns
@ -1142,7 +1142,7 @@ See :ref:`cli_add_column`.
::
Usage: sqlite-utils add-column [OPTIONS] PATH TABLE COL_NAME
[[integer|int|float|text|str|blob|bytes]]
[[integer|int|float|real|text|str|blob|bytes]]
Add a column to the specified table

View file

@ -1247,7 +1247,7 @@ To stop inserting after a specified number of records - useful for getting a fas
A progress bar is displayed when inserting data from a file. You can hide the progress bar using the ``--silent`` option.
By default every column inserted from a CSV or TSV file will be of type ``TEXT``. To automatically detect column types - resulting in a mix of ``TEXT``, ``INTEGER`` and ``FLOAT`` columns, use the ``--detect-types`` option (or its shortcut ``-d``).
By default every column inserted from a CSV or TSV file will be of type ``TEXT``. To automatically detect column types - resulting in a mix of ``TEXT``, ``INTEGER`` and ``REAL`` columns, use the ``--detect-types`` option (or its shortcut ``-d``).
For example, given a ``creatures.csv`` file containing this:
@ -1274,7 +1274,7 @@ Will produce this schema:
CREATE TABLE "creatures" (
"name" TEXT,
"age" INTEGER,
"weight" FLOAT
"weight" REAL
);
You can set the ``SQLITE_UTILS_DETECT_TYPES`` environment variable if you want ``--detect-types`` to be the default behavior:
@ -1589,7 +1589,7 @@ This will result in the following schema:
CREATE TABLE "images" (
"path" TEXT PRIMARY KEY,
"md5" TEXT,
"mtime" FLOAT
"mtime" REAL
);
Note that there's no ``content`` column here at all - if you specify custom columns using ``-c`` you need to include ``-c content`` to create that column.
@ -1888,8 +1888,8 @@ The type of the returned values will be taken into account when creating the new
CREATE TABLE "places" (
"location" TEXT,
"latitude" FLOAT,
"longitude" FLOAT
"latitude" REAL,
"longitude" REAL
);
The code function can also return ``None``, in which case its output will be ignored. You can drop the original column at the end of the operation by adding ``--drop``.

View file

@ -548,7 +548,7 @@ This will create a table with the following schema:
"id" INTEGER PRIMARY KEY,
"name" TEXT,
"age" INTEGER,
"weight" FLOAT
"weight" REAL
)
.. _python_api_explicit_create:
@ -1271,11 +1271,11 @@ You can specify the ``col_type`` argument either using a SQLite type as a string
The ``col_type`` is optional - if you omit it the type of ``TEXT`` will be used.
SQLite types you can specify are ``"TEXT"``, ``"INTEGER"``, ``"FLOAT"`` or ``"BLOB"``.
SQLite types you can specify are ``"TEXT"``, ``"INTEGER"``, ``"FLOAT"``, ``"REAL"`` or ``"BLOB"``.
If you pass a Python type, it will be mapped to SQLite types as shown here::
float: "FLOAT"
float: "REAL"
int: "INTEGER"
bool: "INTEGER"
str: "TEXT"
@ -1294,12 +1294,9 @@ If you pass a Python type, it will be mapped to SQLite types as shown here::
np.uint16: "INTEGER"
np.uint32: "INTEGER"
np.uint64: "INTEGER"
np.float16: "FLOAT"
np.float32: "FLOAT"
np.float64: "FLOAT"
.. note::
In sqlite-utils 3.x ``FLOAT`` is used for floating point columns when the correct column type is actually ``REAL``. If you specify ``strict=True`` tables created in strict mode will use the correct column type of ``REAL`` instead. We plan to change this behavior in ``sqlite-utils`` 4.x to always use ``REAL``, but this will represent a minor breaking change and so is being held for the next major release, see issue :issue:`645`.
np.float16: "REAL"
np.float32: "REAL"
np.float64: "REAL"
You can also add a column that is a foreign key reference to another table using the ``fk`` parameter: