Detect CSV/TSV column types by default (#683)

The `--detect-types` option is now automatically turned on for all commands that deal with CSV or CSV.

A new `--no-detect-types` option can be used to have all columns treated as text.

Closes #679

docs/changelog.rst

@@ -11,6 +11,7 @@ Unreleased
 
 - The ``table.insert_all()`` and ``table.upsert_all()`` methods can now accept an iterator of lists or tuples as an alternative to dictionaries. The first item should be a list/tuple of column names. See :ref:`python_api_insert_lists` for details. (:issue:`672`)
 - **Breaking change:** The default floating point column type has been changed from ``FLOAT`` to ``REAL``, which is the correct SQLite type for floating point values. This affects auto-detected columns when inserting data. (:issue:`645`)
+- **Breaking change:** Type detection is now the default behavior for the ``insert`` and ``upsert`` CLI commands when importing CSV or TSV data. Previously all columns were treated as ``TEXT`` unless the ``--detect-types`` flag was passed. Use the new ``--no-detect-types`` flag to restore the old behavior. The ``SQLITE_UTILS_DETECT_TYPES`` environment variable has been removed. (:issue:`679`)
 
 .. _v4_0a0:
 

docs/cli-reference.rst

@@ -285,7 +285,8 @@ See :ref:`cli_inserting_data`, :ref:`cli_insert_csv_tsv`, :ref:`cli_insert_unstr
       --alter                   Alter existing table to add any missing columns
       --not-null TEXT           Columns that should be created as NOT NULL
       --default <TEXT TEXT>...  Default value that should be set for a column
-      -d, --detect-types        Detect types for columns in CSV/TSV data
+      -d, --detect-types        Detect types for columns in CSV/TSV data (default)
+      --no-detect-types         Treat all CSV/TSV columns as TEXT
       --analyze                 Run ANALYZE at the end of this operation
       --load-extension TEXT     Path to SQLite extension, with optional :entrypoint
       --silent                  Do not show progress bar
@@ -342,7 +343,8 @@ See :ref:`cli_upsert`.
       --alter                   Alter existing table to add any missing columns
       --not-null TEXT           Columns that should be created as NOT NULL
       --default <TEXT TEXT>...  Default value that should be set for a column
-      -d, --detect-types        Detect types for columns in CSV/TSV data
+      -d, --detect-types        Detect types for columns in CSV/TSV data (default)
+      --no-detect-types         Treat all CSV/TSV columns as TEXT
       --analyze                 Run ANALYZE at the end of this operation
       --load-extension TEXT     Path to SQLite extension, with optional :entrypoint
       --silent                  Do not show progress bar

docs/cli.rst

@@ -508,7 +508,7 @@ Incoming CSV data will be assumed to use ``utf-8``. If your data uses a differen
 
 If you are joining across multiple CSV files they must all use the same encoding.
 
-Column types will be automatically detected in CSV or TSV data, using the same mechanism as ``--detect-types`` described in :ref:`cli_insert_csv_tsv`. You can pass the ``--no-detect-types`` option to disable this automatic type detection and treat all CSV and TSV columns as ``TEXT``.
+Column types will be automatically detected in CSV or TSV data, as described in :ref:`cli_insert_csv_tsv`. You can pass the ``--no-detect-types`` option to disable this automatic type detection and treat all CSV and TSV columns as ``TEXT``.
 
 .. _cli_memory_explicit:
 
@@ -1263,7 +1263,7 @@ To stop inserting after a specified number of records - useful for getting a fas
 
 A progress bar is displayed when inserting data from a file. You can hide the progress bar using the ``--silent`` option.
 
-By default every column inserted from a CSV or TSV file will be of type ``TEXT``. To automatically detect column types - resulting in a mix of ``TEXT``, ``INTEGER`` and ``REAL`` columns, use the ``--detect-types`` option (or its shortcut ``-d``).
+By default, column types are automatically detected for CSV or TSV files - resulting in a mix of ``TEXT``, ``INTEGER`` and ``REAL`` columns. To disable type detection and treat all columns as ``TEXT``, use the ``--no-detect-types`` option.
 
 For example, given a ``creatures.csv`` file containing this:
 
@@ -1277,9 +1277,9 @@ The following command:
 
 .. code-block:: bash
 
-    sqlite-utils insert creatures.db creatures creatures.csv --csv --detect-types
+    sqlite-utils insert creatures.db creatures creatures.csv --csv
 
-Will produce this schema:
+Will produce this schema with automatically detected types:
 
 .. code-block:: bash
 
@@ -1293,11 +1293,11 @@ Will produce this schema:
        "weight" REAL
     );
 
-You can set the ``SQLITE_UTILS_DETECT_TYPES`` environment variable if you want ``--detect-types`` to be the default behavior:
+To disable type detection and treat all columns as TEXT, use ``--no-detect-types``:
 
 .. code-block:: bash
 
-    export SQLITE_UTILS_DETECT_TYPES=1
+    sqlite-utils insert creatures.db creatures creatures.csv --csv --no-detect-types
 
 If a CSV or TSV file includes empty cells, like this one: