remote/sqlite-utils
1
0
Fork 0
mirror of https://github.com/simonw/sqlite-utils.git synced 2026-07-23 17:34:32 +02:00
Code Issues Projects Releases Packages Wiki Activity
sqlite-utils/docs/tutorial.ipynb

1051 lines
31 KiB
Text
Raw Normal View History

Started a Jupyter notebook tutorial, refs #308
2021-08-04 13:34:30 -07:00
{
"cells": [
{
"cell_type": "markdown",
"id": "27ae18ec",
"metadata": {},
"source": [
"# The sqlite-utils tutorial\n",
"\n",
"[sqlite-utils](https://sqlite-utils.datasette.io/en/stable/python-api.html) is a Python library (and [command-line tool](https://sqlite-utils.datasette.io/en/stable/cli.html) for quickly creating and manipulating SQLite database files.\n",
"\n",
"This tutorial will show you how to use the Python library to manipulate data.\n",
"\n",
"## Installation\n",
"\n",
"To install the library, run:\n",
"\n",
" pip install sqlite-utils\n",
"\n",
"You can run this in a Jupyter notebook cell by executing:\n",
"\n",
" %pip install sqlite-utils\n",
" \n",
"Or use `pip install -U sqlite-utils` to ensure you have upgraded to the most recent version."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "bddee0d2",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: sqlite_utils in /usr/local/Cellar/jupyterlab/3.0.16_1/libexec/lib/python3.9/site-packages (3.14)\n",
"Requirement already satisfied: click-default-group in /usr/local/lib/python3.9/site-packages (from sqlite_utils) (1.2.2)\n",
"Requirement already satisfied: sqlite-fts4 in /usr/local/lib/python3.9/site-packages (from sqlite_utils) (1.0.1)\n",
"Requirement already satisfied: click in /Users/simon/Library/Python/3.9/lib/python/site-packages (from sqlite_utils) (7.1.2)\n",
"Requirement already satisfied: tabulate in /usr/local/lib/python3.9/site-packages (from sqlite_utils) (0.8.7)\n",
Use python-dateutil package instead of dateutils (#324)
2021-11-14 18:25:40 -05:00
"Requirement already satisfied: python-dateutil in /usr/local/Cellar/jupyterlab/3.0.16_1/libexec/lib/python3.9/site-package (from sqlite-utils) (2.8.1)\n",
"Requirement already satisfied: six>=1.5 in /usr/local/Cellar/jupyterlab/3.0.16_1/libexec/lib/python3.9/site-package (from python-dateutil->sqlite-utils) (1.16.0)\n",
Started a Jupyter notebook tutorial, refs #308
2021-08-04 13:34:30 -07:00
"\u001b[33mWARNING: You are using pip version 21.1.1; however, version 21.2.2 is available.\n",
"You should consider upgrading via the '/usr/local/Cellar/jupyterlab/3.0.16_1/libexec/bin/python3.9 -m pip install --upgrade pip' command.\u001b[0m\n",
"Note: you may need to restart the kernel to use updated packages.\n"
]
}
],
"source": [
"%pip install -U sqlite_utils"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "050e85a8",
"metadata": {},
"outputs": [],
"source": [
"import sqlite_utils"
]
},
{
"cell_type": "markdown",
"id": "348bcbfc",
"metadata": {},
"source": [
"You can use the library with a database file on disk by running:\n",
"\n",
" db = sqlite_utils.Database(\"path/to/my/database.db\")\n",
"\n",
"In this tutorial we will use an in-memory database. This is a quick way to try out new things, though you should note that when you close the notebook the data store in the in-memory database will be lost."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "4b2aee7e",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<Database <sqlite3.Connection object at 0x139a16300>>"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"db = sqlite_utils.Database(memory=True)\n",
"db"
]
},
{
"cell_type": "markdown",
"id": "1598ab43",
"metadata": {},
"source": [
"## Creating a table\n",
"\n",
"We are going to create a new table in our database called `creatures` by passing in a Python list of dictionaries.\n",
"\n",
"`db[name_of_table]` will access a database table object with that name.\n",
"\n",
"Inserting data into that table will create it if it does not already exist."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "4a0ac420",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<Table creatures (name, species, age)>"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"db[\"creatures\"].insert_all([{\n",
" \"name\": \"Cleo\",\n",
" \"species\": \"dog\",\n",
" \"age\": 6\n",
"}, {\n",
" \"name\": \"Lila\",\n",
" \"species\": \"chicken\",\n",
" \"age\": 0.8,\n",
"}, {\n",
" \"name\": \"Bants\",\n",
" \"species\": \"chicken\",\n",
" \"age\": 0.8,\n",
"}])"
]
},
{
"cell_type": "markdown",
"id": "049d110b",
"metadata": {},
"source": [
"Let's grab a `table` reference to the new creatures table:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "8d84ad9c",
"metadata": {},
"outputs": [],
"source": [
"table = db[\"creatures\"]"
]
},
{
"cell_type": "markdown",
"id": "ffe45750",
"metadata": {},
"source": [
"`sqlite-utils` automatically creates a table schema that matches the keys and data types of the dictionaries that were passed to `.insert_all()`.\n",
"\n",
"We can see that schema using `table.schema`:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "136cee1e",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CREATE TABLE [creatures] (\n",
" [name] TEXT,\n",
" [species] TEXT,\n",
" [age] FLOAT\n",
")\n"
]
}
],
"source": [
"print(table.schema)"
]
},
{
"cell_type": "markdown",
"id": "9e5c3ae9",
"metadata": {},
"source": [
"## Accessing data\n",
"\n",
"The `table.rows` property lets us loop through the rows in the table, returning each one as a Python dictionary:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "f812914d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'name': 'Cleo', 'species': 'dog', 'age': 6.0}\n",
"{'name': 'Lila', 'species': 'chicken', 'age': 0.8}\n",
"{'name': 'Bants', 'species': 'chicken', 'age': 0.8}\n"
]
}
],
"source": [
"for row in table.rows:\n",
" print(row)"
]
},
{
"cell_type": "markdown",
"id": "60bc6b2c",
"metadata": {},
"source": [
"The `db.query(sql)` method can be used to execute SQL queries and return the results as dictionaries:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "eaadd85f",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'name': 'Cleo', 'species': 'dog', 'age': 6.0},\n",
" {'name': 'Lila', 'species': 'chicken', 'age': 0.8},\n",
" {'name': 'Bants', 'species': 'chicken', 'age': 0.8}]"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(db.query(\"select * from creatures\"))"
]
},
{
"cell_type": "markdown",
"id": "6614467b",
"metadata": {},
"source": [
"Or in a loop:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "88fdd52e",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Cleo is a dog\n",
"Lila is a chicken\n",
"Bants is a chicken\n"
]
}
],
"source": [
"for row in db.query(\"select name, species from creatures\"):\n",
" print(f'{row[\"name\"]} is a {row[\"species\"]}')"
]
},
{
"cell_type": "markdown",
"id": "b81c031c",
"metadata": {},
"source": [
"### SQL parameters\n",
"\n",
"You can run a parameterized query using `?` as placeholders and passing a list of variables. The variables you pass will be correctly quoted, protecting your code from SQL injection vulnerabilities."
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "267035d9",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'name': 'Cleo', 'species': 'dog', 'age': 6.0}]"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(db.query(\"select * from creatures where age > ?\", [1.0]))"
]
},
{
"cell_type": "markdown",
"id": "87cb301b",
"metadata": {},
"source": [
"As an alternative to question marks we can use `:name` parameters and feed in the values using a dictionary:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "83be9a80",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'name': 'Lila', 'species': 'chicken', 'age': 0.8},\n",
" {'name': 'Bants', 'species': 'chicken', 'age': 0.8}]"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(db.query(\"select * from creatures where species = :species\", {\"species\": \"chicken\"}))"
]
},
{
"cell_type": "markdown",
"id": "5e5179cc",
"metadata": {},
"source": [
"### Primary keys\n",
"\n",
"When we created this table we did not specify a primary key. SQLite automatically creates a primary key called `rowid` if no other primary key is defined.\n",
"\n",
"We can run `select rowid, * from creatures` to see this hidden primary key:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "c9d963df",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'rowid': 1, 'name': 'Cleo', 'species': 'dog', 'age': 6.0},\n",
" {'rowid': 2, 'name': 'Lila', 'species': 'chicken', 'age': 0.8},\n",
" {'rowid': 3, 'name': 'Bants', 'species': 'chicken', 'age': 0.8}]"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(db.query(\"select rowid, * from creatures\"))"
]
},
{
"cell_type": "markdown",
"id": "0f87cdfb",
"metadata": {},
"source": [
"We can also see that using `table.pks_and_rows_where()`:"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "d365e405",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1 {'rowid': 1, 'name': 'Cleo', 'species': 'dog', 'age': 6.0}\n",
"2 {'rowid': 2, 'name': 'Lila', 'species': 'chicken', 'age': 0.8}\n",
"3 {'rowid': 3, 'name': 'Bants', 'species': 'chicken', 'age': 0.8}\n"
]
}
],
"source": [
"for pk, row in table.pks_and_rows_where():\n",
" print(pk, row)"
]
},
{
"cell_type": "markdown",
"id": "5b0e9b74",
"metadata": {},
"source": [
"Let's recreate the table with our own primary key, which we will call `id`.\n",
"\n",
"`table.drop()` drops the table:"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "568a0e29",
"metadata": {},
"outputs": [],
"source": [
"table.drop()"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "13ebd3ab",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<Table creatures (does not exist yet)>"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table"
]
},
{
"cell_type": "markdown",
"id": "522aa6d0",
"metadata": {},
"source": [
"We can see a list of tables in the database using `db.tables`:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "f3e62678",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[]"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"db.tables"
]
},
{
"cell_type": "markdown",
"id": "6b80d523",
"metadata": {},
"source": [
"We'll create the table again, this time with an `id` column.\n",
"\n",
"We use `pk=\"id\"` to specify that the `id` column should be treated as the primary key for the table:"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "c9ee8b9f",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<Table creatures (id, name, species, age)>"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"db[\"creatures\"].insert_all([{\n",
" \"id\": 1,\n",
" \"name\": \"Cleo\",\n",
" \"species\": \"dog\",\n",
" \"age\": 6\n",
"}, {\n",
" \"id\": 2,\n",
" \"name\": \"Lila\",\n",
" \"species\": \"chicken\",\n",
" \"age\": 0.8,\n",
"}, {\n",
" \"id\": 3,\n",
" \"name\": \"Bants\",\n",
" \"species\": \"chicken\",\n",
" \"age\": 0.8,\n",
"}], pk=\"id\")"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "523e01ab",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CREATE TABLE [creatures] (\n",
" [id] INTEGER PRIMARY KEY,\n",
" [name] TEXT,\n",
" [species] TEXT,\n",
" [age] FLOAT\n",
")\n"
]
}
],
"source": [
"print(table.schema)"
]
},
{
"cell_type": "markdown",
"id": "811bea70",
"metadata": {},
"source": [
"## Inserting more records\n",
"\n",
"We can call `.insert_all()` again to insert more records. Let's add two more chickens."
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "716df161",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<Table creatures (id, name, species, age)>"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table.insert_all([{\n",
" \"id\": 4,\n",
" \"name\": \"Azi\",\n",
" \"species\": \"chicken\",\n",
" \"age\": 0.8,\n",
"}, {\n",
" \"id\": 5,\n",
" \"name\": \"Snowy\",\n",
" \"species\": \"chicken\",\n",
" \"age\": 0.9,\n",
"}], pk=\"id\")"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "4b1b2476",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'id': 1, 'name': 'Cleo', 'species': 'dog', 'age': 6.0},\n",
" {'id': 2, 'name': 'Lila', 'species': 'chicken', 'age': 0.8},\n",
" {'id': 3, 'name': 'Bants', 'species': 'chicken', 'age': 0.8},\n",
" {'id': 4, 'name': 'Azi', 'species': 'chicken', 'age': 0.8},\n",
" {'id': 5, 'name': 'Snowy', 'species': 'chicken', 'age': 0.9}]"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(table.rows)"
]
},
{
"cell_type": "markdown",
"id": "2af4ae75",
"metadata": {},
"source": [
"Since the `id` column is an integer primary key, we can insert a record without specifying an ID and one will be automatically added.\n",
"\n",
"Since we are only adding one record we will use `.insert()` instead of `.insert_all()`."
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "246c6dd5",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<Table creatures (id, name, species, age)>"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table.insert({\"name\": \"Blue\", \"species\": \"chicken\", \"age\": 0.9})"
]
},
{
"cell_type": "markdown",
"id": "d7c28e4d",
"metadata": {},
"source": [
"We can use `table.last_pk` to see the ID of the record we just added."
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "de012e1e",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"6"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table.last_pk"
]
},
{
"cell_type": "markdown",
"id": "c38edaf4",
"metadata": {},
"source": [
"Here's the full list of rows again:"
]
},
{
"cell_type": "code",
"execution_count": 23,
"id": "7c27075e",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'id': 1, 'name': 'Cleo', 'species': 'dog', 'age': 6.0},\n",
" {'id': 2, 'name': 'Lila', 'species': 'chicken', 'age': 0.8},\n",
" {'id': 3, 'name': 'Bants', 'species': 'chicken', 'age': 0.8},\n",
" {'id': 4, 'name': 'Azi', 'species': 'chicken', 'age': 0.8},\n",
" {'id': 5, 'name': 'Snowy', 'species': 'chicken', 'age': 0.9},\n",
" {'id': 6, 'name': 'Blue', 'species': 'chicken', 'age': 0.9}]"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(table.rows)"
]
},
{
"cell_type": "markdown",
"id": "64931bd0",
"metadata": {},
"source": [
"If you try to add a new record with an existing ID, you will get an `IntegrityError`:"
]
},
{
"cell_type": "code",
"execution_count": 24,
"id": "36327794",
"metadata": {},
"outputs": [
{
"ename": "IntegrityError",
"evalue": "UNIQUE constraint failed: creatures.id",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mIntegrityError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-24-4222c6abc759>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mtable\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minsert\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0;34m\"id\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;36m6\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"name\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m\"Red\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"species\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m\"chicken\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"age\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;36m0.9\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;32m/usr/local/Cellar/jupyterlab/3.0.16_1/libexec/lib/python3.9/site-packages/sqlite_utils/db.py\u001b[0m in \u001b[0;36minsert\u001b[0;34m(self, record, pk, foreign_keys, column_order, not_null, defaults, hash_id, alter, ignore, replace, extracts, conversions, columns)\u001b[0m\n\u001b[1;32m 2027\u001b[0m \u001b[0mcolumns\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mDEFAULT\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2028\u001b[0m ):\n\u001b[0;32m-> 2029\u001b[0;31m return self.insert_all(\n\u001b[0m\u001b[1;32m 2030\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mrecord\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2031\u001b[0m \u001b[0mpk\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpk\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/Cellar/jupyterlab/3.0.16_1/libexec/lib/python3.9/site-packages/sqlite_utils/db.py\u001b[0m in \u001b[0;36minsert_all\u001b[0;34m(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, alter, ignore, replace, truncate, extracts, conversions, columns, upsert)\u001b[0m\n\u001b[1;32m 2143\u001b[0m \u001b[0mfirst\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2144\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2145\u001b[0;31m self.insert_chunk(\n\u001b[0m\u001b[1;32m 2146\u001b[0m \u001b[0malter\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2147\u001b[0m \u001b[0mextracts\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/Cellar/jupyterlab/3.0.16_1/libexec/lib/python3.9/site-packages/sqlite_utils/db.py\u001b[0m in \u001b[0;36minsert_chunk\u001b[0;34m(self, alter, extracts, chunk, all_columns, hash_id, upsert, pk, conversions, num_records_processed, replace, ignore)\u001b[0m\n\u001b[1;32m 1955\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mquery\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparams\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mqueries_and_params\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1956\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1957\u001b[0;31m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mquery\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparams\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1958\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mOperationalError\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1959\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0malter\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m\" column\"\u001b[0m \u001b[0;32min\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/Cellar/jupyterlab/3.0.16_1/libexec/lib/python3.9/site-packages/sqlite_utils/db.py\u001b[0m in \u001b[0;36mexecute\u001b[0;34m(self, sql, parameters)\u001b[0m\n\u001b[1;32m 255\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_tracer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msql\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparameters\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 256\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mparameters\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 257\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconn\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msql\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparameters\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 258\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 259\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconn\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msql\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mIntegrityError\u001b[0m: UNIQUE constraint failed: creatures.id"
]
}
],
"source": [
"table.insert({\"id\": 6, \"name\": \"Red\", \"species\": \"chicken\", \"age\": 0.9})"
]
},
{
"cell_type": "markdown",
"id": "2e00692f",
"metadata": {},
"source": [
"You can use `replace=True` to replace the matching record with a new one:"
]
},
{
"cell_type": "code",
"execution_count": 25,
"id": "2be75589",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<Table creatures (id, name, species, age)>"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table.insert({\"id\": 6, \"name\": \"Red\", \"species\": \"chicken\", \"age\": 0.9}, replace=True)"
]
},
{
"cell_type": "code",
"execution_count": 26,
"id": "83281675",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'id': 1, 'name': 'Cleo', 'species': 'dog', 'age': 6.0},\n",
" {'id': 2, 'name': 'Lila', 'species': 'chicken', 'age': 0.8},\n",
" {'id': 3, 'name': 'Bants', 'species': 'chicken', 'age': 0.8},\n",
" {'id': 4, 'name': 'Azi', 'species': 'chicken', 'age': 0.8},\n",
" {'id': 5, 'name': 'Snowy', 'species': 'chicken', 'age': 0.9},\n",
" {'id': 6, 'name': 'Red', 'species': 'chicken', 'age': 0.9}]"
]
},
"execution_count": 26,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(table.rows)"
]
},
{
"cell_type": "markdown",
"id": "d7122b76",
"metadata": {},
"source": [
"## Updating a record\n",
"\n",
"We will rename that row back to `Blue`, this time using the `table.update(pk, updates)` method:"
]
},
{
"cell_type": "code",
"execution_count": 28,
"id": "43df156d",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<Table creatures (id, name, species, age)>"
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table.update(6, {\"name\": \"Blue\"})"
]
},
{
"cell_type": "code",
"execution_count": 32,
"id": "0b8f8422",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'id': 6, 'name': 'Blue', 'species': 'chicken', 'age': 0.9}]"
]
},
"execution_count": 32,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(db.query(\"select * from creatures where id = ?\", [6]))"
]
},
{
"cell_type": "markdown",
"id": "58142b86",
"metadata": {},
"source": [
"## Extracting one of the columns into another table\n",
"\n",
"Our current table has a `species` column with a string in it - let's pull that out into a separate table.\n",
"\n",
"We can do that using the [table.extract() method](https://sqlite-utils.datasette.io/en/stable/python-api.html#extracting-columns-into-a-separate-table)."
]
},
{
"cell_type": "code",
"execution_count": 34,
"id": "6ab69111",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<Table creatures (id, name, species_id, age)>"
]
},
"execution_count": 34,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table.extract(\"species\")"
]
},
{
"cell_type": "markdown",
"id": "dca327b2",
"metadata": {},
"source": [
"We now have a new table called `species`, which we can see using the `db.tables` method:"
]
},
{
"cell_type": "code",
"execution_count": 35,
"id": "76e95b36",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[<Table species (id, species)>, <Table creatures (id, name, species_id, age)>]"
]
},
"execution_count": 35,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"db.tables"
]
},
{
"cell_type": "markdown",
"id": "5ea43bf5",
"metadata": {},
"source": [
"Our creatures table has been modified - instead of a `species` column it now has `species_id` which is a foreign key to the new table:"
]
},
{
"cell_type": "code",
"execution_count": 37,
"id": "c0438bff",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CREATE TABLE \"creatures\" (\n",
" [id] INTEGER PRIMARY KEY,\n",
" [name] TEXT,\n",
" [species_id] INTEGER,\n",
" [age] FLOAT,\n",
" FOREIGN KEY([species_id]) REFERENCES [species]([id])\n",
")\n",
"[{'id': 1, 'name': 'Cleo', 'species_id': 1, 'age': 6.0}, {'id': 2, 'name': 'Lila', 'species_id': 2, 'age': 0.8}, {'id': 3, 'name': 'Bants', 'species_id': 2, 'age': 0.8}, {'id': 4, 'name': 'Azi', 'species_id': 2, 'age': 0.8}, {'id': 5, 'name': 'Snowy', 'species_id': 2, 'age': 0.9}, {'id': 6, 'name': 'Blue', 'species_id': 2, 'age': 0.9}]\n"
]
}
],
"source": [
"print(db[\"creatures\"].schema)\n",
"print(list(db[\"creatures\"].rows))"
]
},
{
"cell_type": "markdown",
"id": "0452c201",
"metadata": {},
"source": [
"The new `species` table has been created and populated too:"
]
},
{
"cell_type": "code",
"execution_count": 39,
"id": "5d38c3a8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CREATE TABLE [species] (\n",
" [id] INTEGER PRIMARY KEY,\n",
" [species] TEXT\n",
")\n",
"[{'id': 1, 'species': 'dog'}, {'id': 2, 'species': 'chicken'}]\n"
]
}
],
"source": [
"print(db[\"species\"].schema)\n",
"print(list(db[\"species\"].rows))"
]
},
{
"cell_type": "markdown",
"id": "a0312d1e",
"metadata": {},
"source": [
"We can use a join SQL query to combine data from these two tables:"
]
},
{
"cell_type": "code",
"execution_count": 44,
"id": "6734ed5d",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'id': 1, 'name': 'Cleo', 'age': 6.0, 'species_id': 1, 'species': 'dog'},\n",
" {'id': 2, 'name': 'Lila', 'age': 0.8, 'species_id': 2, 'species': 'chicken'},\n",
" {'id': 3, 'name': 'Bants', 'age': 0.8, 'species_id': 2, 'species': 'chicken'},\n",
" {'id': 4, 'name': 'Azi', 'age': 0.8, 'species_id': 2, 'species': 'chicken'},\n",
" {'id': 5, 'name': 'Snowy', 'age': 0.9, 'species_id': 2, 'species': 'chicken'},\n",
" {'id': 6, 'name': 'Blue', 'age': 0.9, 'species_id': 2, 'species': 'chicken'}]"
]
},
"execution_count": 44,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(db.query(\"\"\"\n",
" select\n",
" creatures.id,\n",
" creatures.name,\n",
" creatures.age,\n",
" species.id as species_id,\n",
" species.species\n",
" from creatures\n",
" join species on creatures.species_id = species.id\n",
"\"\"\"))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5c4802ac",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Reference in a new issue Copy permalink