Question

I’m inserting values into my table (from python code) as follows:

cur.execute("insert into t(a, b, c) values (?, ?, ?)", (a, b, c))

There is a unique constraint on column c. What is a common way of insert if I want to cover the case when we’re inserting duplicate value for c column?
I have some ideas

1. select everything from t to list and test before inserting if the value is already present
2. try-catch block
3. some more sophisticated sqlite statement

How would you test it?

thank you

Answer 1

You could use INSERT OR REPLACE to update rows with a unique constraint,
or INSERT OR IGNORE to ignore inserts which conflict with a unique constraint:

import sqlite3

def insert_or_replace():
    # https://sqlite.org/lang_insert.html
    connection=sqlite3.connect(':memory:')
    cursor=connection.cursor()
    cursor.execute('CREATE TABLE foo (bar INTEGER UNIQUE, baz INTEGER)')
    cursor.execute('INSERT INTO foo (bar,baz) VALUES (?, ?)',(1,2))
    cursor.execute('INSERT OR REPLACE INTO foo (bar,baz) VALUES (?, ?)',(1,3))
    cursor.execute('SELECT * from foo')
    data=cursor.fetchall()
    print(data)
    # [(1, 3)]


def on_conflict():
    # https://sqlite.org/lang_insert.html
    connection=sqlite3.connect(':memory:')
    cursor=connection.cursor()
    cursor.execute('CREATE TABLE foo (bar INTEGER UNIQUE, baz INTEGER)')
    cursor.execute('INSERT INTO foo (bar,baz) VALUES (?, ?)',(1,2))
    cursor.execute('INSERT OR IGNORE INTO foo (bar,baz) VALUES (?, ?)',(1,3))
    cursor.execute('SELECT * from foo')
    data=cursor.fetchall()
    print(data)
    # [(1, 2)]    

insert_or_replace()
on_conflict()

These sqlite commands are probably faster than writing Python code to do the same thing, though to test this you could use Python’s timeit module to test the speed of various implementations. For example, you could run

python -mtimeit -s'import test' 'test.insert_or_replace()'

versus

python -mtimeit -s'import test' 'test.filter_nonunique_rows_in_Python()'

versus

python -mtimeit -s'import test' 'test.insert_with_try_catch_blocks()'