Question

I am working on a small project and I have created a helper function that will write a string of comma separated values to a database as if they were values. I realise there are implications to doing it this way but this is small and i need to get it going until i can do better

def db_insert(table,data):
    """
    insert data into a table, the data should be a tuple
    matching the number of columns with null for any columns that
    have no value. False is returned on any error, error is logged to
    database log file."""

    if os.path.exists(database_name):
        con = lite.connect(database_name)

    else:
        error = "Database file does not exist."
        to_log(error)
        return False

    if con:
        try:
            cur = con.cursor()
            data = str(data)
            cur.execute('insert into %s values(%s)') % (table, data)
            con.commit()
            con.close()

        except Exception, e:
            pre_error = "Database insert raised and error;\n"
            thrown_error = pre_error + str(e)
            to_log(thrown_error)

        finally:
            con.close()


    else:
        error = "No connection to database"
        to_log(error)
        return False

database_name etc… are defined elsewhere in the script.

Barring any other obvious glaring errors;
what i need to be able to do (by this method or some other if there are suggestions) is allow somebody to create a list where each value represents a column value. As I will not know how many columns are being populated.

so somebody uses it as follows:
data = [“null”, “foo”,”bar”]
db_insert(“foo_table”, data)

this insert that data into the table name foo_table. It is up to the user to know how many columns are in the table and supply the correct number of elements to satisfy that.
I realise that it is better to use sqlite parameters but there are two problems.
first you cannot use a parameter to specify the table only the values.
second is that you need to know how many values you are supplying. you have to do;

cur.execute(‘insert into table values(?,?,?), val1,val2,val3)

you need to be able to specify the three ?’s.
I am trying to write a general function that allows me to take an arbitrary number of values and insert them into an arbitrary table name.
Now, it was working relatively ok until i tried to pass in ‘null’ as a value.
One of the columns is the primary key and has an autoincrement. So passing in null will allow it to autoincrement. There will also be other instances where nulls would be required.
The problem is that python keeps wrapping my null in single quotes which sqlite complains about as a datatype mismatch as the primary key is an integer field. If I try passing None as the python null equivalent then the same thing happens.
So two problems.

How to insert an arbitrary number of columns.
How to pass a null.

Thank you for all your help on this and past questions.

Sorry, this looks like a duplicate of this
Using Python quick insert many columns into Sqlite\Mysql

my apologies I did not find it until after I wrote this.

Results in the following which works;

def db_insert(table,data):
"""
insert data into a table, the data should be a tuple
matching the number of columns with null for any columns that
have no value. False is returned on any error, error is logged to
database log file."""

if os.path.exists(database_name):
    con = lite.connect(database_name)

else:
    error = "Database file does not exist."
    to_log(error)
    return False

if con:
    try:
        tuple_len = len(data)
        holders = ','.join('?' * tuple_len)

        sql_query = 'insert into %s values({0})'.format(holders) % table
        cur = con.cursor()
        #data = str(data)
        #cur.execute('insert into readings values(%s)') % table
        cur.execute(sql_query, data)
        con.commit()
        con.close()

    except Exception, e:
        pre_error = "Database insert raised and error;\n"
        thrown_error = pre_error + str(e)
        to_log(thrown_error)

    finally:
        con.close()


else:
    error = "No connection to database"
    to_log(error)
    return False

Answer 1

The second problem is a “Works for me”. When I pass None as value it will correctly convert that value back and forth to and from the db.

import sqlite3
conn = sqlite3.connect("test.sqlite")

data = ("a", None)
conn.execute('INSERT INTO "foo" VALUES(' + ','.join("?" * len(data)) + ')', data)

list(conn.execute("SELECT * FROM foo"))      # -> [("a", None)]