Question

I’m try to insert values into my sqlite table using a python script.

It was working perfectly until I tried to add another column called ‘information’ – it then threw the following error:

You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings

So I added:

conn.text_factory = str

Then I got this error:

Incorrect number of bindings supplied. The current statement uses 7, and there are 3 supplied.

I think the problem is that this new ‘information’ column contains a few lines of text so I may be specifying it incorrectly as ‘text’. My python script code:

import sqlite3;
from datetime import datetime, date;
import time
conn = sqlite3.connect('mynewtable.sqlite3')
conn.text_factory = str
c = conn.cursor()
c.execute('drop table if exists mynewtable')
c.execute('create table mynewtable(id integer primary key autoincrement, rank integer, placename text, information text, nooftimes integer, visit text, fav integer, year integer)')

def mysplit (string):
quote = False
retval = []
current = ""
for char in string:
    if char == '"':
        quote = not quote
    elif char == ',' and not quote:
        retval.append(current)
        current = ""
    else:
        current += char
retval.append(current)
return retval

# Read lines from file, skipping first line
data = open("mynewtable.csv", "r").readlines()[1:]
for entry in data:
# Parse values
vals = mysplit(entry.strip())

# Insert the row!
print "Inserting %s..." % (vals[0])
sql = "insert into mynewtable values(NULL, ?, ?, ?, ?, ?, ?, ?)"
c.execute(sql, vals)

# Done!
conn.commit()

Answer 1

It seems you are trying to re-invent the wheel a bit here 🙂

Try using python’s csv module; I’ve used it extensively and it works very well:
http://docs.python.org/library/csv.html

It works perfectly well with properly formed csv files that have multi-line texts.

EDIT:

For example, you can use the csv rows (which are lists) directly in your execute function:

import csv
for row in csv.reader(open('allnamesallyearsn.csv')):
    c.execute(sql, row)

2nd EDIT:

As per my last comment, here is the code you posted making use of the csv module:

import sqlite3, csv, time
from datetime import datetime, date

conn = sqlite3.connect('mynewtable.sqlite3')
conn.text_factory = str
c = conn.cursor()
c.execute('drop table if exists mynewtable')
c.execute('create table mynewtable('
          'id integer primary key autoincrement, '
          'rank integer, '
          'placename text, '
          'information text, '
          'nooftimes integer, '
          'visit text, '
          'fav integer, '
          'year integer)')

sql_insert = "insert into mynewtable values(NULL, ?, ?, ?, ?, ?, ?, ?)"
csv_reader = csv.reader(open('mynewtable.csv', 'rb'))
csv_reader.next() # skip headers
for csv_row in csv_reader:
    print "Inserting %s..." % (csv_row)
    c.execute(sql_insert, csv_row)

conn.commit()