Question

I’m very new to python and I have a basic python script where I need to insert values from a CSV file into an SQLite database using Python. I’ve done that part successfully, however, I need to not include columns 26, 27 and 28 out of 29. Whenever I try to do this, it just skips over the entire row, however I only want it to skip that specific column in general. I tried to not include the columns when creating the table originally however, I would get an error each time when I ran the script

This is the code I’m working with with regards to this process:

import sys, sqlite, csv

  try: 
    cx = sqlite.connect("database")

except sqlite.Error, errmsg:
    print "Could not open the database file: " + str(errmsg)
    sys.exit()

#create the table        
try:
    cu = cx.cursor()
    cu.execute('DROP TABLE IF EXISTS table_name')

    sql = """ CREATE TABLE table_name(col1 TEXT, col2 TEXT, col3 TEXT, col4 TEXT, col5 TEXT, col6 TEXT, col7 TEXT, col8 TEXT, col9 TEXT, col10 TEXT, col11 TEXT, col12 TEXT, col13 TEXT, col14 TEXT, col15 TEXT, col16 TEXT, col17 TEXT, col18 TEXT, col19 TEXT, col20 TEXT, col21 TEXT, col22 TEXT, col23 TEXT, col24 TEXT, col25 TEXT, col26 TEXT, col27 TEXT, col28 TEXT, col29 TEXT); """

    cu.execute(sql)
    cx.commit()
except sqlite.Error, errmsg:
    print "Could not execute the query: " + str(errmsg)
    sys.exit()

#Load the CSV file into the csv reader


fin = open("test.csv", "rb")
creader = csv.reader(fin, delimiter=',')

# Interate through the CSV Reader, inserting each value into the database
    # NEW REVISION
  sql_insert = "INSERT INTO table_name VALUES(%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,,,,%s);"    


      for row in creader:
         cu.execute(sql_insert, row)
         print row

fin.close()
cx.commit()
cx.close()

Answer 1

You are iterating over the output of csv.reader() as if they are columns. That is incorrect. They are in fact whole rows. It should be more like:

for row in reader:
    # Now to skip the mentioned columns just don't use those in your sql insert statement
    # for example column 25 is row[24]
    # Instead, put a blank in your SQL
    row[25]=""
    row[26]=""
    row[27]=""

    sql_insert = "INSERT INTO table_name   VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?);"  
    cursor.execute(sql_insert , row)

insert instead of %s for those columns