Question

I’m trying to populate an sqlite3 database with a list of filenames from a directory using Python. I’m not sure why this is not working:

conn = sqlite3.connect( "databasefile.db" )
cur = conn.cursor()
cur.execute('CREATE TABLE IF NOT EXISTS filenames (filename TEXT)')
for root, dirnames, filenames in os.walk(directory):
    for filename in filenames:
        if filename.endswith(".ext"):
            cur.executemany('INSERT INTO filenames (filename) VALUES (?)', (filename))
            conn.commit()

I’d like it to produce this in the database file:

   Column
1  1st.ext
2  2nd.ext
3  3rd.ext

But instead it results in this:

   Column
1  1
2  r
3  s
4  t
5  .
6  e
7  x
8  t
9  2
10 n
11 d
12 .

etc.

Could anyone help me understand where I’m going wrong?

EDIT: Thank you, I didn’t realize iterating over a string gives the individual characters. Added the comma to make ‘filename’ a tuple and resolved.

Answer 1

First off, you don’t need to use executemany, since you’re not actually doing more than one insert at a time. Secondly, change (filename) to (filename,). Both strings and tuples support the iterator protocol, and without that comma, the executemany function is interpreting the variable filename as the thing to iterate over. When you iterate over a string, you get individual characters.

The other thing you thing you might do is as follows:

for root, dirnames, filenames in os.walk(directory):
    cur.executemany("INSERT INTO filenames (filename) VALUES (?)",
                    [(filename,) for filename in filenames if filename.endswith(".ext")])
    conn.commit()

Now you’re actually taking advantage of executemany, and generally, fewer jaunts off to your database is better.