Home/ Questions/Q 97165
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-10T23:57:09+00:00

I have a Python program that uses the threading module. Once every second, my

  • 0

I have a Python program that uses the ‘threading’ module. Once every second, my program starts a new thread that fetches some data from the web, and stores this data to my hard drive. I would like to use sqlite3 to store these results, but I can’t get it to work. The issue seems to be about the following line:

conn = sqlite3.connect('mydatabase.db')
  • If I put this line of code inside each thread, I get an OperationalError telling me that the database file is locked. I guess this means that another thread has mydatabase.db open through a sqlite3 connection and has locked it.
  • If I put this line of code in the main program and pass the connection object (conn) to each thread, I get a ProgrammingError, saying that SQLite objects created in a thread can only be used in that same thread.

Previously I was storing all my results in CSV files, and did not have any of these file-locking issues. Hopefully this will be possible with sqlite. Any ideas?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. You can use consumer-producer pattern. For example you can create queue that is shared between threads. First thread that fetches data from the web enqueues this data in the shared queue. Another thread that owns database connection dequeues data from the queue and passes it to the database.

    • 0