Question

I have a 100 megabytes SQLite database file that I would like to load to memory before performing SQL queries. Is it possible to do that in Python?

Answer 1

apsw is an alternate wrapper for SQLite, which enables you to backup an on-disk database to memory before doing operations.

From the documentation:

###
### Backup to memory
###

# We will copy the disk database into a memory database

memcon=apsw.Connection(":memory:")

# Copy into memory
with memcon.backup("main", connection, "main") as backup:
    backup.step() # copy whole database in one go

# There will be no disk accesses for this query
for row in memcon.cursor().execute("select * from s"):
    pass

connection above is your on-disk database.