Question

Related to this question:

Wildcards in column name for MySQL

Basically, there are going to be a variable number of columns with the name “word” in them.

What I want to know is, would it be faster to do a separate database call for each row (via getting the column information from the information schema), with a generated Python query per row, or would it be faster to simply SELECT *, and only use the columns I needed? Is it possible to say SELECT * NOT XYZ? As far as I can tell, no, there is no way to specifically exclude columns.

There aren’t going to be many different rows at first – only three. But there’s the potential for infinite rows in this. It’s basically dependent on how many different types of search queries we want to put on our website. Our whole scalability is based around expanding the number of rows.

Answer 1

If all you are doing is limiting the number of columns returned there is no need to do a dynamic query. The hard work for the database is in selecting the rows matching your WHERE clause; it makes little difference to send you 5 columns out of 10, or all 10.

Just use a “SELECT * FROM …” and use Python to pick out the columns from the result set. You’ll use just one query to the database, so MySQL only has to work once, then filter out your columns:

cursor.execute('SELECT * FROM ...')
cols = [i for i, col in enumerate(cursor.description) if col[0].startswith('word')]
for row in cursor:
    columns = [row[c] for c in cols]

You may have to use for row in cursor.fetchall() instead depending on your MySQL python module.