Question

I just spent a very long time debugging an issue in python, using the web.py framework, and it has me wondering about a way to check this kind of thing in the future.

In short, one of the methods of web.py’s database class was returning a storage object that, I can only surmise, was a reference to a value in memory, instead of a copy. The result was that no matter what I tried to do, I could not append multiple results of a query into a list, because the list just wound up with copies of the same row of data.

To solve it, I had to create another dictionary and convert all values in the web.py storage object explicitly to another datatype (I just used strings to get it working) so that it would be forced to create a new object.

In code, this is what was happening:

>>> result = db.query(query_params)
>>> for row in result:
>>>     print row
...     result_list.append(row)
{"result" : "from", "row" : "one"}
{"result" : "from", "row" : "two"}
>>> print result_list
[{"result":"from", "row" : "two"}, {"result" : "from", "row" : "two}]

This is what clued me into the fact that this was some sort of reference issue. The list was storing the location of “row”, instead of a copy of it.

The first thing I tried was something along the lines of:

 copy = {}
 for row in result:
    for value in row:
        copy[value] = row[value]
    result_list.append(copy)

But this lead to the same problem.
I only found a solution by tweaking the above to read:

            copy[value] = str(row[value])

So, my question is two-fold, really:

1. Is there an easy way to tell how something is being stored and/or passed around?
2. Is there a way to explicitly request a copy instead of a reference?

Answer 1

Use copy. It allows simple shallow- and deep-copying of objects in python:

import copy    
result_list.append(copy.copy(row))