Question

I’m trying to build a web app on App Engine that, at a certain point, I need a function to construct a list property with one member from the distinct groups. I’m able to get this working if I use an Sqlite database and web.py database module to maniputate, I can implement what I need like this:

db = web.database(dbn='sqlite',db=':memory:')
db.query("CREATE TABLE seq_list (seq TEXT, seq_pickle TEXT);")

# do some stuff to the database

def getUnique():
    uniq_entries = db.query("SELECT DISTINCT seq FROM seq_list;")
    if not uniq_sequences:
        for entry in uniq_entries:
            seq_query = db.query("SELECT * FROM seq_list WHERE seq='" + str(entry.seq) +"';")
            seq_obj = pickle.loads(seq_query[0].seq_pickle)
            self.uniq_sequences.append(seq_obj)

I’m having trouble getting this to work in App engine though.

class SeqObj(db.Model):
    seq = db.TextProperty(required = True)
    seq_pickle = db.TextProperty(required = True)

# do some stuff to the database

def getUnique():
    entries = db.GqlQuery("SELECT * FROM SeqObj")
    entries = list(entries)
    if not uniq_sequences:
        unique_entries = set([entry.seq for entry in entries])

        # this prints the unique entries correctly
        print unique_entries

        for entry in unique_entries:
            try:
                 q = "SELECT * FROM SeqObj WHERE seq='%s'"% entry
                 print q
                 seq_query = db.GqlQuery(q)

                 # these are the lines that break it
                 seq_obj = pickle.loads(seq_query[0].seq_pickle)
                 uniq_sequences.append(seq_obj)
                 print entry
            except:
                 pass

The try/except is for showing the print statements. This outputs for some test data:

set([u'DXTMT', u'DIXTX', u'XSXDV', u'XI*MT'])
SELECT * FROM SeqObj WHERE seq='DXTMT'
SELECT * FROM SeqObj WHERE seq='DIXTX'
SELECT * FROM SeqObj WHERE seq='XSXDV'
SELECT * FROM SeqObj WHERE seq='XI*MT'

When I don’t put everything the try catch blocks, I get

IndexError: The query returned fewer than 1 results

I don’t know what’s going wrong because I the unique sequences are returned, the queries look like I expect them too, and I can verify that the entries are in the datastore. Any help is appreciated.

Answer 1

The problem is that you cannot filter on unindexed properties (TextProperty is unindexed). See https://developers.google.com/appengine/docs/python/datastore/queries#Filtering_on_Unindexed_Properties_Returns_No_Results for details.

Try using a StringProperty instead if your seq field is known to be less than 500 characters.

Also in these cases I always suggest to first run the query in the DataStore viewer. It suggests what indexes you need to run the desired query.