Question

I have a datastore thats has ~850 groups and ~19,000 items. Each item may belong to only one group, I have two models in my app that represent a Group and an Item:

class Group(db.Model):
    Id = db.IntegerProperty()
    Name = db.StringProperty()
    # ... some other  properties

class Item(db.Model):
    Id = db.IntegerProperty()
    Name = db.StringProperty()
    Group = db.ReferenceProperty(Group, collection_name="groupItems")
    # ... some other properties

I can use the datastore admin to view a specific item (i.e. WHERE Id = 34) and see that it is connected correctly to a Group –

SELECT * FROM Item WHERE Id = 34

This gives me a group with the following properties:

Decoded entity key: Group: id=10321
Entity key: agtzfmV2ZS1taW5lcnIMCxIFR3JvdXAY0VAM
Id: 18

If I alter my GQL query to retrieve all items for this Group I get no results! –

SELECT * FROM Item WHERE Group = KEY('agtzfmV2ZS1taW5lcnIMCxIFR3JvdXAY0VAM') -- No Results
SELECT * FROM Item WHERE Group = KEY('Group', 'agtzfmV2ZS1taW5lcnIMCxIFR3JvdXAY0VAM') -- No Results

If I retrieve just the group, it works as expected –

SELECT * FROM Group WHERE __key__ = KEY('agtzfmV2ZS1taW5lcnIMCxIFR3JvdXAY0VAM')  -- Returns 1 Group

This equally applies in my Python code. Calling:

group = Group.gql("WHERE Id = :1", 18).get()
items = Item.gql("WHERE Group = :1", group).fetch(50)

results in a list containing no items. Similarly

group.groupItems.fetch(500) -- Returns no results

My question is – am I doing something particularly stupid? I have created a dummy project with a similar structure to prove to myself that it wasnt a naming problem (i.e. that Group wasn’t a reserved word) and that returns just fine. (Attached if anyone is interested).

What am I doing wrong?

EDIT 1 :

As requested, here’s the creation code. Reader is a csv reader (using the inbuilt csv library) which opens a CSV stored in the BLOB store. For all intents and purposes its merely parsing a CSV file. As mentioned above. In the data viewer through my dashboard my Item is bound correctly to a group (a group is listed alongside the item and I can click through it’s link to view the group) however when passing the group as part of a filter to the Item no results are returned –

Group –

reader = UploaderBase().open_from_blobstore(Settings().get_group_csv_key())
upNum = 0
groupsToPut = []
for row in reader:
    group = Group(Id=int(row[0]))
    group.Name = row[2]
    groupsToPut.append(group)

db.put(groupsToPut)

Item –

groupCache = {}
for group in Group.all().fetch(1000):
    groupCache[group.Id] = group
logging.info("Cached %d group entries locally" % len(groupCache))

items = []
reader = UploaderBase().open_from_blobstore(Settings().get_items_csv_key())
upNum = 0
for row in reader:
    logging.debug("Adding row %d" % upNum)
    item = Item(Id=int(row[0]))

    if not row[1] is None and  row[1] != "":
        item.Group = groupCache[int(row[1])]

    item.Name = row[2]
    items.append(item)

db.put(items)

Answer 1

The most likely explanation is that when you inserted the data, the reference property was set as indexed=False. Changes to a model only affect entities written after the model was changed, so rows inserted while indexing was disabled for that column will not have index rows.