Question

I am trying to do some simple reporting in the datastore viewer in GAE. Using GQL I want to show just a few fields of a record. Is this possible?

How do I take entity with fields:

f1 f2 f3 f4 f5 f6

and show

f1 f3 f5 f6

Answer 1

This is not possible. From the GQL Reference documentation:

Every GQL query always begins with either SELECT * or SELECT __key__.

And from the Differences with SQL section of the datastore overview:

When querying the datastore, it is not currently possible to return
only a subset of kind properties. The App Engine datastore can either
return entire entities or only entity keys from a query.

---

As for why this kind of limitation exists, the article about How Entities and Indexes are Stored gave a good insight regarding the technical aspect behind Google’s Bigtable, the distributed database system powering App Engine’s datastore. (And other Google products)

From the article, datastore entities are stored in several different Bigtables. An Entity Bigtable stores the entire properties of the entity, and several Index Bigtables stores the entity key sorted according to indexes of the entity.

When we perform a query, basically there are two step that happen. The first step is our query is being executed against the Index Bigtables, producing a set of entity key that matches our query. The second step is that the set of keys is then used to fetch the whole entity from the Entity Bigtable.

Therefore, when you execute your query starting with SELECT __key__, the datastore only need to do the first step and immediately return with the set of keys. When you execute your query starting with SELECT *, the datastore did both steps and return with the set of entities.

Now, regarding why queries like SELECT f1, f3, f5, f6 is not supported by the datastore, we need to look into further detail on what happened during the second step stated above. From the article, it is stated that on the Entity Bigtable:

Instead of storing each entity property as an individual column in the corresponding Bigtable row, a single column is used which contains a binary-encoded protocol buffer containing the names and values for every property of a given entity.

Since the low level protocol buffer stores the entire entity’s properties as a single serialized data, it means querying only a subset of the entity’s property actually would take an extra post-processing step of filtering the result set and taking only the queried properties. This would entail a performance degradation of the datastore, and is probably why it is not supported by Google at the moment.