Question

I want to construct model with properties in form of lists.

class Data(db.Model):
    listOfIntergers = db.ListProperty(int)
    listOfStrings = db.ListProperty(str)

On both list I want to keep list of values in range <0, 255>.

What property type will take less space including index listOfIntergers or listOfStrings in data store?

Consider that str could be characters or encoded hex value i.e. 0 = 0, 255 = ff.

Answer 1

Be careful! First of all, integers are encoded using a variable-length encoding, so not all int64 values take up 8 bytes — small values take up 1-2 bytes. However, the representation of list properties repeats the property name for each value in the list. I rewrote your example using NDB (because the API for getting the encoded protobuf out of it is easier to remember) and found the following:

>>> from ndb import *
>>> class Data(Model):
...   listOfIntegers = IntegerProperty(repeated=True)
...   listOfStrings = StringProperty(repeated=True)
... 
>>> len(Data()._to_pb().Encode())
20
>>> len(Data(listOfIntegers=[0]*100)._to_pb().Encode())
2420
>>> len(Data(listOfIntegers=[255]*100)._to_pb().Encode())
2520
>>> len(Data(listOfStrings=['\x00']*100)._to_pb().Encode())
2420

But notice how deceptive: 100 integers of value 0 take 24 bytes per value (most of that is the property name ‘listOfIntegers’), but 100 integers of value 255 take 25 bytes per value! That’s the variable-length encoding for you. 100 strings of value ‘\x00’ take 24 bytes per value too — but notice that ‘listOfStrings’ is one character shorter than ‘listOfIntegers’ so the 1-byte string takes up 1 byte more than the integer 0, and the same amount of space as the integer 255.

Unless you really need this field to be indexed, the proper solution is probably to declare a BlobProperty (not a list/repeated property) and store a single string consisting of the desired number of bytes.

>>> class Data(Model):
...  b = BlobProperty()
... 
>>> d = Data(b=''.join(map(chr, range(100))))
>>> len(d._to_pb().Encode())
133
>>> 

That’s nearly 20x as compact for the same amount of information! (Admittedly you could get the list property version down a bit by using a shorter property name, but it would still be way longer.)