Question

Using Google app engine, is it possible to initialize a globally accessible singleton on app startup? I have a large static tree structure that I need to use on every request and want to initialize it beforehand. The tree structure is too large (20+MB) to be put into Memcache and I am trying to figure out what other alternatives I have.

EDIT: Just to add some clarity based on the answers I’ve received so far. I’m loading a dictionary of words into a trie/prefix tree structure. The trie is immutable as the dictionary of words is fixed. I’m generating anagrams based on an input string of characters, so one request may access a fair amount of the trie on a single request, possibly more the 1MB, however I’m not certain.

Here’s is the python structure that I’m loading the dictionary of words into as well.

class Node(object):

    def __init__(self, letter='', final=False):
        self.letter = letter
        self.final = final
        self.children = {}

    def add(self, letters):
        node = self
        for index, letter in enumerate(letters):
            if letter not in node.children:
                node.children[letter] = Node(letter, index==len(letters)-1)
            node = node.children[letter]

Answer 1

How much of this tree do you need to access on a single request? In what manner do you query it? Does it ever change?

If it’s immutable, you don’t really need a ‘singleton’ – which implies mutability – just a way to access the data on each instance. Depending on how you need to access it, you could store it as a data file, a blob, or as data in the datastore.