Question

Im curious to know how lazy evaluation is implemented at higher levels, ie in libraries, etc. For example, how does the Django ORM or ActiveRecord defer evaluation of query until it is actually used?

Answer 1

Let’s have a look at some methods for django’s django.db.models.query.QuerySet class:

class QuerySet(object):
    """
    Represents a lazy database lookup for a set of objects.
    """
    def __init__(self, model=None, query=None, using=None):
        ...
        self._result_cache = None
        ...

     def __len__(self):
        if self._result_cache is None:
          ...
        elif self._iter:
          ...
        return len(self._result_cache)

    def __iter__(self):
        if self._result_cache is None:
          ...
        if self._iter:
          ...
        return iter(self._result_cache)

    def __nonzero__(self):
        if self._result_cache is not None:
          ...

    def __contains__(self, val):
        if self._result_cache is not None:
          ...
        else:
          ...
        ...

    def __getitem__(self, k):
        ...
        if self._result_cache is not None:
        ...
        ...

The pattern that these methods follow is that no queries are executed until some method that really needs to return some result is called. At that point, the result is stored in self._result_cache and any subsequent call to the same method returns the cached value.