Question

From time to time I have to write some simple summarized reports using Django.

First I tried using Django ORM aggregates and interleaving results, but it can get a bit messy and I loose all the ORM laziness – just doesn’t feels right.

Lately I wrote a generic iterator class that can group/summarize a dataset. In the view it works like:

s_data = MyIterator(dataset, group_by='division', \
                                   sum_fields=[ 'sales', 'travel_expenses'])

In the template it works like:

{% for g, reg in s_data %}
    {% if g.group_changed %}
        <tr><!-- group summary inside the loop --> 
            <td colspan="5">{{ g.group }} Division</td>
            <td>{{ g.group_summary.sales }}</td>
            <td>{{ g.group_summary.travel_expenses }}</td>
        </tr>
    {% endif %}
    <tr><!-- detail report lines -->
        <td>{{ reg.field }}<td>
        <td>{{ reg.other_field_and_so_on }}<td>
        ...
    </tr>
{% endfor %}
<tr><!-- last group summary -->
    <td colspan="5">{{ s_data.group }} Division</td>
    <td>{{ s_data.group_summary.sales }}</td>
    <td>{{ s_data.group_summary.travel_expenses }}</td>
</tr>
<tr>
    <td colspan="5">Total</td>
    <td>{{ s_data.summary.sales }}</td>
    <td>{{ s_data.travel_expenses }}</td>
</tr>

I think it is a lot more elegant than my previous approach but having to repeat the code for the last group summary violates the DRY principle.

I had a look at “Geraldo reporting” but it was not “love at the first sight”.

Why there is no group/summary template tag, should I write one?

Answer 1

I figured it out, it can be done with iterators, no need for a template tag. The trick is to delay iteration one cycle.

class GroupSummaryIterator(object):
    """GroupSummaryIterator(iterable, group_by, field_list)

- Provides simple one level group/general summary for data iterables.
- Suports both key and object based records
- Assumes data is previously ordered by "group_by" property *before* use

Parameters:
===========
  iterable: iterable data
  group_by: property or key name to group by
field_list: list of fileds do sum

Example:
======
data =  [{'label': 'a', 'field_x': 1, 'field_c': 2},
         {'label': 'a', 'field_x': 3, 'field_c': 4},
         {'label': 'b', 'field_x': 1, 'field_c': 2},
         {'label': 'c', 'field_x': 5, 'field_c': 6},
         {'label': 'c', 'field_x': 1, 'field_c': 2}]

s = GroupSummaryIterator(data, 'label', ['field_x', 'field_c'])
for x,y in s:
     print y
     if x['group_changed']:
         print x['group'], 'summary:', x['group_summary']
print 'general summary:', s.summary

    """
    def __init__(self, iterable, group_by, field_list):
        self.iterable = iterable
        self.group_by = group_by
        self.field_list = field_list
    def _a(self, obj, key):
        """Get property or key value"""
        if isinstance(key, basestring) and hasattr(obj, key):
            return getattr(obj, key)
        try:
            return obj[key]
        except:
            return None
    def _sum(self, item):
        if self.group_changed:
            self.cur_summary = dict()
        for field in self.field_list:
            value = self._a(item, field)
            if not value:
                value = 0.0
            else:
                value = float(value)
            if self.summary.has_key(field):
                self.summary[field] += value
            else:
                self.summary[field] = value
            if self.cur_summary.has_key(field):
                self.cur_summary[field] += value
            else:
                self.cur_summary[field] = value
    def _retval(self, item, summary):
        """If each item from the source iterable is itself an iterable, merge
        everything so you can do "for summ, a, b in i" where you would have
        done "for a, b in i" without this object."""
        if isinstance(item, dict):
            retval = (item,)
        else:
            try:
                retval = tuple(item)
            except:
                retval = (item,)
        return (dict(group_changed=self.group_changed, group=self.group, group_summary=summary),) + retval
    def __iter__(self):
        self.cur_group = None
        self.group = None
        self.finished = False
        self.group_changed = False
        self.cur_item = None
        self.last_item = None
        self.summary = dict()
        self.group_summary = dict()
        self.cur_summary = dict()
        for item in self.iterable:
            self.group = self.cur_group
            self.group_summary = self.cur_summary
            self.cur_group = self._a(item, self.group_by)
            self.group_changed = self.group and self.cur_group != self.group
            self.last_item = self.cur_item
            self.cur_item = item
            self._sum(item)
            if self.last_item is None:
                continue
            yield self._retval(self.last_item, self.group_summary)
        if self.cur_item:
            self.group_changed = True
            yield self._retval(self.cur_item, self.group_summary)