Question

The following class is implemented to provide a generic object that can be passed through network as a json-encoded dictionary. I’m actually trying to json encode a dict (!) but it won’t work.

I know it will work with custom encoder class, but I don’t see why it’s necessary when I’m just encoding a dict.

Can someone explain the TypeError or offer a way to encode this without subclassing JSONEncoder?

Here is the bad behavior.

>>> def tree(): return CustomDict(tree)
>>> d = tree()
>>> d['one']['test']['four'] = 19
>>> d.dict
{ 'one' : { 'test': {'four': 19}}}
>>> type(d.dict)
<type 'dict'> 
>>> import json
>>> json.dumps(d.dict)
# stacktrace removed
TypeError: {'one': {'test': {'four': 19}}} is not JSON serializable
>>> normal_d = {'one': {'test': {'four': 19}}}
>>> type(normal_d)
<type 'dict'>
>>> json.dumps(normal_d)
"{'one': {'test': {'four': 19}}}"
>>> normal_d == d
True

I would love to be able to do the following

>>>> json.dumps(dict(d))
"{'one': {'test': {'four': 19}}}"

but I added the dict property to ‘force it’ (didn’t work obviously). Now it’s an even bigger mystery. Here is the code for the CustomDict class

class CustomDict(collections.MutableMapping):                                
    """                                                                         
    A defaultdict-like object that can also have properties and special methods 
    """                                                                         

    def __init__(self, default_type=str, *args,  **kwargs):                     
        """                                                                     
        instantiate as a default-dict (str if type not provided). Try to update 
        self with each arg, and then update self with kwargs.                                                                

        @param default_type: the type of the default dict                       
        @type default_type: type (or class)                                     
        """                                                                     
        self._type = default_type                                               
        self._store = collections.defaultdict(default_type)                     
        self._dict = {}                                                         

        for arg in args:                                                        
            if isinstance(arg, collections.MutableMapping):                     
                self.update(arg)                                                

        self.update(kwargs)                                                     

    @property                                                                   
    def dict(self):                                                             
        return self._dict                                                       

    def __contains__(self, key):                                                
        return key in self._store                                               

    def __len__(self):                                                          
        return len(self._store)                                                 

    def __iter__(self):                                                         
        return iter(self._store)                                                

    def __getitem__(self, key):                                                 
        self._dict[key] = self._store[key]                                      
        return self._store[key]                                                 

    def __setitem__(self, key, val):                                            
        self._dict[key] = val                                                   
        self._store[key] = val                                                  

    def __delitem__(self, key):                                                 
        del self._store[key]                                                    

    def __str__(self):                                                          
        return str(dict(self._store))   

Answer 1

You want to make your type a subclass of dict, not of collections.MutableMapping, really.

Even better still, use collections.defaultdict directly instead, it already is a subclass of dict and can be used to implement your tree ‘type’ easily:

from collections import defaultdict

def Tree():
    return defaultdict(Tree)

tree = Tree()

Demonstration:

>>> from collections import defaultdict
>>> def Tree():
...     return defaultdict(Tree)
... 
>>> tree = Tree()
>>> tree['one']['two'] = 'foobar'
>>> tree
defaultdict(<function Tree at 0x107f40e60>, {'one': defaultdict(<function Tree at 0x107f40e60>, {'two': 'foobar'})})
>>> import json
>>> json.dumps(tree)
'{"one": {"two": "foobar"}}'

If you must add your own methods and behaviour, then I’d subclass defaultdict and build upon that base:

class CustomDict(defaultdict):
    pass

As this is still a subclass of dict, the json library will happily convert that to a JSON object without special handling.