Question

How do I filter a nested dictionary in python based on key values:

d = {'data': {'country': 'US', 'city': 'New York', 'state': None},
     'tags': ['US', 'New York'],
     'type': 'country_info',
     'growth_rate': None
     }

I want to filter this dictionary to eliminate NoneType values so the resulting dict should be:

d = {'data': {'country': 'US', 'city': 'New York'},
     'tags': ['US', 'New York'],
     'type': 'country_info',
     }

Also, the dict can have multiple levels of nesting. I want to remove all NoneType values from the dict.

Answer 1

You can define this recursively pretty easily with a dict comprehension.

def remove_keys_with_none_values(item):
    if not hasattr(item, 'items'):
        return item
    else:
        return {key: remove_keys_with_none_values(value) for key, value in item.items() if value is not None}

Recursion isn’t too optimised in Python, but given the relatively small number of nestings that are likely, I wouldn’t worry.

Looking before we leap isn’t too Pythonic, I think it is a better option than catching the exception – as it’s likely that the value will not be a dict most of the time (it is likely we have more leaves than branches).

Also note that in Python 2.x, you probably want to swap in iteritems() for items().