Question

I have some deeply randomly nested dictionary as follows.

{'CompilationStatistics': {'CodeGeneration': {'EndTime': '2010-04-21T14:03:11',
                                              'StartTime': '2010-04-21T14:03:11',
                                              'StepList': {'EliminatingDuplicates': {'EndTime': '2010-04-21T14:03:11',
                                                                                     'NumberOfFilesEliminated': '14',
                                                                                     'StartTime': '2010-04-21T14:03:11'},
                                                           'ModuleGenerator': {'EndTime': '2010-04-21T14:03:11',
                                                                               'StartTime': '2010-04-21T14:03:11'},
                                                           'Munger': {'EndTime': '2010-04-21T14:03:11',
...

How can I transform this into 1D dictionary as follows.

dict["CompilationStatistics_CodeGeneration_EndTime"] = '2010-04-21T14:03:11'
dict["CompilationStatistics_CodeGeneration_StartTime"] = '2010-04-21T14:03:11'
...
dict["CompilationStatistics_directory] = "/abc"

Answer 1

Simplest is to do it recursively:

import collections

def flattendict(d, prefix=()):
  r = {}
  for k, v in d.iteritems():
    pk = prefix + (k,)
    if isinstance(v, collections.Mapping):
      r.update(flattendict(v, pk))
    else:
      r['_'.join(pk)] = v
  return r

Here’s an example use:

d = {'foo': 'bar',
     'baz': {'fie': 'foo', 'zip': 'zap'},
     'bam': {'fie': 'foo', 'zip': {'zap': 'zup', 'mep': 'mop'}},
    }
print flattendict(d)

producing (possibly, of course, in different order)

{'baz_zip': 'zap', 'bam_fie': 'foo', 'foo': 'bar', 'bam_zip_mep': 'mop',
 'baz_fie': 'foo', 'bam_zip_zap': 'zup'}