Question

Consider this short python list of dictionaries (first dictionary item is a string, second item is a Widget object):

raw_results =  
     [{'src': 'tag', 'widget': <Widget: to complete a form today>},   # dupe 1a
      {'src': 'tag', 'widget': <Widget: a newspaper>},                # dupe 2a
      {'src': 'zip', 'widget': <Widget: to complete a form today>},   # dupe 1b
      {'src': 'zip', 'widget': <Widget: the new Jack Johnson album>},
      {'src': 'zip', 'widget': <Widget: a newspaper>},                # dupe 2b
      {'src': 'zip', 'widget': <Widget: premium dog food >}]

I want to go through that list and remove the duplicates, which this SO question answered for me:

Remove duplicates in a list while keeping its order (Python)

    known_widgets= set()
    processed_results = []

    for x in raw_results:
        widget = x['widget']
        if widget in known_widgets: 
            continue
        else:
            processed_results.append(x)
            known_widgets.add(widget)

However, after I remove the duplicate row (e.g. dupe 1b), I want to change the remaining duplicate’s (e.g. dupe 1a) “src” data. I would like to append the removed duplicates “src” to the original. This is what I’d like to end up with:

processed_results =  
    [{'src': 'tag-zip', 'widget': <Widget: to complete a form today>},  # dupe 1a
     {'src': 'tag-zip', 'widget': <Widget: a newspaper>},               # dupe 2a
     {'src': 'zip', 'widget': <Widget: the new Jack Johnson album>},
     {'src': 'zip', 'widget': <Widget: premium dog food >}]

I’m sure this is easy to do, but my head is spinning after too much coffee and many hours circling this problem. I’d love and really appreciate the help of an expert. Thank you!

Answer 1

def find_widget(widget, L):
    for i, v in enumerate(L):
      if v[widget] == widget:
          return i

known_widgets= set()
processed_results = []

for x in raw_results:
    widget = x['widget']
    if widget in known_widgets:
        processed_widgets[find_widget(widget, processed_results)]['src'] += '-%s' % x['tag']
        continue
    else:
        processed_results.append(x)
        known_widgets.add(widget)

Could probably be done better (as this is two passes for each duplicate widget).