Question

I have text with following strings:

{whatever}:::duplicateString:::{whatever}
{whatever}:::duplicateString:::{whatever}
....
{whatever}:::duplicateString:::{whatever}
{whatever}:::duplicateString:::{whatever}

How to remove duplicateString from text: the main idea is remove second word from line if it occurs more then one times.

First idea was to read them line-by-line and split by “:::” in order to create arrays and iterate over arrays with adding entries to TreeSet. Good. But how then to glue up lines again?

I can’t to remember any mechanism to figure out such task.. Language doesn’t matter, just approx solution?

Example text:

Appliances:::Main
Appliances:::Main:::Appliance Warranties
Appliances:::Main:::Beer Keg Refrigerators
Appliances:::Main:::Beverage Refrigerators
Appliances:::Main:::Ceiling Fans & Accessories
Appliances:::Main:::Ceiling Fans & Accessories:::Accessories
Appliances:::Main:::Ceiling Fans & Accessories:::Accessories:::Downrod Couplers
Appliances:::Main:::Ceiling Fans & Accessories:::Accessories:::Downrods
Appliances:::Main:::Ceiling Fans & Accessories:::Accessories:::Fan Replacement Blades

ideally it must be like:

Appliances:::Main
Appliances:::Appliance Warranties
Appliances:::Beer Keg Refrigerators
Appliances:::Beverage Refrigerators
Appliances:::Ceiling Fans & Accessories
Appliances:::Ceiling Fans & Accessories:::Accessories
Appliances:::Ceiling Fans & Accessories:::Accessories:::Downrod Couplers
Appliances:::Ceiling Fans & Accessories:::Accessories:::Downrods
Appliances:::Ceiling Fans & Accessories:::Accessories:::Fan Replacement Blades

Answer 1

If duplicateString possibly only occurs as the 2nd word, you could do (in Python):

lastWord = None
for line in open('file.txt'):
  w = line.split(':::')
  thisWord = w[1]
  if lastWord==w[1]:
    del w[1]
  lastWord = thisWord
  print ':::'.join(w)