Question

I would like to do something a bit tricky and I’m not sure the best way to go about it.

I have a two dimensional array that is in the form a nested list. Each “Row” in the list has the following structure:

['171000', 'Thing..Mega~Corporate~Thing..Mid~Dairy~Thing..Micro~Cheese', 'Cheese', '0.012174']

What I would like to do is loop through the entire array, row by row, and create a new row that splits the string in the second position into three new strings in the list so the result is as follows:

['171000', 'Mega~Corporate', 'Mid~Dairy', 'Micro~Cheese', 'Cheese', '0.012174']

A couple things I have to mention. In the above case, I split the string into three sub strings, Mega, Mid, Micro. Not every string will have a “Mid” and a “Micro” portion of it. Some of these strings could simply be shorter versions of the same format:

'Thing..Mega~Corporate'

For this case, I’d like to insert a placeholder to preserve the position of each element in the array (so that the numbers in the final position always have the same index).

The array I’m working with is rather large, upwards of 100k rows. Any help would be greatly appreciated as I’m really struggling with this one.

Answer 1

If Thing.. represents an arbitrary text (not a literal data in the string):

import re

def explode(s, keywords):
    for k in keywords:
        m = re.search(r'(%s~[^~]*)(?:~|$)' % (re.escape(k),), s)
        yield m and m.group(1)

for row in lst:
    row[1:2] = explode(row[1], "Mega Mid Micro".split())

Example

import re
from pprint import pprint

def explode(s, keywords):
    for k in keywords:
        m = re.search(r'(%s~[^~]*)(?:~|$)' % (re.escape(k),), s)
        yield m and m.group(1)


lst = [
 ['171000', 'Thing..Mega~Corporate~Thing..Mid~Dairy~Thing..Micro~Cheese', 'Cheese', '0.012174'],
 ['171000', 'Thing..Mega~Corporate', 'Cheese', '0.012174'],
]

print("Before:")
pprint(lst)

for row in lst:
    row[1:2] = explode(row[1], "Mega Mid Micro".split())

print("\nAfter:")
pprint(lst)

Output

Before:
[['171000',
  'Thing..Mega~Corporate~Thing..Mid~Dairy~Thing..Micro~Cheese',
  'Cheese',
  '0.012174'],
 ['171000', 'Thing..Mega~Corporate', 'Cheese', '0.012174']]

After:
[['171000',
  'Mega~Corporate',
  'Mid~Dairy',
  'Micro~Cheese',
  'Cheese',
  '0.012174'],
 ['171000', 'Mega~Corporate', None, None, 'Cheese', '0.012174']]