Question

I have data in text file and I need to store it in database and now I little bit confused how to do it easier.

Here is an example of my data:

a: text1
b: text2
c: text3
blah blah not necessary text
a: text4
b: text5
c: text6
etc

So can you help me to parse this data.
I already done next code

import urllib2 as ur

def getPageData(url):
    return ur.urlopen(url).readlines()

checkList = ['a', 'b', 'c']

if __name__ == '__main__':
    textList = getPageData(url)
    res = []
    for i in textList:
        for y in checkList:
            if y in i:
                print i

I create a table in base something like

id | a varchar | b varchar | c varchar |

and I expect next result

id | a varchar | b varchar | c varchar |
1  | text1     | text2     | text3     |
2  | text4     | text5     | text6     |
n  | text      | text      | text      |

if read text file line by line how can I divide this text to logical blocks, for example I get a,b,c create dictionary with this data, and next when this block ends I append dictionary to list, and after that I have list of dict to store it to base. But I little bit confused how to create this list with this dictionary what I need to check and how to avoid unnecessary data in it? Is there more elegant way to do this?

Answer 1

I’d split on the : colon and test if the first part is in the set of allowed prefixes:

checkList = set(['a', 'b', 'c'])

for i in textList:
    check, rest = i.split(':', 1)
    if check.strip() not in checkList:
        continue
    data = rest.strip()
    # insert data into database; check is your column name.