Question

I am getting information from a large xml file and I am using python with lxml target parsing interface to do it. I would like the possibility to set a limit after which parsing stops. Here is some code:

Parser target code:

class TitleTarget(object):
    def __init__(self,  limit=None):
        self.limit = limit
        self.counter = 0

    def start(self, tag, attrib):
        if self.limit and self.counter > self.limit:
            #### BREAK HERE ####
            return False
        #doProcessing(attrib)
        self.counter = self.counter + 1

    def end(self, tag):
        pass

    def data(self, data):
        pass

def close(self):
    pass

Code initiating the parsing:

parser = etree.XMLParser(target = TitleTarget(limit)) 
etree.parse(file, parser)  

I know that the processing goes to the “BREAK HERE” -line, but I haven’t found any method to stop the parsing. I have tried returning True, False, [], and raising Error, none seem to work. It always processes until the file ends.

Is there a way to stop processing bu using this method.

Answer 1

Instead of using etree.parse(file, parser), you can loop over the lines in file and call parser.feed on each line. This gives you control over when to break out of the loop.

Now you can just set a self.done=True in the target, and test for target.done in the feed loop:

---

import lxml.etree as ET
class HaltingTarget(object):
    def __init__(self, limit=None):
        self.done=False
        self.limit=limit
        self.counter=0
        self.result=[]
    def start(self, tag, attrib):
        if self.limit and self.counter>self.limit:
            self.done=True
            return
        if attrib:
            self.result.append(attrib)
        self.counter+=1
    def end(self, tag):
        pass
    def data(self, data):
        pass
    def comment(self, text):
        pass
    def close(self):
        return
def halt_parser():
    content='''\
        <node1>
          <Title a1="x1"> My Title </Title>
          <node2 a1="x2"> ... </node2>
          <node2 a1="x1"> ... </node2>
        </node1>
        '''
    target=HaltingTarget()
    parser=ET.XMLParser(target=target)
    for line in content.splitlines():
        parser.feed(line.strip())
        if target.done: break
    # We can't call parser.close() since the XML we've fed it is probably 
    # incomplete. We don't plan to use `parser` anymore, so delete it.
    del parser
    print(target.result)
    # [{'a1': u'x1'}, {'a1': u'x2'}, {'a1': u'x1'}]