Question

In Python3 many methods are returning iterator or generator objects (instead of lists or other heavy objects in python2).

However, I’ve found that splitting string still returns list instead of generator or iteator :

~$ python3
Python 3.2.2
(...)
>>> type('a b c d'.split())
<class 'list'>

Is there buildin for splitting string using generator or iterator ?

(I know we can split it by ourself and write nice generator function. I am curious if there is something in standard library or language to do this)

Answer 1

Check out re.finditer from the re module => Python Docs

In brief:

“””
Returns an iterator yielding match objects over all non-overlapping matches for the RE pattern in string. The string is scanned left-to-right, and matches are returned in the order found. Empty matches are included in the result unless they touch the beginning of another match.
“””

I think it will do what you need. For example:

import re
text = "This is some nice text"
iter_matches = re.finditer(r'\w+', text)
for match in iter_matches:
    print(match.group(0))