Question

I am new python programmer,what I have understood so far,”yield” keyword returns an object in lieu of that the generator function returns only the generator object.

so if I have a list which contains 10K items,how I can make smart,pythonic solution without appending values in a list and make it large.

That means,I am appending some values to a list and finally a large list is created like below:

def example():
    final_list = []
    for i in range(0,10000):
        final_list.append(i)
    return final_list

This is just an example,not a real problem,I used range() just for generating loop nothing else,in my real problem,there are no sequential data,it will be random strings,and the findla list will contain 10K string.
so how can I get all the values without appending to a list in an efficient pythonic manner.

Thanks.

Answer 1

You said:

I am getting some string after crawling a web page and appending those
string to a list,as the crawler geting large data,the list is getting
bigger,finally when I want to iterate the list,its time and memory
consuming

So:

# generator function to crawl web pages
def crawler():
    while iStillHaveWebPagesToCrawl():
        someStrings = getSomeStringsFromAWebPage()
        for aString in someStrings:
            yield aString

def oneStringAtATime():
    for aString in crawler():
        doSomethingWith( aString )

When you call oneStringAtATime(), it sets up the generator function called crawler(); each time crawler() executes yield, the loop in oneStringAtATime() iterates once with that string. When crawler() runs out of web pages and exits the function, the oneStringAtATime() loop exits.