Question

I am relatively new in python, was working on C a lot. Since I was seeing so many new functions in python that I do not know, I was wondering if there is a function that can request 10000 lines from a file in python.

Something like this is what I expect if that kind of function exist:

lines = get_10000_lines(file_pointer)

Did python have a build-in function or is there any module that I can download for this?
If not, how do I do this to be easiest way. I need to analyze a huge file so I want to read 10000 lines and analyze per time to save memory.

Thanks for helping!

Answer 1

from itertools import islice

with open(filename) as f:
    first10000 = islice(f, 10000)

This sets first10000 to an iterable object, i.e. you can loop over it with

for x in first10000:
    do_something_with(x)

If you need a list, do list(islice(f, 10000)) instead.

When the file contains less than 10k lines, this will just return all the lines in the file, with no padding (unlike the range-based solution). When reading a file in chunks, EOF is signaled by there being <10000 lines in the results:

with open(filename) as f:
    while True:
        next10k = list(islice(f, 10000))  # need list to do len, 3 lines down
        for ln in next10k:
            process(ln)
        if len(next10k) < 10000:
            break