Question

Just curious, is there any difference between the two ways below for file reading? Eso. in terms of memory usage.

with open(...) as f:
    for line in f:
        <do something with line>

f=open(...)
  for line in f:
      #process line

Also I know for gzip file, the first one with ‘with’ cannot work.
thx

Answer 1

No, they’re quite identical, except that the first one makes sure the file is closed. The second does not. In other words, within the body of the with statement, f is just a file object which is exactly equivalent to the f object you get after just calling open in the second code snippet.

As you might know (and if you do not, make sure to read the informative doc), the with statement accepts an object that implements the context manager interface and invokes the __enter__ method of the object on entry, and its __exit__ method when it’s done (whether naturally, or with an exception.

Looking at the source code (Objects/fileobject.c), here’s the mapping (part of the file_methods structure) for these special methods:

{"__enter__", (PyCFunction)file_self,     METH_NOARGS,  enter_doc},
{"__exit__",  (PyCFunction)file_exit,     METH_VARARGS, exit_doc},

So, the file object’s __enter__ method just returns the file object itself:

static PyObject *
file_self(PyFileObject *f)
{
    if (f->f_fp == NULL)
        return err_closed();
    Py_INCREF(f);
    return (PyObject *)f;
}

While its __exit__ method closes the file:

static PyObject *
file_exit(PyObject *f, PyObject *args)
{
    PyObject *ret = PyObject_CallMethod(f, "close", NULL);
    if (!ret)
        /* If error occurred, pass through */
        return NULL;
    Py_DECREF(ret);
    /* We cannot return the result of close since a true
     * value will be interpreted as "yes, swallow the
     * exception if one was raised inside the with block". */
    Py_RETURN_NONE;
}