Question

Am exporting data to csv. after 25000 records , memory exhausted.
Memory limit increasing is ok.

If i have 100000 rows, can i write it as 4 process.
write first 25000 rows, then next 25000 then next…

Is this possible in csv export?
Will this have any advantage? Or this is same exporting whole data?

Any multiple processing or parallel processing have some advantage?

Answer 1

Well, this depends on how you’re generating the CSV.

Assuming that you’re doing it as the result of a database query (or some other import), you could try streaming instead of building and then returning.

Basically, you turn off output buffering first:

while(ob_get_level() > 0) {
    ob_end_flush();
}

Then, when you’re building it, echo it out row by row:

foreach ($rows as $row) {
    echo '"'.$row[0].'","'.$row[1].'"'."\n";
}

That way, you’re not using too much memory in PHP.

You could also write the data to a temporary file, and then stream that file back:

$file = tmpfile();
foreach ($rows as $row) {
    fputcsv($file, $row);
}
rewind($file);
fpassthru($file);  // Sends the file to the client
fclose($file);

But again, it all depends on what you’re doing. It sounds to me like you’re building the CSV in a string (which is eating all your memory). That’s why I suggested these two options…